JPPF Issue Tracker
star_faded.png
Please log in to bookmark issues
feature_request_small.png
OPEN  Feature request JPPF-39  -  Load balancing with neural nets/genetic algorithms
Posted Aug 08, 2012 - updated Feb 09, 2020
action_vote_minus_faded.png
0
Votes
action_vote_plus_faded.png
lolo4j (lolocohen) has been working on this issue since September 29, 2017 (08:28)
Issue details
  • Type of issue
    Feature request
  • Status
     
    New
  • Assigned to
     lolo4j
  • Type of bug
    Not triaged
  • Likelihood
    Not triaged
  • Effect
    Not triaged
  • Posted by
     lolo4j
  • Owned by
    Not owned by anyone
  • Category
    Server
  • Resolution
    Not determined
  • Priority
    Normal
  • Targetted for
    icon_milestones.png JPPF 6.3
Issue description
From this SF.net forum thread (replies 5 and 6):

The idea is to use machine learning techniques such as neural nets and/or genetic algorithms to implement an efficient load balancing that could learn from the data patterns fed to the JPPF grid, and "replay" the laod balancing parameters associated witht he patterns.

#3
Comment posted by
 lolo4j
Sep 29, 08:28
I've started exploring how to use GAs as as the base for a load-balancer algorithm. So far, my feeling is that GAs are a poor fit for this, unless I'm missing stuff that would be obvious to people more knowledgeable than I am - and that would be a lot of people. What makes it a poor fit? I believe it is the dimensionality of the problem to solve. JPPF load-balancing assumes that overall performance is a direct function of how many tasks of job are dispatched to each node aka "bundle size". That's just one dimension. How do you encode that as a chromosome with a set of genes? That's an open question, and I'll take all the suggestions and help I can get. As of now, I think this problem is better solved using other techniques like monte-carlo alogrithms or reinforcment learning ("rl2" algorithm).

Instead, I'm currently exploring the use of GAs to optimize the set of parameters of an existing load-balancing algorithm. The idea is to encode the parameters as genes in a chromosome and perform the usual genetic operations on a population of chromosomes (selection, crossover, mutation, etc.). Fitness is computed as the total time it takes to execute a set of jobs in a given topology, where the server is configured with the load-balancing algorithm and its parameters encoded in each chromosome. My first results are interesting and promising, I'm even discovering features of the load-balancers I had overlooked or just not thought of ;)

Anyway, this is a fascinating area of research, so let's have fun with it.