JPPF, java, parallel computing, distributed computing, grid computing, parallel, distributed, cluster, grid, cloud, open source, android, .net
JPPF, java, parallel computing, distributed computing, grid computing, parallel, distributed, cluster, grid, cloud, open source, android, .net
JPPF

The open source
grid computing
solution

 Home   About   Features   Download   Documentation   Forums 

Load-balancer state persistence

From JPPF 6.0 Documentation

Jump to: navigation, search

Contents

Main Page > Load Balancing > State persistence


As seen previoulsly, most of the existing load-balancing algorithms are adaptive and often go through a convergence phase before reaching a state of optimal effciency. Unfortunately, this state disappears whenever a node, driver or client (depending on where load-balancing applies) is stopped.

As of JPPF 6.0, it is now possible to persist the state of load-balancer instances and restore them whenever a node reconnects to a driver (server-side load-balancing) or when a client reconnects to a driver (client-side load-balancing).

As for many other JPPF features, this state persistence facility is pluggable, which means it is possible to create user-defined implemntations. The full details of how to do this are provided in the Custom load-balancer state persistence section.

1 Identification of persisted load-balancer states

Persistence and restoration of a load-balancer state requires an identifier that is both unique and repeatable accross reconnection and restart of the peers involved (node + driver or driver + client). Using combinations of the components' UUIDs will not work here, since UUIDs only exist as long as each component's JVM is alive and are generated anew at each restart.

Instead, JPPF computes a resilient identifier which includes the IP address of both parties, the related driver's port and other properties found in the JPPF component's configuration. A load-balancer state is further idenfied by the name of the of its algorithm. For normalization purposes, both parts of the identfier are transformed using a hash function. This function defaults to SHA-1 and can be changed using the following configuration property:

jppf.load.balancing.persistence.hash = SHA-1

2 Configuring

A load-balancer state persistence facility is essentially a pluggable implementation of the LoadBalancerPersistence interface. A driver or JPPFClient instance supports a single persistence implementation at a time, configured via the "jppf.load.balancing.persistence" configuration property:

jppf.load.balancing.persistence = <implementation class name> param1 ... paramN

where:

  • "implementation class name" is the fully qualified class name of the LoadBalancerPersistence implementation
  • "param1 ... paramN" are optional string parameters used by the persistence implementation


For example, to configure the built-in file persistence with a root directory named "lb_persistence" in the driver's working directory, we would configure the following:

pkg = org.jppf.load.balancer.persistence
jppf.load.balancing.persistence = ${pkg}.FileLoadBalancerPersistence lb_persistence

If no load-balancer persistence is configured, then persitence of load-balancer states is disabled.

3 Built-in implementations

3.1 File-driven persistence

File-driven persistence of load-balancer states is imlmented in the class FileLoadBalancerPersistence. It relies on a directory structure like this:

 persistence_root
 |_channel_identifier1
 | |_algorithm1.data
 | |_...
 | |_algorithmP1.data
 |_...
 |_channel_identifiern
   |_algorithm1.data
   |_...
   |_algorithmPn.data

Where:

  • Each channel_identifieri represents a hash of a string concatenated from various properties of the channel. A channel represents a connection between a node and a driver for server-side load-balancing, or between a driver and a client for client-side load-balancing. This id is unique for each channel and resilient over restarts of both related peers, contrary to their uuids, which are recreated each time a component starts. Using a hash also ensures that it can be used a s a valid folder name in a file system
  • Each algorithmi prefix is the hash of the related load-balancing algorithm name. Again, this ensures it can be used to form a valid file name
  • Each algorithmi.data file represents the serialized state of the related load-balancer


The file-driven persistence is configured as:

pkg = org.jppf.load.balancer.persistence
jppf.load.balancing.persistence = ${pkg}.FileLoadBalancerPersistence <root_dir>

where root_dir can be either an absolute file path, or a path relative to the JVM's current working directory. If it is omitted, it defaults to "lb_persistence".

3.2 Database-driven persistence

Database-driven persistence is implemented in the class DatabaseLoadBalancerPersistence and stores load-balancer states in a single database table. The table has the following structure:

CREATE TABLE <table_name> (
  NODEID varchar(250) NOT NULL,
  ALGORITHMID varchar(250) NOT NULL,
  STATE blob NOT NULL,
  PRIMARY KEY (NODEID, ALGORITHMID)
);

Where:

  • the NODEID column represents a hash of a string concatenated from various properties of the node. This id is unique for each node and resilient over node restarts, contrary to the node uuid, which is recreated each time a node starts
  • the ALGORITHMID column is a hash of the load-balancer's algorithm name.
  • the STATE column represents the serialized state of the load-balancer, such as provided by PersistentState.getState()


Very important: the table definition should be adjusted depending on the database you are using. For instance, in MySQL the BLOB type has a size limit of 64 KB, thus storing load-balancer states larger than this size will always fail. In this use case, the MEDIUMBLOB or LONGBLOB type should be used instead.


The database-driven persistence is configured as follows:

pkg = org.jppf.load.balancer.persistence
jppf.load.balancing.persistence = ${pkg}.DatabaseLoadBalancerPersistence <table_name> <datasource_name>

where:

  • table_name is the name of the table in which to store load-balancer states
  • datasource_name is the name of a data source defined elsewhere in the configuration. See the database services section of the documentation for details.


If both are unspecified, they will default to "load_balancer" and "loadBalancerDS", respectively. If only the table name is specified, the data source name defaults to "loadBalancerDS". If the table does not exist, JPPF will attempt to create it. If this fails for any reason, for instance if the user does not have sufficient privileges, then persistence will be disabled.

Here is a full example configuration:

# persistence definition
pkg = org.jppf.load.balancer.persistence
jppf.load.balancing.persistence = ${pkg}.DatabaseLoadBalancerPersistence MY_TABLE loadBalancerDS

# datasource definition
jppf.datasource.lb.name = loadBalancerDS
jppf.datasource.lb.driverClassName = com.mysql.jdbc.Driver
jppf.datasource.lb.jdbcUrl = jdbc:mysql://localhost:3306/testjppf
jppf.datasource.lb.username = testjppf
jppf.datasource.lb.password = testjppf
jppf.datasource.lb.minimumIdle = 5
jppf.datasource.lb.maximumPoolSize = 10
jppf.datasource.lb.connectionTimeout = 30000
jppf.datasource.lb.idleTimeout = 600000

3.3 Asynchronous persistence

Asynchronous persistence is an asynchronous wrapper for any other load-balancer persistence implementation. The methods of LoadBalancerPersistence that do not return a result (void return type) are non-blocking and return immediately. All other methods will block until they are executed and their result is available.

The execution of the interface's methods is delegated to a thread pool, whose size can be defined in the configuration or defaults to 1.

This asynchronous persistence can be configured in two forms:

# shorten the configuration value for clarity
async = org.jppf.load.balancer.persistence.AsynchronousLoadBalancerPersistence
# asynchronous persistence with default thread pool size
jppf.load.balancing.persistence = ${async} <actual_persistence> <param1> ... <paramN>
# asynchronous persistence with a specified thread pool size
jppf.load.balancer.persistence = ${async} <pool_size> <actual_persistence> <param1> ... <paramN>

Here is an example configuration for an asynchronous database persistence:

pkg = org.jppf.load.balancer.persistence
# asynchronous database persistence with pool of 4 threads,
# a table named 'JPPF_TEST' and datasource named 'loadBalancerDS'
jppf.load.balancing.persistence = ${pkg}.AsynchronousLoadBalancerPersistence 4 ${pkg}.DatabaseLoadBalancerPersistence JPPF_TEST loadBalancerDS

Note: the asynchronous persistence relies, for storage operations, on a queue where elements are uniquely indexed on a (channelID, alogrithmID) key. If or when load-balancer states are put in the queue faster than they are persisted, the queue will not grow, because the persistence implementation always replaces entries with an existing key. In other words, the queue size is always bounded by number_of_channels * number_of_algorithms. The implication is that you should never see an out of memory condition caused by the asynchronous persistence's footprint, provided the JVM's heap is properly sized.

4 Managing the persistence store

4.1 LoadBalancerPersistenceManager interface

Managing the load-balancer persistence store is done via the LoadBalancerPersistenceManagement interface, which is defined ad follows:

public interface LoadBalancerPersistenceManagement {
  // Whether load-balancer persistence is enabled
  boolean isPersistenceEnabled();
  // List all the channels that have an entry in the persistence store
  List<String> listAllChannels() throws LoadBalancerPersistenceException;
  // List all algorithms for which a channel has an entry in the persistence store
  List<String> listAlgorithms(String channelID) throws LoadBalancerPersistenceException;
  // List all channels that have an entry for the specified algorithm
  List<String> listAllChannelsWithAlgorithm(String algorithm) throws LoadBalancerPersistenceException;
  // Determine whether a channel has an entry for the specified algorithm
  boolean hasAlgorithm(String channelID, String algorithm) throws LoadBalancerPersistenceException;
  // Delete all entries in the persistece store
  void deleteAll() throws LoadBalancerPersistenceException;
  // Delete all entries for the specified channel
  void deleteChannel(String channelID) throws LoadBalancerPersistenceException;
  // Delete the specified algorithm state from all the channels that have it
  void deleteAlgorithm(String algorithm) throws LoadBalancerPersistenceException;
  // Delete the specified algorithm state from the specified channel
  void delete(String channelID, String algorithm) throws LoadBalancerPersistenceException;
}

This interface is persistence implementation-agnostic and is designed to work the same way no matter which persistence is configured.

Note: all "algorithm" parameters used in the methods of LoadBalancerPersistenceManagement are the real algorithm names, such as "proportional", "rl2", "autotuned",

Here is an example usage:

LoadBalancerPersistenceManagement mgt = ...;
try {
  // list all nodes with an entry in the persistence store
  List<String> nodeIDs = mgt.listAllChannels();
  for (String nodeID: nodeIDs) {
    // if the node has an entry for the "proportional" algorithm, delete it
    if (mgt.hasAlgorithm("proportional")) {
      mgt.delete(nodeID, "proportional");
    }
  }
  // ensure there is no longer any entry for the "proportional" algorithm
  assert mgt.listAllChannelsWithAlgorithm("proportional").isEmpty();
} catch (LoadBalancerPersistenceException lbpe) {
  lbpe.printStackTrace();
}

4.2 Managing client-side load-balancing persistence

For client-side load-balancing, the persistence store management interface can be obtained directly via the JPPFClient's getLoadBalancerPersistenceManagement() method, for example:

JPPFClient client = new JPPFClient();
LoadBalancerPersistenceManagement mgt = client.getLoadBalancerPersistenceManagement();
// list all driver connections with an entry in the persistence store
List<String> channelIDs = mgt.listAllChannels();
// do something with the list ...

4.3 Managing driver-side load-balancing persistence

To manage its load-balancer persistence store, the JPPF driver provides a management MBean that can be used via standard JMX APIs. The MBean interface is LoadBalancerPersistenceManagerMBean, defined as follows:

public interface LoadBalancerPersistenceManagerMBean extends LoadBalancerPersistenceManagement {
  // The object name under which this MBean is registered with the MBean server
  String MBEAN_NAME = "org.jppf:name=loadBalancerPersistenceManager,type=driver";
}

As we can see, it merely extends LoadBalancerPersistenceManagement, adding a constant for its object name. The JPPF driver management APIs provide an easy way to access it vith the JMXDriverConnectionWrapper class:

via a JPPF client:

JPPFClient client = new JPPFClient();
JMXDriverConnectionWrapper jmx = client.awaitWorkingConnectionPool.awaitWorkingJMXConnection();
LoadBalancerPersistenceManagement mgt = jmx.getLoadBalancerPersistenceManagement();

or via direct instantiation:

JMXDriverConnectionWrapper jmx = new JMXDriverConnectionWrapper("www.myhost.com", 11098, false);
LoadBalancerPersistenceManagement mgt = jmx.getLoadBalancerPersistenceManagement();

This MBean is also directly usable via JMX-based tools such as JConsole or VisualVM:

MBeansJConsole.gif

Main Page > Load Balancing > State persistence

JPPF Copyright © 2005-2017 JPPF.org Powered by MediaWiki