Load-balancer state persistence

As seen previoulsly, most of the existing load-balancing algorithms are adaptive and often go through a convergence phase before reaching a state of optimal effciency. Unfortunately, this state disappears whenever a node, driver or client (depending on where load-balancing applies) is stopped.

As of JPPF 6.0, it is now possible to persist the state of load-balancer instances and restore them whenever a node reconnects to a driver (server-side load-balancing) or when a client reconnects to a driver (client-side load-balancing).

As for many other JPPF features, this state persistence facility is pluggable, which means it is possible to create user-defined implemntations. The full details of how to do this are provided in the Custom load-balancer state persistence section.

1 Identification of persisted load-balancer states

Persistence and restoration of a load-balancer state requires an identifier that is both unique and repeatable accross reconnection and restart of the peers involved (node + driver or driver + client). Using combinations of the components' UUIDs will not work here, since UUIDs only exist as long as each component's JVM is alive and are generated anew at each restart.

Instead, JPPF computes a resilient identifier which includes the IP address of both parties, the related driver's port and other properties found in the JPPF component's configuration. A load-balancer state is further idenfied by the name of the of its algorithm. For normalization purposes, both parts of the identfier are transformed using a hash function. This function defaults to SHA-1 and can be changed using the following configuration property:

jppf.load.balancing.persistence.hash = SHA-1

2 Configuring

A load-balancer state persistence facility is essentially a pluggable implementation of the LoadBalancerPersistence interface. A driver or JPPFClient instance supports a single persistence implementation at a time, configured via the "jppf.load.balancing.persistence" configuration property:

jppf.load.balancing.persistence = <implementation class name> param1 ... paramN

where:

• "implementation class name" is the fully qualified class name of the LoadBalancerPersistence implementation
• "param1 ... paramN" are optional string parameters used by the persistence implementation

For example, to configure the built-in file persistence with a root directory named "lb_persistence" in the driver's working directory, we would configure the following:

pkg = org.jppf.load.balancer.persistence
jppf.load.balancing.persistence = ${pkg}.FileLoadBalancerPersistence lb_persistence

If no load-balancer persistence is configured, then persitence of load-balancer states is disabled.

3 Built-in implementations

3.1 File-driven persistence

File-driven persistence of load-balancer states is imlmented in the class FileLoadBalancerPersistence. It relies on a directory structure like this:

 persistence_root
 |_channel_identifier1
 | |_algorithm1.data
 | |_...
 | |_algorithmP1.data
 |_...
 |_channel_identifiern
   |_algorithm1.data
   |_...
   |_algorithmPn.data

Where:

• Each channel_identifier_i represents a hash of a string concatenated from various properties of the channel. A channel represents a connection between a node and a driver for server-side load-balancing, or between a driver and a client for client-side load-balancing. This id is unique for each channel and resilient over restarts of both related peers, contrary to their uuids, which are recreated each time a component starts. Using a hash also ensures that it can be used a s a valid folder name in a file system
• Each algorithm_i prefix is the hash of the related load-balancing algorithm name. Again, this ensures it can be used to form a valid file name
• Each algorithm_i.data file represents the serialized state of the related load-balancer

The file-driven persistence is configured as:

pkg = org.jppf.load.balancer.persistence
jppf.load.balancing.persistence = ${pkg}.FileLoadBalancerPersistence <root_dir>

where root_dir can be either an absolute file path, or a path relative to the JVM's current working directory. If it is omitted, it defaults to "lb_persistence".

3.2 Database-driven persistence

Database-driven persistence is implemented in the class DatabaseLoadBalancerPersistence and stores load-balancer states in a single database table. The table has the following structure:

CREATE TABLE <table_name> (
  NODEID varchar(250) NOT NULL,
  ALGORITHMID varchar(250) NOT NULL,
  STATE blob NOT NULL,
  PRIMARY KEY (NODEID, ALGORITHMID)
);

Where:

• the NODEID column represents a hash of a string concatenated from various properties of the node. This id is unique for each node and resilient over node restarts, contrary to the node uuid, which is recreated each time a node starts
• the ALGORITHMID column is a hash of the load-balancer's algorithm name.
• the STATE column represents the serialized state of the load-balancer, such as provided by PersistentState.getState()

The database-driven persistence is configured as follows:

pkg = org.jppf.load.balancer.persistence
jppf.load.balancing.persistence = ${pkg}.DatabaseLoadBalancerPersistence <table_name> <datasource_name>

where:

• table_name is the name of the table in which to store load-balancer states
• datasource_name is the name of a data source defined elsewhere in the configuration. See the database services section of the documentation for details.

If both are unspecified, they will default to "load_balancer" and "loadBalancerDS", respectively. If only the table name is specified, the data source name defaults to "loadBalancerDS". If the table does not exist, JPPF will attempt to create it. If this fails for any reason, for instance if the user does not have sufficient privileges, then persistence will be disabled.

Here is a full example configuration:

# persistence definition
pkg = org.jppf.load.balancer.persistence
jppf.load.balancing.persistence = ${pkg}.DatabaseLoadBalancerPersistence MY_TABLE loadBalancerDS

# datasource definition
jppf.datasource.lb.name = loadBalancerDS
jppf.datasource.lb.driverClassName = com.mysql.jdbc.Driver
jppf.datasource.lb.jdbcUrl = jdbc:mysql://localhost:3306/testjppf
jppf.datasource.lb.username = testjppf
jppf.datasource.lb.password = testjppf
jppf.datasource.lb.minimumIdle = 5
jppf.datasource.lb.maximumPoolSize = 10
jppf.datasource.lb.connectionTimeout = 30000
jppf.datasource.lb.idleTimeout = 600000

3.3 Asynchronous persistence

Asynchronous persistence is an asynchronous wrapper for any other load-balancer persistence implementation. The methods of LoadBalancerPersistence that do not return a result (void return type) are non-blocking and return immediately. All other methods will block until they are executed and their result is available.

The execution of the interface's methods is delegated to a thread pool, whose size can be defined in the configuration or defaults to 1.

This asynchronous persistence can be configured in two forms:

# shorten the configuration value for clarity
async = org.jppf.load.balancer.persistence.AsynchronousLoadBalancerPersistence
# asynchronous persistence with default thread pool size
jppf.load.balancing.persistence = ${async} <actual_persistence> <param1> ... <paramN>
# asynchronous persistence with a specified thread pool size
jppf.load.balancer.persistence = ${async} <pool_size> <actual_persistence> <param1> ... <paramN>

Here is an example configuration for an asynchronous database persistence:

pkg = org.jppf.load.balancer.persistence
# asynchronous database persistence with pool of 4 threads,
# a table named 'JPPF_TEST' and datasource named 'loadBalancerDS'
jppf.load.balancing.persistence = ${pkg}.AsynchronousLoadBalancerPersistence 4 ${pkg}.DatabaseLoadBalancerPersistence JPPF_TEST loadBalancerDS

4 Managing the persistence store

4.1 LoadBalancerPersistenceManager interface

Managing the load-balancer persistence store is done via the LoadBalancerPersistenceManagement interface, which is defined ad follows:

public interface LoadBalancerPersistenceManagement {
  // Whether load-balancer persistence is enabled
  boolean isPersistenceEnabled();
  // List all the channels that have an entry in the persistence store
  List<String> listAllChannels() throws LoadBalancerPersistenceException;
  // List all algorithms for which a channel has an entry in the persistence store
  List<String> listAlgorithms(String channelID) throws LoadBalancerPersistenceException;
  // List all channels that have an entry for the specified algorithm
  List<String> listAllChannelsWithAlgorithm(String algorithm) throws LoadBalancerPersistenceException;
  // Determine whether a channel has an entry for the specified algorithm
  boolean hasAlgorithm(String channelID, String algorithm) throws LoadBalancerPersistenceException;
  // Delete all entries in the persistece store
  void deleteAll() throws LoadBalancerPersistenceException;
  // Delete all entries for the specified channel
  void deleteChannel(String channelID) throws LoadBalancerPersistenceException;
  // Delete the specified algorithm state from all the channels that have it
  void deleteAlgorithm(String algorithm) throws LoadBalancerPersistenceException;
  // Delete the specified algorithm state from the specified channel
  void delete(String channelID, String algorithm) throws LoadBalancerPersistenceException;
}

This interface is persistence implementation-agnostic and is designed to work the same way no matter which persistence is configured.

Here is an example usage:

LoadBalancerPersistenceManagement mgt = ...;
try {
  // list all nodes with an entry in the persistence store
  List<String> nodeIDs = mgt.listAllChannels();
  for (String nodeID: nodeIDs) {
    // if the node has an entry for the "proportional" algorithm, delete it
    if (mgt.hasAlgorithm("proportional")) {
      mgt.delete(nodeID, "proportional");
    }
  }
  // ensure there is no longer any entry for the "proportional" algorithm
  assert mgt.listAllChannelsWithAlgorithm("proportional").isEmpty();
} catch (LoadBalancerPersistenceException lbpe) {
  lbpe.printStackTrace();
}

4.2 Managing client-side load-balancing persistence

For client-side load-balancing, the persistence store management interface can be obtained directly via the JPPFClient's getLoadBalancerPersistenceManagement() method, for example:

JPPFClient client = new JPPFClient();
LoadBalancerPersistenceManagement mgt = client.getLoadBalancerPersistenceManagement();
// list all driver connections with an entry in the persistence store
List<String> channelIDs = mgt.listAllChannels();
// do something with the list ...

4.3 Managing driver-side load-balancing persistence

To manage its load-balancer persistence store, the JPPF driver provides a management MBean that can be used via standard JMX APIs. The MBean interface is LoadBalancerPersistenceManagerMBean, defined as follows:

public interface LoadBalancerPersistenceManagerMBean extends LoadBalancerPersistenceManagement {
  // The object name under which this MBean is registered with the MBean server
  String MBEAN_NAME = "org.jppf:name=loadBalancerPersistenceManager,type=driver";
}

As we can see, it merely extends LoadBalancerPersistenceManagement, adding a constant for its object name. The JPPF driver management APIs provide an easy way to access it vith the JMXDriverConnectionWrapper class:

via a JPPF client:

JPPFClient client = new JPPFClient();
JMXDriverConnectionWrapper jmx = client.awaitWorkingConnectionPool.awaitWorkingJMXConnection();
LoadBalancerPersistenceManagement mgt = jmx.getLoadBalancerPersistenceManagement();

or via direct instantiation:

JMXDriverConnectionWrapper jmx = new JMXDriverConnectionWrapper("www.myhost.com", 11098, false);
LoadBalancerPersistenceManagement mgt = jmx.getLoadBalancerPersistenceManagement();

This MBean is also directly usable via JMX-based tools such as JConsole or VisualVM: