Node throttling

1 Definition and rationale

This pluggable mechanism allows a JPPF node to warn a driver that it can no longer accept jobs, whenever a condittion is reached. When the condition no longer applies, a new notification is sent to the driver to specify that the node accepts jobs again.

The main purpose of this facility is to avoid rsource exhaustion in the nodes, since by default a node can process an unlimited number of jobs at any given time. However, it can be used for other purposes, for instance to define time windows during which a node will or will not accept work.

2 How it works

The node throttling relies on the service provider interface (SPI) mechanism. The service interface, which is the contract between the node and a plugin implementation, is represented by the JPPFNodeThrottling interface, defined as:

public interface JPPFNodeThrottling {
  // Determine whether the node accepts new jobs
  boolean acceptsNewJobs(Node node);
}

An implementation of JPPFNodeThrottling will be invoked at two points in the node's life cycle:

• during the initial handshake with the driver
• at regular intervals via a dedicated timer

The duration of the interval between periodic checks is configurable in the node with the following property, expressed in milliseconds and defaulting to 2000 milliseconds:

# interval between throttling checks in milliseconds
jppf.node.throttling.check.period = 2000

3 Implementing a node throttling plugin

Let's take a simple example implementation which causes the node to no longer accept new jobs when the heap usage is more than 90% of the maximum heap size:

public class HeapBasedThrottling implements JPPFNodeThrottling {
  @Override
  public boolean acceptsNewJobs(final Node node) {
    final double pct = getUsedMemoryPct();
    final boolean acceptsJobs = pct < 90d;
    // invoke a GC so that at next check the node may be accepting jobs again
    if (!acceptsJobs) {
        System.gc();
    }
    return acceptsJobs;
  }

  // Compute the percentage of currently used heap memory
  // where used heap ratio = (current heap size - current free heap) / max heap sze
  private static double getUsedMemoryPct() {
    final Runtime rt = Runtime.getRuntime();
    final double usedMemory = rt.totalMemory() - rt.freeMemory();
    return 100d * usedMemory / rt.maxMemory();
  }
}

4 Deploying the plugin

As usual with an SPI-based service, follow these steps to deploy:

• create a service file in META-INF/services named “org.jppf.node.throttling.JPPFNodeThrottling”
• in this file, add the fully qualified class name of your implementation of the JPPFNodeThrottling interface
• copy the jar file or class folder containing your implementation and service file to either the JPPF driver's class path, if you want it deployed to all nodes connected to that driver, or to the classpath of individual nodes, if you only wish specific nodes to have the plugin.

5 Built-in implementation: heap usage-based throttling

The class MemoryThresholdThrottling is a slightly more sophisticated version of the example we have seen above. It also causes a node to not accept jobs whenever heap usage reaches a configured percentage of the maximum heap size, and can additionally be configured with the following properties:

- it is deactivated by default and can be activated by setting this property (false by default):

jppf.node.throttling.memory.threshold.active = true

- the threshold can be set as a percentage of the maximum heap size with this property (90% by default):

jppf.node.throttling.memory.threshold = 87.5

- it can also call System.gc(), in an attempt to mitigate the high heap usage, after acceptsNewJobs(Node) returns false a configured consecutive number of times. The corresponding configuration property is set as (3 by default):

jppf.node.throttling.memory.threshold.maxNbTimesFalse = 2

Example configuration:

# activate the throttling plugin
jppf.node.throttling.memory.threshold.active = true
# % of maximum heap size that causes the node to refuse jobs
jppf.node.throttling.memory.threshold = 87.5
# number of time the check returns false before invoking System.gc()
jppf.node.throttling.memory.threshold.maxNbTimesFalse = 5