Class loading in JPPF

1 How it works

The distributed class loading framework in JPPF is the mechanism that makes it possible to execute code in a node that has not been explicitely deployed to the node's environment. Through this, JPPF tasks whose code (the actual bytecode to execute) is only defined in a JPPF client application, can be executed on remote nodes without the application developer having to worry about how this code will be transported there.

While this mechanism is fully transparent from the client application's perspective, it has a number of implications and particularities that may impact various aspects of JPPF tasks execution, including performance and integration with external libraries.

Let's have a quick view of the path followed by a class loading request at the time a JPPF task is executed within a node:

We can see that this class loading request is executed in four steps:

1. the node sends a network request to the remote server for the class
2. the server forwards the request to the identfied remote client
3. the client provides a response (the bytecode of the class) to the server
4. the server forwards the response to the node

Once these steps are performed, the node holds the bytecode of the class and can effectively define and load it as for any standard Java class.

This use case is a simplification of the overall class loading mechanism in JPPF, however it illustrates what actually takes place when a task is executed in a node.This also raises a number of questions that need clarification:

• how does the server know which client to forward a request to?
• how does this fit into the Java class loader delegation model?
• how does it work in complex JPPF topologies with multiple servers?
• how does it apply to JPPF customizations and add-ons or external libraries that are available in the server or node's classpath?
• what is the impact on execution performance?
• what possibilities does this open up for JPPF applications?

We will address these questions in details in the next sections.

2 Class loader hierarchy in JPPF nodes

The JPPF class loader mechanism follows a hierarchy based on parent-child relationships between class loader instances, as illustrated in the following picture:

The system class loader is used to start the JPPF node. With most JVMs, it will be an instance of the class java.net.URLClassloader and its usage and creation are handled by the JVM.

The server class loader is a concrete implementation of the class AbstractJPPFClassloader and provides remote access to classes and resources in the server's classpath. It is created at the time the node establishes a connection to the server. It is also discarded when the node disconnects from the server. The parent of the server class loader is the system class loader. Please note that AbstractJPPFClassLoader is also a subclass of URLClassLoader.

The client class loaders are also concrete implementations of AbstractJPPFClassloader and provide remote access to classes and resources in one or more clients' classpaths. Each client class loader is created the first time the node executes a job which was submitted by that client. Thus, the node may hold many client class loaders.

It is important to note that, by design, the JPPF node holds a single network connection to the server, shared by all instances of AbstractJPPFClassLoader, including the server and clients class loaders. This design avoids a lot of potential confusion, inconsistencies and synchronization pitfalls when performing multiple class loading requests in parallel.

By default, a JPPF class loader follows the standard delegation policy to its parent. This means that, when a class is requested from a client class loader, it will first delegate to its parent, the server class loader, who will in turn first delegate to the system class loader. If a class is not found by the parent, then the class loader will look it up in the classpath to which it has access.

Thus, the flow of a request, for a class that is only in a client's class path, becomes a little more complex:

Here, the first two steps are initiated by the server class loader, as a result of the client class loader delegating to its parent. What is missing from this picture are the calls to the system class loader, since they are only meaningful if the requested class is in the node's local classpath.

3 Relationship between JPPF UUIDs and class loaders

We have seen in the Development Guide that each JPPF client has its own identifier, unique across the entire JPPF grid. This is also true of servers and nodes. The client UUID is what allows a JPPF node to know which client a class loader is associated with, and use it to route a class loading request from the node down to the client that submitted the job.

If a node only knows the client UUID, then it will only be able to handle the routing of class loading requests in the simplest JPPF grid topology: a topology which only has one server. However, there is a mechanism that allows the class loading to work in much more complex topologies, such as this one:

To this effect, each job executed on a node will transport, in addition to the originating client's UUID, the UUID of each server in the chain of servers that had to be traversed to get to the node. In JPPF terminology, this ordered list of UUIDs is called a UUID path. With this information known, it is possible to route a class loading request through any chain of servers, as illustrated in the picture below:

It is also important to note that this does not change anything to the class loader hierarchy within the node. In effect, there is still only one server class loader, which is associated with the server the node is directly connected to. This implies that the parent delegation model will not cause a class loading request to traverse the server chain multiple times.

Another implication of using client UUIDs is that it is possible to have multiple versions of the same code running within a node. Let's imagine a situation where two distinct JPPF clients, with separate UUIDs, submit the same tasks. From the node's point of view, the classes will be loaded by two distinct client class loaders, and therefore the classes from the first client will be different from those of the second client, even if they have the exact same bytecode and are downloaded from the same jar file.

The reverse situation may also happen, when two clients with the same UUID submit tasks that use different versions of the same classes. In this case, the tasks will be exposed to errors, especially at deserialization time, if the two versions are incompatible.

4 Built-in optimizations

JPPF provides a number of built-in optimizations and capabilities that enable to reduce the class loading overhead and avoid excessive non-heap memory consumption when the number of classes that are loaded becomes large. We will review these features in the next sections.

4.1 Deployment to specific grid components

In some situations, there can be a large number of classes to load before a JPPF task can be execute by a node. Even though this class loading overhead is a one-time occurrence, it can take a significant amount of time, especially if the network communication between node and server, or between server and client, is slow. This may happen, for instance, when the tasks rely on many external libraries, causing the loading of the classes within these libraries in addtiion to the classes in the application.

One way to overcome this issue is to deploy the external libraries to the JPPF server or node's classpath, to significantly reduce the time needed to load the classes in these libraries. The main drawback is that it requires to manage the deployed libraries, to ensure that they are consistently deployed across the grid, especially at such times when some of the libraries must be upgraded or removed, or new ones added. However, it is considered a good practice in production environment where few or no changes are expected during long periods of time.

4.2 Using a constant JPPF client UUID

In the Development Guide, we have seen that it is possible to set the UUID of a JPPF client to a user-defined value, using the constructor JPPFClient(String uuid). This can be leveraged to force the nodes to reuse the client class loader for the specified UUID, even after the client application is terminated and has been restarted. It also implies that, if multiple clients use the same UUID, the same client class loader will also be used in the nodes. Thus, this feature limits the initial class loading overhead to the first time a job is submitted by the first client to run.

The main drawback is that, if the code of the tasks is changed on the client side, the changes will not be automatically taken into account by the nodes, and some errors may occur, due to an incompatibility between class versions in the node and in the client. If this happens, then you will have to change the client UUID or restart the nodes to force a reload of the classes by the nodes.

4.3 Node class loader cache

Each JPPF node maintains a cache of client class loaders. This cache has a bounded size, in order to avoid out of memory conditions caused by too many classes loaded in the JVM. This cache has an eviction policy based on the least recently created class loader. Thus, when the cache size limit is reached and a new class loader needs to be created, the oldest class loader that was created is removed from the cache, which frees up a slot for the new class loader.

As described in the Configuration Guide, the cache size is defined in the node's configuration file as follows:

jppf.classloader.cache.size = n

where n is a strictly positive integer

4.4 Local caching of network resources

The class loader also caches locally, either in memory or on the node's local file system, all resources found in the classpath that are not class definitions, when one of its methods getResourceAsStream(), getResource(), getResources() or getMultipleResources() is called. This avoids a potentially large network overhead the next time the same resources are requested.

The resources cache can be enabled or disabled with a configuration property:

# whether the cache is enabled, defaults to 'true'
jppf.resource.cache.enabled = true

The type of storage for these resources can also be configured:

# either 'file' (the default) or 'memory'
jppf.resource.cache.storage = file

When “file” persistence is configured, the node will fall back to memory persistence if the resource cannot be saved to the file system for any reason. This could happen for instance when the file system runs out of space.

When the resources are stored on the local file system, the root of this local file cache is located at the default temp directory, such as determined by a call to System.getProperty("java.io.tmpdir"). This can be overriden using the following JPPF node configuration property:

jppf.resource.cache.dir = some_directory

In fact, the full determination of the root for the resources cache is done as follows:

• if the node configuration property "jppf.resource.cache.dir" is defined, then use its value
• otherwise, if the system property "java.io.tmpdir" is defined, then use it
• otherwise, if the system property "user.home" is defined, then use it
• otherwise, if the system property "user.dir" is defined, then use it
• otherwise, use the current directory "."

Additionally, to avoid confusion with any other applications storing temporary files, the JPPF node will store temporary resources in a directory named “.jppf” under the computed cache root. For instance, if the computed root location is “/tmp”, the node will store resources under “/tmp/.jppf”.

4.5 Batching of class loading requests

There are two distinct mechanisms that allow an efficient grouping of outgoing class laoding requests:

In the node:

Class loading requests issued by the node's processing threads are not immediately sent to the server. Instead, the node will collect requests for a very short time (by default 100 nanoseconds or more, depending on th system timer accuracy), then send them at regular intervals. While collecting requests, the node will also identify and handle duplicate requests (i.e. parallel requests from multiple threads for the same class). Thus grouped, mutliple requests (and their responses) will require much less nextwork transport time.

The batching period can be specified with the following node configuration property:

# batching period for class loading requests, in nanoseconds (defaults to 100)
jppf.node.classloading.batch.period = 100

In the server:

A similar mechanism exists for the class loading requests forwardied by the server to a client. In this case, however, the server doesn't wait for a fixed time to send the requests. Instead it will take advantage of the time taken to send a request and receive its response. During that time, multiple nodes may be requesting the same resource, and the server will be able to send the request only once and dispatch the response to multiple nodes. The performance gains are variable but substantial: our stress tests show a class loading speedup going from 8% with 1 node, up to 30% with 50 nodes.

4.6 Classes cache in the JPPF server

Each JPPF server maintains an in-memory cache of classes and resources loaded via the class loading mechanism. This cache speeds up the class loading process by avoiding network lookups on the JPPF clients that hold the requested classes in their classpath. To avoid potential out of memory conditions, this cache uses soft references to store the bytecode of classes. This means that these classes may be unloaded from the cache by the garbage collector if the memory becomes scarce in the server. However, in most situations the cache still provides a significant speedup.

This cache can be disabled in the server's configuration:

# Specify whether the class cache is enabled. Default is 'true'
jppf.server.class.cache.enabled = false

4.7 Node customizations

As seen in the chapter Extending and Customizing JPPF > Flow of customizations in JPPF, most node customizations (except for the JMX logger and Initialization hooks) are loaded after the node has established a connection with the server. This enables these customizations to be loaded via the server class loader, which means they can be deployed to the server's classpath and then automatically downloaded from the server by the node.

You may also choose to deploy the customizations to the node's local classpath, in which case you will have to do it for all nodes that require this customization. In this case, the customizations will load faster but they incur the overhead of redeploying new versions to all the nodes.

5 Class loader delegation models

As we have seen previously, the JPPF class loaders follow by default the parent-first delegation model. We have also seen that the base class AbstractJPPFClassloader is a subclass of URLClassloader, which maintains a set of URLs for its classpath, each URL pointing to a jar file or class folder. One particularity of AbstractJPPFClassloader is that it overrides the addURL(URL) method to make it public instead of protected. The implication is that any node customization or JPPF task will have access to this method, and will be able to dynamically extend the classpath of the JPPF class loaders.

To take advantage of this, the node provides an additional delegation model for its class loaders, which will cause them to first lookup in their URL classpath as specified with call to addURL(URL), and then lookup in the remote server or client.

When this delagtion model is activated, the lookup for a class or resource from a client class loader will follow these steps:

• Lookup in the URL classpath
  • client class loader: delegate to the server class loader
  • server class loader: lookup in the URL classpath only
  • if the class is found, then end of lookup
  • otherwise, back to the client class loader, lookup in the URL classpath only
  • if the class is found, end of lookup
• Otherwise lookup in the server or client classpath
  • client class loader: delegate to the server class loader
  • server class loader: send a class loading request to the server
  • if the class is found in the server's classpath or cache, end of lookup
  • otherwise, the client class loader sends a request to the server to lookup in the client's classpath
  • if the class is found, end of lookup
  • otherwise throw a ClassNotFoundException

To summarize: when the URL-first delegation model is active, the node will first lookup classes and resources in the local hierarchy of URL classpaths, and then on the network via the JPPF server.

The delegation model is set JVM-wide in a node, it is not possible to specify different models for different class loader instances. There are three ways to specifiy the class loader delegation model in a node:

Statically in the node configuration:

# possible values: parent | url, defaults to parent
jppf.classloader.delegation = parent

Dynamically by API:

public abstract class AbstractJPPFClassLoader
  extends AbstractJPPFClassLoaderLifeCycle {

  // Determine the class loading delegation model currently in use
  public static synchronized DelegationModel getDelegationModel()

  // Specify the class loading delegation model to use
  public static synchronized void setDelegationModel(final DelegationModel model)
}

The delegation model is defined as the type safe enum DelegationModel:

public enum DelegationModel {
  // Standard delegation to parent first
  PARENT_FIRST,
  // Delegation to local URL classpath first
  URL_FIRST
}

Dynamically via JMX:

The related getter and setter are available in the interface JPPFNodeAdminMBean, which is also implemented by the JMX client JMXNodeConnectionWrapper. These allow you to dynamically and remotely change the node's delegation model:

public interface JPPFNodeAdminMBean extends JPPFAdminMBean {
  // Get the current class loader delegation model for the node
  DelegationModel getDelegationModel() throws Exception;

  // Set the current class loader delegation model for the node
  void setDelegationModel(DelegationModel model) throws Exception;
}

There is one question we are entitled to ask: what are the benefits of using the URL-first delegation model? The short answer is that it essentially provides a significant speedup of the class loading in the node, by providing the ability to download entire jar files and libraries and adding them dynamically to the node's class path. The next section of this chapter will detail how this, among other possibilities, can be achieved.

6 JPPF class loading extensions

6.1 Dynamically adding to the classpath

As we have seen previously, the class AbstractJPPFClassloader, or more accurately its direct superclass AbstractJPPFClassLoaderLifeCycle, exposes the addURL(URL) method, which is a protected method in the JDK's URLClassLoader. This means that it is possible to add jar files or class folders to the class path of a JPPF class loader at run time.

The main benefit of this feature is that it is possible to download entire libraries, then add them to the classpath, and thus dramatically speed up the class loading in the node. In effect, for applications that use a large number of classes, downloading a jar file will take much less time than loading classes one by one from the JPPF server or client.

Furthermore, the downloaded libraries can then be stored on the node's local file system, so they don't have to be downloaded again when the node is restarted. They can also be managed automatically (with custom code) to handle new versions of the libraries and remove old ones.

6.2 Downloading multiple resources at once

AbstractJPPFClassloader provides an additional method to download multiple resources in a single request:

public URL[] getMultipleResources(final String...names)

This is an equivalent to the getResource(String name) method, except that it works with multiple resources at once. The returned array of URLs may contain null values, which means the corresponding resources were not found in the class loader's classpath. The main advantage of this method is that it performs all resources lookups in a single request, which implies a single network round-trip when looking up in the server or client's classpath.

For instance this could be used to download a set of jar files and add them dynamically to the classpath, as seen in the previous section.

6.3 Resources lookup on the file system

When requesting a resource via one of the getResourceAsStream(), getResource(), getResources() or getMultipleResources() methods, the JPPF class loader will lookup the specified resources in the server or client's local file system if they are not found in the class path.

This is provided as a basic convenient way to download files from a JPPF server or client, without having to use or code a specific file download faciliy (such as having an FTP server on the JPPF server of client).

However, there is a limitation to this facility: the resource path should always be relative to the server or client's current directory (determined via a System.getProperty(“user.dir”) call). In particular, using an absolute path will lead to unpredictable results.

6.4 Resetting the node's current task class loader

As JPPF class loader instances and their connection to the driver are separate entities, it is possible to create new client class loader instances, for the same client UUID, at some points in the node's life cycle. This provides the ability to use an entirely different class path for jobs submitted by the same client.

This is done by calling the Node.resetTaskClassLoader() method, which is available in two node extension points:

• in the node life cycle listener, where the node is available from the jobHeaderLoaded() notifications
• in pluggable node MBeans, where the node is provided in the MBean provider's createMBean() factory method

Keep in mind that, when resetTaskClassLoader() is invoked, the old task class loader is invalidated (closed) and should no longer be used. The class loader instance returned by the method must be used instead. Here is an example usage in a NodeLifeCycleListener:

public class MyNodeListener extends NodeLifeCycleListenerAdapter {
  @Override
  public void jobHeaderLoaded(NodeLifeCycleEvent event) {
    try {
      URL url = ...;
      // the old class loader is invalidated (closed) and a new one is created
      AbstractJPPFClassLoader newCL =
        (AbstractJPPFClassLoader) event.getNode().resetTaskClassLoader();
      newCL.addURL(url);
      URL[] urls = newCL.getURLs();
      // display the added urls
      System.out.println("list of urls: " + Arrays.asList(urls));
    } catch (Exception e) {
      e.printStackTrace();
    }
  }
}

7 Related sample

Please look at the Extended Class Loading sample in the JPPF samples pack.