Custom discovery of peer drivers

1 Rationale

The current built-in peer driver discovery mechanisms in the JPPF server have potential shortcomings:

• automatic discovery relies on the UDP multicast protocol, which is not available in all environments. For instance, cloud environments do not allow it.
• the manual configuration of the connections requires that all possible peer drivers be known in advance, which prevents new, previously unknown drivers from being discovered

To overcome these limitations, and to allow more flexibility as to how peer drivers are discovered, you can now implement your own custom disocovery mechanism.

2 Implementation

A driver discovery mechanism is implemented by extending the class PeerDriverDiscovery, defined as:

public abstract class PeerDriverDiscovery extends DriverDiscovery<DriverConnectionInfo> {
}

As we can see, this class has no method of its own, and the interesting methods are in its superclass DriverDiscovery:

public abstract class DriverDiscovery<E extends DriverConnectionInfo> {
  // Perform the driver discovery. This method runs in its own separate thread
  public abstract void discover() throws InterruptedException;

  // Notify that a new driver was discovered
  protected void newConnection(E info)

  // Shut this discovery down. This method is intended to be overriden
  // in subclasses to allow user-defined cleanup operations
  public void shutdown()
}

By default, the shutdown() method does nothing and is intended to be overriden in subclasses if needed. It is called when the JPPF driver is shut down and is thus part of the discovery's life cycle.

The actual discovery is performed within the discover() method. From this method, for each discovered driver you must call the newConnection() method, passing an instance of DriverConnectionInfo defined as:

public class DriverConnectionInfo
  // Get the name given to this connection
  public String getName()

  // Determine whether secure (with SSL/TLS) connections should be established
  public boolean isSecure()

  // Get the driver host name or IP address
  public String getHost()

  // Get the driver port to connect to
  public int getPort()

  // Get the number of connections (pool size) to etablish with the peer driver
  public int getPoolSize()
}

Here is an example of a very simple implementation:

public class SimplePeerDiscovery extends PeerDriverDiscovery {
  @Override
  public void discover() throws InterruptedException {
    // new peer connection named "myPeerDriver" with a pool size of 2
    newConnection(
      new DriverConnectionInfo("myPeerDriver", false, "www.myhost.org", 11111, 2));
  }
}

The discover() method of a PeerDriverDiscovery is not limited to calling newConnection() only once. You can invoke this method with as many instances of DriverConnectionInfo as you wish, allowing for the discovery of as many peer drivers as required.

Additonally, the discover() method runs in its own separate thread and doesn't have to terminate immediately. This means that you can run a loop inside that may, for instance, perform a periodic polling of an (external) service to discover new peer drivers over time. Here is a more complex example illustrating this:

public class PollingDiscovery extends PeerDriverDiscovery {
  // whether this discovery was shutdown
  private boolean shutdownFlag = false;

  @Override
  public void discover() throws InterruptedException {
    while (!isShutdown()) {
      List<DriverConnectionInfo> peers = externalLookup();
      if (drivers != null) {
        for (DriverConnectionInfo peer: peers) newConnection(peer);
      }
      synchronized(this) {
        // wait 5 seconds before the next lookup
        wait(5000L);
      }
    }
  }

  // Query an external service for discovered drivers
  public List<DriverConnectionInfo> externalLookup() {
    return ...;
  }

  public synchronized boolean isShutdown() {
    return shutdownFlag;
  }

  @Override
  public synchronized void shutdown() {
    shutdownFlag = false;
    // wake up the discover() thread
    notify();
  }
}

3 Deployment via SPI

A driver discovery mechanism can be automatically loaded and installed via the Service Provider Interface (SPI):

• in your source or resources folder, create a file META-INF/services/org.jppf.discovery.PeerDriverDiscovery
• edit this file and add a line with the fully qualified class name of your implementation of PeerDriverDiscovery, for instance:
  org.jppf.example.SimplePeerDiscovery
• make sure that this file, along with the implementation class, is in the classpath of your server

4 Deployment via API

The class JPPFDriver provides an API to add or remove PeerDriverDiscovery implementations at any time:

public class JPPFDriver extends AbstractGenericClient {
  // Add a custom driver discovery mechanism to those already registered, if any
  public void addDriverDiscovery(PeerDriverDiscovery discovery)

  // Remove a custom driver discovery mechanism from those already registered
  public void removeDriverDiscovery(PeerDriverDiscovery discovery)
}

To obtain an instance of JPPFDriver, you will need to wrap the code in a driver extension that provides it, for instance a driver startup class:

public class MyDiscoveryInitializer implements JPPFDriverStartupSPI {
  JPPFDriver driver;

  @Override
  public void run() {
    // create and add the peer discovery provider
    driver.addDriverDiscovery(new SimplePeerDiscovery());
  }

  // JPPF will call this method before run()
  public void setDriver(JPPFDriver driver) {
    this.driver = driver;
  }
}

5 Tip

To prevent the built-in discovery mechanisms from starting, set the following server configuration properties:

jppf.peer.discovery.enabled = false
# set an empty value or simply comment it out
jppf.peers =

This way, only custom peer driver discovery mechanisms will be used.