Job monitoring API

In the same way that the topology monitoring API maintains a representation of the grid topology, the job monitoring API maintains a representation of the jobs being processed within the grid, including which jobs are processed by each drivers and which nodes the jobs are dispatched to.

1 Job monitor and jobs hierarchy

The class JobMonitor maintains a representation of the jobs as a hierarchy with 3 levels: driver / job / job dispatch, as shown in this picture:

Note that a JPPF job can be submitted to multiple drivers, either in parallel from the same client, or from one driver to another in a multi-driver topology. JobMonitor provides an API to retrieve all the drivers a job is submitted to.

Each element of the jobs hierarchy has its own class, represented in this object model:

As we can see, job elements share a common super class with the toplogy elements: AbstractComponent, which provides the base API to navigate a tree of elements identified by their uuid.

JobMonitor provides the following API to navigate the job hierarchy:

public class JobMonitor extends TopologyListenerAdapter {
  // Get the driver with the specified uuid
  public JobDriver getJobDriver(String driverUuid)

  // Get the drivers monitored by this job monitor
  public List<JobDriver> getJobDrivers()

  // Get the drivers to which a job was submitted
  public List<JobDriver> getDriversForJob(String jobUuid)

  // Get the dispatches of the specified job accrosss the entire topology
  // This returns job dispatches with nodes that may be connected to different servers
  public List<JobDispatch> getAllJobDispatches(String jobUuid)
}

The class JobDriver is essentially a wrapper for a TopologyDriver element, which represents a topology hierarchy that is orthogonal to the jobs hierarchy. It is defined as follows:

public class JobDriver extends AbstractJobComponent {
  // Get the proxy to the driver MBean that manages and monitors jobs
  public DriverJobManagementMBean getJobManager()

  // Get the associated driver from the topology manager
  public TopologyDriver getTopologyDriver()

  // Get a job handled by this driver from its uuid
  public Job getJob(final String jobUuid)

  // Get the list of jobs handled by this driver
  public List<Job> getJobs()
}

The Job class wraps a JobInformation object and provides information on the latest known state of a job, as seen by the driver it is submitted to. It is defined as follows:

public class Job extends AbstractJobComponent {
  // Get the information on the job
  public JobInformation getJobInformation()

  // Get the driver that holds this job
  public JobDriver getJobDriver()

  // Get the dispatch with the specified node uuid for this job
  public JobDispatch getJobDispatch(String nodeUuid)

  // Get the job dispatches for this job
  public List<JobDispatch> getJobDispatches()
}

JobDispatch represents a subset of a job that was sent to a node for execution and is the association of a JobInformation object with a TopologyNode:

public class JobDispatch extends AbstractJobComponent {
  // Get the information on the node for ths job dispatch
  public TopologyNode getNode()

  // Get the information on the job
  public synchronized JobInformation getJobInformation()

  // Get the job to which this dispatch belongs
  public Job getJob()
}

2 Creating and configuring a job monitor

2.1 Constructors

To create and initialize a JobMonitor, you have to call one of its two constructors:

public class JobMonitor extends TopologyListenerAdapter {
  // Create this jo mManager with the specified topology manager in
  // IMMEDIATE_NOTFICATIONS mode
  public JobMonitor(TopologyManager topologyManager, JobMonitoringListener...listeners)

  // Create this job manager with the specified topology manager and event mode
  public JobMonitor(JobMonitorUpdateMode updateMode, long period,
    TopologyManager topologyManager, JobMonitoringListener...listeners)
}

Examples:

JPPFClient client = new JPPFClient();
TopologyManager tManager = new TopologyManager(client);
JobMonitoringListener listener1 = new JobMonitoringListenerAdapter() { ... };
JobMonitoringListener listener2 = new JobMonitoringListenerAdapter() { ... };
JobMonitoringListener[] listenerArray = {  listener1,  listener2 };

JobMonitor monitor;
monitor = new JobMonitor(tManager);
monitor = new JobMonitor(tManager, listener1);
monitor = new JobMonitor(tManager, listenerArray);
monitor = new JobMonitor(UpdateMode.IMMEDIATE_NOTIFICATIONS, -1L, tManager);
monitor = new JobMonitor(UpdateMode.POLLING, 1000L, tManager, listener1, listener2);

Remarks:

1) A call to:

new JobMonitor(topologyManager, listenerArray)

is equivalent to:

new JobMonitor(UpdateMode.IMMEDIATE_NOTIFICATIONS , -1L, topologyManager, listenerArray)

2) The vararg parameter listeners is optional and needs not be specified if you do not wish to register a job monitor listener at construction time.

3) As we can see, a JobMonitor requires a TopologyManager to work with, since it will use its TopologyDriver and TopologyNode objects to construct the JobDriver and JobDispatch instances in the jobs hierarchy.

2.2 Update modes

As we have seen, a JobMonitor is given an update mode at construction time, whether implicitely or explicitely. The update mode is one of the elements of the JobMonitorUpdateMode enum, defined as follows:

public enum JobMonitorUpdateMode {
  // Updates are computed by polling job information from the drivers
  // at regular intervals
  POLLING,

  // Updates are computed from JMX notifications and pushed immediately
  // as job monitor events. This means one event for each jmx notification
  IMMEDIATE_NOTIFICATIONS,

  // Updates are computed from JMX notifications and published periodically as job
  // monitor events. Notifications are merged/aggregated in the interval between
  // publications
  DEFERRED_NOTIFICATIONS
}

In POLLING and DEFERRED_NOTIFICATIONS modes, updates are published periodically as events, hence the need for the period parameter in the constructor, which represents the interval between publications in milliseconds. There is, however, a significatant difference between these two modes: in POLLING mode, the period represents the interval between two JMX requests to the driver(s). Since this involves network communication, it can be a slow operattion and the value for period should not be too small, otherwise the job monitor may not be able to cope. A period of 1000L (1 second) or more is recommended. In DEFERRED_NOTIFICATIONS mode, the information to publish is already present in memory and the period represents the interval between two publications as events. Thus the period can be significantly smaller than in POLLING mode. For instance, when displaying the updates in a desktop GUI, you might want to make sure that the user's eyes will perceive them as continuous updates. For this you need to have at least 25 updates per second, which means a period of 40 milliseconds or less.

In IMMEDIATE_NOTIFICATIONS mode, updates are immediately pushed as events, therefore no period parameter is needed and the value used in the constructor is simply ignored.

To conclude, the purpose of the update modes is to provide options for the tradeoff between the accuracy of the job updates and the ability for the application to cope with the generated workload. For instance, a GUI application may not need to display all the updates, in which case the POLLING mode would be sufficient. If you need to receive all updates, then IMMEDIATE_NOTIFICATIONS is the mode to use. Finally, also note that POLLING is the mode which generates the least network traffic.

3 Receiving job monitoring events

JobMonitor allows registering one or more listeners to job events, either from its constructors, as we have seen in the previous section, or after construction time, with the following API:

public class JobMonitor extends TopologyListenerAdapter {
  // Initialize with the specified topology manager and listeners
  public JobMonitor(TopologyManager topologyManager, JobMonitoringListener...listeners)

  // Initialize with the specified topology manager, event mode and listeners
  public JobMonitor(UpdateMode updateMode, long period, TopologyManager topologyManager,
    JobMonitoringListener...listeners)

  // Add a listener to the events emitted by this job monitor
  public void addJobMonitoringListener(JobMonitoringListener listener)

  // Remove a listener to the events emitted by this job monitor
  public void removeJobMonitoringListener(JobMonitoringListener listener)
}

A listener is an implementation of the JobMonitoringListener interface, which provides the following notification methods:

public interface JobMonitoringListener extends EventListener {
  // Called when a new driver is added to the topology
  void driverAdded(JobMonitoringEvent event);

  // Called when a new driver is added to the topology
  void driverRemoved(JobMonitoringEvent event);

  // Called when a job is added to the driver queue
  void jobAdded(JobMonitoringEvent event);

  // Called when a job is removed from the driver queue
  void jobRemoved(JobMonitoringEvent event);

  // Called when the state a job has changed
  void jobUpdated(JobMonitoringEvent event);

  // Called when a job is dispatched to a node
  void jobDispatchAdded(JobMonitoringEvent event);

  // Called when a job dispatch returns from a node
  void jobDispatchRemoved(JobMonitoringEvent event);
}

If you do not need to implement all the methods, you may instead extend the JobMonitoringListenerAdapter class, which provides an empty implementation of each method in JobMonitoringListener.

These methods provide an input parameter of type JobMonitoringEvent:

public class JobMonitoringEvent extends EventObject {
  // Get the job monitor which emitted this event
  public JobMonitor getJobMonitor()

  // Get the job driver from which this event originates
  public JobDriver getJobDriver()

  // Get the related job, if any
  public Job getJob()

  // Get the related job dispatch, if any
  public JobDispatch getJobDispatch()
}

Note that getJob() and getJobDispatch() may return null, depending on the notification method called:

• getJob() will return null for the driverAdded() and driverRemoved() notifications
• getJobDispatch() only returns a value for the jobDispatchAdded() and jobDispatchRemoved() notifications.

As an example, the following code sample prints all the job monitoring events to the system console:

public class MyJobMonitoringListener implements JobMonitoringListener {
  @Override public void driverAdded(JobMonitoringEvent event) {
    print("driver ", event.getJobDriver(), " added");
  }

  @Override public void driverRemoved(JobMonitoringEvent event) {
    print("driver ", event.getJobDriver(), " removed");
  }

  @Override public void jobAdded(JobMonitoringEvent event) {
    print("job ", event.getJob(), " added to driver ", event.getJobDriver());
  }

  @Override public void jobRemoved(JobMonitoringEvent event) {
    print("job ", event.getJob(), " removed from driver ", event.getJobDriver());
  }

  @Override public void jobUpdated(JobMonitoringEvent event) {
    print("job ", event.getJob(), " updated");
  }

  @Override public void jobDispatchAdded(JobMonitoringEvent event) {
    print("job ", event.getJob(), " dispatched to node ", event.getJobDispatch());
  }

  @Override public void jobDispatchRemoved(JobMonitoringEvent event) {
    print("job ", event.getJob(), " returned from node ", event.getJobDispatch());
  }

  private void print(Object...args) {
    StringBuilder sb = new StringBuilder();
    for (Object o: args) {
      if (o instanceof AbstractComponent)
        sb.append(((AbstractComponent) o).getDisplayName());
      else sb.append(o);
    }
    System.out.println(sb.toString());
  }
}

TopologyManager tManager = ...;
// register the listener with the job monitor
JobMonitor monitor = new JobMonitor(tManager, new MyJobMonitoringListener());
// or, later on
monitor.addJobMonitoringListener(new MyJobMonitoringListener());