JPPF Issue Tracker
star_faded.png
Please log in to bookmark issues
feature_request_small.png
CLOSED  Feature request JPPF-11  -  Ability for the nodes to work offline
Posted Aug 08, 2012 - updated Oct 18, 2013
action_vote_minus_faded.png
0
Votes
 action_vote_plus_faded.png
icon_info.png This issue has been closed with status "Closed" and resolution "RESOLVED".
Issue details
  • Type of issue
    Feature request
  • Status
     
    		 Closed
  • Assigned to
    lolo4j
  • Progress
       
  • Type of bug
    Not triaged
  • Likelihood
    Not triaged
  • Effect
    Not triaged
  • Posted by
    lolo4j
  • Owned by
    lolo4j
  • Category
    Server
  • Resolution
    RESOLVED
  • Priority
    High
  • Targetted for
    icon_milestones.png JPPF 4.0
Issue description
This is about the ability to persist jobs at the server and/node level, so that the following becomes possible:
  • after a server crash or shutdown, the server could read pending jobs from a store and resume their execution
  • if a client gets disconnected before getting the results of one or more job, it maybe reconnected/restarted and get the results back from the server, and this wouldn't prevent the job from being executed by the nodes (some conditions would have to apply, essentially related to class loading)

#5
Comment posted by
lolo4j
Aug 01, 06:43
I'm currently working on the node side. I have implemented an "offline" mode for the nodes, which for now allows them to work without any remote class loading (class loader is instantiated but not connected to the server). This is done via the configuration: "jppf.node.offline = true". It is committed to SVN trunk revision 2837.

This implies that the node must have additional entries in its classpath: server.jar and common.jar, plus eventually any class supporting the execution of the tasks. In addition to this, I would like to allow the jobs to transport additional classpath elements, so that new code can be executed without heavy deployment effort on the node side. This means that the additional classpath has to be specified as:
  • a local (to the node) path,
  • a remote path reachable from the node, for instance as a url,
  • or transported along with the job. This means as part of the job SLA (preferred) or job metadata. It cannot be in the DataProvider, since the classes it would hold may be needed to deserialize it.
What we need is to provide a convenient API to specify a class path in the job SLA. It seems the Location API could serve as a good basis for this.

#6
Comment posted by
lolo4j
Aug 10, 09:34
I just committed a frst working version for offline nodes.

When a node is in offline mode, it gets a job from the server, then immediately disconnects. It then executes the job and reconnects just before sending the results and getting a new job. With this mechanism, it is now possible to have tens of thousands, or even millions of nodes working with a single server (nodes might have to wait until they can connect). Addtionally an offline node has no classloader connection and no JXM server: it is not manageable, by design. Thus a single TCP connection to the server is needed. I'm thinkng the same mechaism could be applied to clients: with no class loader connection, the client would then only support libraries embedded within the job SLA.

I added a get/setClassPpath() API to the job SLA, to allow libraries to be dynamically added to the classpath (this resets the classloader). Otherwise it is possible to use static libraries in the node's main class loader classpath. Each classpath element specified in the SLA points to a library vie a Location object. This makes it very easy to handle.

See SVN trunk revision 2854