JPPF Issue Tracker
star_faded.png
Please log in to bookmark issues
feature_request_small.png
OPEN  Feature request JPPF-480  -  Jobs persistence in the driver
Posted Oct 27, 2016 - updated Mar 16, 2017
action_vote_minus_faded.png
0
Votes
action_vote_plus_faded.png
lolo4j (lolocohen) has been working on this issue since October 27, 2016 (08:00)
Issue details
  • Type of issue
    Feature request
  • Status
     
    New
  • Assigned to
     lolo4j
  • Type of bug
    Not triaged
  • Likelihood
    Not triaged
  • Effect
    Not triaged
  • Posted by
     lolo4j
  • Owned by
    Not owned by anyone
  • Category
    Core
  • Resolution
    Not determined
  • Priority
    High
  • Targetted for
    icon_milestones.png JPPF 6.0
Issue description
The goal is to provide persistence of the jobs in the driver queue, to allow recovery of the jobs whenever the driver is stopped or killed, without needing the client to resubmit.

Some preliminary considerations:
  • granularity of persistence indicator: should it be at job level or driver level? (I'd rather set it per job)
  • how should the jobs be persisted, i.e. write-behind vs write-through
  • persistence should be pluggable, meaning we should provide an interface and at least one default implementation (e.g. file-based like we do for memory-based offload) and allow users to plug their own custom implementations
  • make sure the limitations due to class loading are well understood and handled properly. In particular, we may need to revisit the job classpath management in the SLA to make it more usable and automated (i.e. ability to force the node to use the job classpath)
  • what to do when the client reconnects to the driver? Should the resubmit behavior be deactivated?
  • once tasks are executed, their results shall be persisted as well
  • clients should have the ability to query the driver for pending/completed jobs, allowing them to retrieve the results