JPPF Issue Tracker
star_faded.png
Please log in to bookmark issues
bug_report_small.png
CLOSED  Bug report JPPF-103  -  Node local resources cache may lead to grid hang in some environments
Posted Dec 13, 2012 - updated Dec 27, 2014
icon_info.png This issue has been closed with status "Closed" and resolution "RESOLVED".
Issue details
  • Type of issue
    Bug report
  • Status
     
    		 Closed
  • Assigned to
    lolo4j
  • Progress
       
  • Type of bug
    Not triaged
  • Likelihood
    Not triaged
  • Effect
    Not triaged
  • Posted by
    lolo4j
  • Owned by
    Not owned by anyone
  • Category
    Node
  • Resolution
    RESOLVED
  • Priority
    High
  • Reproducability
    Not determined
  • Severity
    Normal
  • Targetted for
    icon_milestones.png JPPF 3.1.x
Issue description
When a node is running on a system with limited or no disk space, the node's local resources cache (class org.jppf.classloader.ResourceCache) can fill up the available space and lead to corrupted or unavailable resource files, and prevent the node from working properly, potentially causing hangs of the entire grid.
Steps to reproduce this issue
We need to setup a configuration in which this problem occurs, possibly by using a very small quota for the folder(s) where the resosurces are stored.

#3
Comment posted by
lolo4j
Dec 13, 11:55
Right now I'm considering 2 possible fixes:
  • relatively easy: provide the ability to disable the cache using a configuration option
  • hard: provide the ablity to cache resources in memory instead of in the file system. This will require a serious refactoring of the whole caching mechanism. This might be doable by using the location API or something equivalent, so as to have an easily switchable generic storage mechanism.
If we chose the hard solution, then we'd also implement the easy one.

#6
Comment posted by
lolo4j
Dec 15, 09:39
This is what I have done:

  • I implemented an abstraction of the resources loaded by the class loader via the methods getResource(), getResources(), getResourceAsStream() and getMultipleResources()
  • this allows the same interface for resources stored on the file system or in memory
  • since the class loader methods return URLs (getResourceAsStream() actually returns an InputStream obtained from a URL), I had to write a URLStreamHandler which handles the "jppfres:" protocol, designed to handle resources stored in the class laoder's resource cache
  • I added a node configuration property "jppf.resource.cache.storage = memory|file", with a default value of "file". When "file" is configured, the resource cache will attempt to store each resource on the file system. If the operation fails for any reason, it will automatically fall back to memory storage for that resource. When "memory" is configured, there is no fall back. This can also serve as a performance optimization, at the cost of a greater memory footprint: the usual tradeoff.
  • added a static initializer in JPPFDriver and NodeRunner to ensure the protocol is registered via system property "java.protocol.handler.pkgs", without having to add a -D in the JVM options.


This is implemented in 3.1 branch and I still need to port it to the trunk.
#7
Comment posted by
lolo4j
Dec 15, 19:24
Fixed in:

The issue was updated with the following change(s):
  • This issue has been closed
  • The status has been updated, from Confirmed to Closed.
  • This issue's progression has been updated to 100 percent completed.
  • The resolution has been updated, from Not determined to RESOLVED.
  • Information about the user working on this issue has been changed, from lolo4j to Not being worked on.