JPPF currently has a very basic API for job streaming. However, ths is wholly insufficient for real stream processing, as many things are missing which have to be implemented by the users in an ad-hoc manner:

• ability to plug-in stream data sources (from files, databases, event-based systems, etc.)
• stateful computations over streams of data (aggregates, analytics, etc.)
• map/reduce capabilities
• ... ?

We propose to explore what can be done in that area. Some ideas that popped into my (tiny) head:

• make the job's data provider read-write instead of read-only, so as to provide state at the job level instead of only at the task level. We could then associate a merge strategy / reducer to each job. The data provider seems a good candidate, as opposed to the job metadata, because it is loaded the same class loader as the tasks.
• provide a common API for various sources of straming data and plug them into a job sream