vendredi 28 août 2015

Will Spark program be properly used as real-time back-end in web application

The background & requirement is like this, web application framework would have two main parts: web pages, and back-end spark/sparkR computing program which is ran on cluster. There is also another java program as middleware to make those two communicate.

Java program would get some parameters from web side, then it wants to call remote spark/sparkR program by way of passing previous parameters, next it also hopes to get the returning results from spark/sparkR program, finally the java program will do some process about the results and back-pass web side.

All would like to be done in one java program as normal function calling, since real-time dispatcher is needed.

The problems are: 1. Is spark/sparkR properly used in web application for real-time requirement, how is the skill framework like...

  1. Then, how the java program will call the spark program by way of leapping over different machines/clusters . By gateway or some more skills?

  2. Would java program conveniently get the returning results from spark program as normal function calling?

  3. If all the above is OK, then, if it comes to sparkR program, is it conveniently called by java program? How will the java program use the sparkR returning object?

I shall appreciate your all help very much!




Aucun commentaire:

Enregistrer un commentaire