We (some people at the DGS-RP Group) are developing a Python-based system to share code snippets over the network. We started this project because some of our group wanted to use numpy et al. from within the Jython Pixelman (aladdin.utef.cvut.cz/ofat/others … elman.html) console. This system is in its infancy; currently it only supports Python. What we would like to do next is add support for Root. The aim would be to have an open “Root as a service” platform. Root developers can publish Root algorithms as remote procedures so other scientists can call and reuse these functions remotely.
Does something similar already exist? Would others also find such a system useful? … We’d love to get some feedback/comments!
Thanks
[quote=“joel.vogt”]We (some people at the DGS-RP Group) are developing a Python-based system to share code snippets over the network. We started this project because some of our group wanted to use numpy et al. from within the Jython Pixelman (aladdin.utef.cvut.cz/ofat/others … elman.html) console.[/quote]just so that I understand it: you ran into the lack of a Python C-API in Jython and that’s why you chose this approach? (That also means that technically, the remote process does not need to be python, right?)
For data streaming (as per the last slide in the .pdf you provided): PyROOT allows the pickling of ROOT/C++ objects. This is not super efficient (objects are written into a buffer, then copied into a string for pickling), but it does allow for easy transfer of basic C++ data objects through a python layer.
In a similar vein, there is a problem with extension modules and PyPy in the same way as for Jython (just to a lesser degree). Offloading to a PyPy process, over the network or locally, would then allow a speedup, without loosing the ability of using all extension modules one wants.
Yes, exactly. The C-API issue with Jython was the initial motivation to build a workaround. Thanks for pointing out parallel python and PYROOT’s pickling abilities. This is a super helpful information.
The other part of this proposal is to have an open “Root” (et al.) services platform. When we started working on CERNthon (that’s how we call our system), the idea came up that we could share the algorithms on the CERNthon server with others. Also we have some great Root code, which is only being used by the developer himself, because others in the group use different frameworks. That’s why we’d like to make that Root code too accessible remotely. A bit like a remote library to call domain specific algorithms.
What’s your thought on that?
[quote=“joel.vogt”]The other part of this proposal is to have an open “Root” (et al.) services platform. When we started working on CERNthon (that’s how we call our system), the idea came up that we could share the algorithms on the CERNthon server with others. Also we have some great Root code, which is only being used by the developer himself, because others in the group use different frameworks. That’s why we’d like to make that Root code too accessible remotely. A bit like a remote library to call domain specific algorithms.
What’s your thought on that?[/quote]
that description has the answer to the question in there. There are two reasons why such a setup could work. A) The software offered this way really is so great that users will jump through hoops to get at it; or B) it is actually more convenient to access than locally. The thing about most of ROOT itself, is that is very easily accessible locally. Probably the best feature that a service could offer is easy access (e.g. through a web browser) to the combination of algorithm and data. Point being that even if the former is easy to setup locally, the other probably isn’t.