I am very glad to announce that gLitePROOF project is being upgraded to “PROOF on Demand” - a POD project.
This is not only a change of the name of the project, but mainly the significant upgrade of usability and usage fields.
Starting from gLitePROOF version 2.0.7, the project got a new name and a plug-in support. It now supports different job managers, for example, current version already supports gLite plug-in and LSF plug-in.
That simply means, users are able to get PROOF workers on gLite sites or local LSF batch farms or on all kinds at the same time (it’s also possible) by only one click.
Very soon there will be releases of plug-ins for PBS, Globus and we planning to use Condor as well.
POD doesn’t require root privileges and has negligible number of requirements, has a simple installation procedure. It setups a PROOF cluster on the fly.
Project currently is in a transformation stage, but you can already check out and try:
From now on PoD supports plug-ins. It provides already LSF and gLite plug-ins. That means you can get your PROOF cluster on the LSF or/and gLite workers just with one click.
Very soon we want to provide a Condor plug-in as well.
The main goal of this development was to reduce resource requirements PoD server need to handle its workers, in terms of a number of threads and memory. It looks like the task was successfully accomplished. Resource requirements were reduced by a factor of 3.
Now, since GSI ALICE group started to use PoD, PoD is tested with an ALICE PROOF-tutorial analysis (an example of a standard ALICE PROOF analysis) using 80 dynamic PROOF workers on LSF batch system.
I’ve just released PoD 2.0.10.
This is a patch release, it addresses a hanging-on-merging issue (which could happen with PoD 2.0.8-2.0.9) and brings some performance improvements.
This time PoD got lots of improvements and most of all the pod-agent - a core module, has been upgraded and now made a use of the thread pool pattern. This gives a much smaller virtual memory footprint and also improves the performance by a huge factor in a network intensive operations (such as a file list validation when thousands of files are in the list and a merging) when many worker nods are used.
All users are kindly asked to stop using all prevues versions of PoD, including betas and move to v2.1.1.
The first reason is that this version is a factor of 10 faster in network intensive operations (like file validation and merging) and a virt. memory footprint of it is now reduced by a factor of 14 in compare to the prevues release + many many more improvements.
The second, since I am the only developer I can’t give much support for older versions. So, if you have a problem with PoD the first question would be, “are you using the latest one?”…
BTW, starting from this version, PoD generates a helper header file. The header should help you to automat a PROOF connection string in your scripts.
Read more in user’s manual.
There are many more other features added and fixed. You can find more information in the Release Notes.
It is my pleasure to announce the release of PoD 2.1.2.
Some Highlights of this release
[ul]
[li]Now PoD officially supports AFS. You can install PoD at CERN. See pod.gsi.de/PoD/Documentation.html (PoD at CERN’s AFS).[/li]
[li]It is very important to mention, that the code of PoD’s LSF plug-in got a lot of improvements. It is now smarter, faster, and, most importantly, LSF and resource friendly.[/li]
[li]Now users can configure whether to receive LSF jobs’ std/err output via emails or files (files - by default). It’s configurable in PoD user defaults ($POD_LOCATION/etc/PoD.cfg). Only for the LSF plug-in.[/li]
[li]Now user can configure whether PoD workers should upload log files at the end of the session or not (not - by default). It’s configurable in PoD user defaults ($POD_LOCATION/etc/PoD.cfg). Only for the LSF plug-in.[/li]
[li]Now users can define $ROOTSYS as a ROOT version for workers. Workers will use that version of ROOT, which is defined on a PoD server machine (in case of shared home) or WN will use any version defined on it under $ROOTSYS. It’s configurable in PoD user defaults ($POD_LOCATION/etc/PoD.cfg), see worker.my_rootsys. Now it is a default value.[/li]
[li]GUI (pod-console) now can detect that there is no user activity. And if it lasts for 15 minutes, it switches all sensors automatically off and shows an information message. This prevents unnecessary system usage when pod-console is left without a user attention.[/li][/ul]
Actually you find all needed information on pod.gsi.de
PoD works out of the box! Most users won’t even need to know the location of PoD user defaults - configuration file
Since 2.1.2 contains important changes, including security fixes, it is very much recommended to start your next dynamic PROOF session with PoD 2.1.2.
BTW, it is about one month since GSI (Darmstadt) switched to a dynamic PROOF cluster using PoD, instead of a static one.
PoD is a toolset, which helps to unleash another powerful side of such a great product like PROOF.
In case of any problem or questions on this version please immediately contact to A.Manafov@gsi.de
You feedback will be always appreciated. The development of PoD is only driven by your feedback!
I am very glad to announce the release of PoD v2.1.3.
Some Highlights of this release
[ul]
[li]This version makes it possible to setup a native PROOF client/server connection in PoD environment, rather than to have it packet forwarded. if your workers are behind a firewall than PoD will automatically use its packet forwarding algorithms to maintain the PROOF traffic between server and workers. But if a direct connection is possible, than PoD will setup a native PROOF connection for such a workers. That means, by using PoD you can now setup a dynamic PROOF cluster and get a FULL functionality of PROOF on the native PROOF speed. For example you could now retrieve PROOF logs from a ROOT session or use PROOF’s new submerger functionality to speed up the merging phase + all other nice and useful PROOF features.[/li][/ul]
Minimally required version of ROOT is 5.25.04 or higher. Otherwise PoD will not be able to detect idle status when a native PROOF connection is used and will consider that your session is always idle and will shutdown it after a defined timeout.
I’ve just finished the first stable version of a TORQUE/PBS plug-in for PoD.
One more plug-in which will be shipped by default with PoD distributions, additionally to gLite and LSF.
Worth to mention, that the LSF plug-in is already in production for ALICE users at GSI.
From now on PoD user can also make use of PBS facilities and setup private PROOF clusters on the fly using PoD.
The plug-in will be released with PoD v2.1.4 and currently it is in a beta.
I am looking for test users, who has access to PBS clusters and want help and try/test PoD. Any feedback is appreciated.
If you want to test PoD (with pbs), just drop me an email on A.Manafov@gsi.de.
You can also just download the latest nightly and give it a try. More information here.
BTW, v2.1.4 will also come with a new command line interface. So now finally we have both, GUI and CLI
The next plug-in is already in development - it is SSH plug-in. With help of it, any machine on which you have ssh access can become your PROOF worker on demand.
This version is very much recommended to jump on.
It is not only brings you a faster startup time of workers, resolves several security issues, but also it brings an Oracle Grid Engine (OGE) support (in both PoD GUI and PoD CLI).
