Hi,
I have been trying to get Python Selectors to run with PROOF-lite so I can run over larger ntuples. I’ve finally gotten it to work, but needed several work-arounds, which I thought would be worth posting. Of course it would be better if the underlying bugs can be fixed - or someone can tell me what I’m doing wrong in the first place.
Following the example in the TPySelector documentation, I tried:
proof = TProof.Open ('workers=4')
dataset.Process ('TPySelector', 'pyselect')
but that crashes (ROOT 5.28/00b, but the same problems are in 5.26).
The first problem is that user selector module (“pyselect” in my example) is not in the workers’ CWD, so cannot be loaded by the usual Python module import. My work-around was to add the current directory to the workers’ module load path:
proof.Exec ('TPython::Exec("%s");' % \
("import sys; sys.path.insert(0,'"+dirname(abspath("pyselect"))+"')"))
(the abspath() stuff is just so it also works with a file in another directory.) I guess a better solution would be to send the file to the worker (would work also with non-Lite PROOF), but I don’t know how to do that.
The next problem is that Python doesn’t seem to be correctly initialised on the workers, complaining about a missing $PYTHONHOME (this only happens if a non-default Python is used, not Python from under /usr). I can set PYTHONHOME like this:
ROOT.TProof.AddEnvVar ("PYTHONHOME", sys.prefix+":"+sys.exec_prefix)With that, my pyselect runs, but then crashes in the event loop. The traceback says it’s in TEventIterTree::GetTrees (TEventIter.cxx:519 in 5.28/00b). Since that code is only run if we have UseTreeCache set, the work-around is:
proof.SetParameter("PROOF_UseTreeCache", 0)I don’t know what the consequences of that setting are, but I didn’t see any performance problems (eg. comparing non-PROOF without it and PROOF-lite with one worker).
After all that, PROOF-Lite runs my Python Selector just fine. (Just for the record: I had another problem where the std.vector class was not functional on the workers, but that turned out to be specific to the ATLAS ROOT 5.26 installation. The work-around was to explicitly load it with gSystem->Load(“vector”).)
Can these bugs be fixed properly in a future version of ROOT?
Thanks,
Tim.