(This is part of another thread (Prevent PROOF from splitting files), but since the topic changed, I will create a thread with a more precise topic)
I am having problems using a TEntryList together with PROOF (Lite).
I am working with PROOF Datasets, therefore i use the Process method of the PROOF class in the following way:
p->Process( dsname , "D3PDSelector.C+", "", -1, 0, tel); //tel being the ENtryList with content shown below
Like that, PROOF crashes. If i reformulate the code to:
POOF, doesn’t crash anymore, but shows the same errors:
Info in <TPacketizerAdaptive::TPacketizerAdaptive>: no valid or non-empty file found: setting invalid
Error in <TProofPlayerLite::InitPacketizer>: instantiated packetizer object 'TPacketizerAdaptive' is invalid
Error in <TProofPlayerLite::Process>: cannot init the packetizer
Info in <TProofLite::UpdateDialog>: processing was aborted - 0 events processed
the content of the TEntryList is ( Print(“all”) )
susy file:///terabig/ctopfel/PROOF/D3PDAnalysis/DataSetSymLinkRepository/user10.GeorgeRedlinger.SUSYD3PD.mc09_7TeV.106400.SU4_jimmy_susy.merge.AOD.e496_s765_s767_r1250_r1260.100505.1/user10.GeorgeRedlinger.SUSYD3PD.mc09_7TeV.106400.SU4_jimmy_susy.merge.AOD.e496_s765_s767_r1250_r1260.100505.1.D3PD._00003.root
12
16
22
.. (and so on, 3 files with many entries)
The crash happens because, for some reason, the information in the TEntryList is not
matched to the one in the dataset. (Of course it should not crash; there is a probably a
protection missing somewhere).
The second case in this thread
does not work just because the TDSet object is empty (no files).
So, we have to understand why the first (correct) case does not work.
To that purpose, can please provide the following information:
Which ROOT version you are running
As many details as you can about the creation of the TEntryList
If you process locally the files in a TChain, does the thing work?
I am using root version 5.26 and 5.27 (sorry, should have told that in the initial post)
I am basically using what is available in h1analysis.C and h1analysis.h (the tutorials). I copied the relevant pieces of code. (But I think you are right that the error must be in the creation of the EntryList.)
At the moment this results in a strange thing: If I look at the TEntryList with Print(“All”) I get all the files double, but only one kind (the ones with file:///) carry events. Here is a snippet from the output.
This leads to
3. If I run with TChain and without proof (As you suggested above), it works, BUT takes no events (I assume it takes the file entries above without event entries). If I run the same piece of code with PROOF ( c->SetProof() ), I get (the same as when running on datasets directly)
Info in <TPacketizerAdaptive::InitStats>: no valid or non-empty file found: setting invalid
Error in <TProofPlayerLite::InitPacketizer>: instantiated packetizer object 'TPacketizerAdaptive' is invalid
Error in <TProofPlayerLite::Process>: cannot init the packetizer
Info in <TProofLite::UpdateDialog>: processing was aborted - 0 events processed
I will post code snippets about the creation of the TEntryList later (will double and triple-check vs the h1analysis-files), but maybe you already have an idea about this double-occurence of files in the EntryList.
Let me also say at this point how much I appreciate the help you are providing here!
//IN D3PDSelector::Begin
elist = new TEntryList("elist", "H1 selection from Cut");
// Add to the input list for processing in PROOF, if needed
if (fInput)
{
fInput->Add(new TNamed("fillList",""));
fInput->Add(elist);
}
//IN D3PDSelector::SlaveBegin
if ((elist = (TEntryList *) fInput->FindObject("elist")))
elist = (TEntryList *) elist->Clone();
if (elist)
fOutput->Add(elist);
//IN D3PDSelector::Terminate
m_elist = dynamic_cast<TEntryList*>(fOutput->FindObject("elist"));
if (m_elist) {
TFile efile(trgindexfile.c_str(),"recreate");
//elist->SetName(dsn.c_str());
m_elist->Write();
} else {
Error("Terminate", "entry list requested but not found in output");
}
//IN D3PDSelector::Notify
elist->SetTree(fChain);
//At some point int the Analysis-Code
elist->Enter(m_curentry);
The fact that it does not work locally with a TChain (it does not crash but it does not process
the entries that you expect) is suspicious.
But I do not see anything wrong in the way you fill the list.
Still, I believe the problem is in the matching between entrylist elements and dataset elements.
How do you specify the files in the TChain or TDSet object? I mean, “file:///terabig/ctopfel …” or
just “/terabig/ctopfel …”? You can do, for example, TDSet::Print(“a”) and you get the full internal representation.
Actually I am using the PROOF facility for managing datasets, I’m not instantiating a TDSet.
Here is an excerpt from the python script I am using:
coll = TFileCollection()
coll.SetDefaultTreeName("/susy")
for file in os.listdir(dir):
if require_string in file and ".root" in file:
thfinf = TFileInfo(dir + "/" + file) # <- this will result in /terabig/ctopfel... etc
#print "Adding file " + file
coll.Add(thfinf)
#print "Created collection"
proof.RegisterDataSet(DSName, coll)
later I’m using the following code to process the added dataset:
TProof * p = TProof::Open("");
p->Process("mydatasetname", "D3PDSelector.C+");
Can it be that, when adding files via TFileInfo objects, I should use “file:///” in order for it to work?