CMSSW/Python won't write the ROOT output file

Hi,

I am trying to understand why CMSSW/python does not write the root output file onto the disk after processing the analyzer/job.
If I use the method:
TFile* output = new TFile(“output.root”,“RECREATE”);
output->cd();
for(map<string,TH1F*>::iterator it_histo = histosTH1F.begin();it_histo != histosTH1F.end(); ++it_histo)(*it_histo).second->Write();
output->Close();
it gives segmentation violation, no output.

If I use:
process.out = cms.OutputModule(“PoolOutputModule”,
fileName = cms.untracked.string(“data_55.root”)
)
process.ep = cms.EndPath(process.out)
it gives an inaccessible huge root file:
3112960 -rw-r–r--. 1 lregisem zh 3187671040 Dec 13 15:00 data_55.root

Here is an excerpt of the code:
// ----------member data ---------------------------

map<string,TH1F*> histosTH1F;
map<string,TH2F*> histosTH2F;

// ------------ method called for each event ------------
void PromptAnalyzer::analyze(const edm::Event& iEvent, const edm::EventSetup& iSetup)
{
// using namespace edm;
edm::Handle tracks;

double pt = itTrack->pt();
histosTH1F[“hpt”]->Fill(pt);

}
// ------------ method called once each job just before starting event loop ------------
void PromptAnalyzer::beginJob()
{
histosTH1F[“hpt”] = new TH1F(“hpt”,“p_{T}”,100,0,5);
//--------------end of my histograms
}
// ------------ method called once each job just after ending the event loop ------------
void PromptAnalyzer::endJob()
{
// Output file
TFile* output = new TFile(“output.root”,“RECREATE”);
output->cd();
for(map<string,TH1F*>::iterator it_histo = histosTH1F.begin();it_histo != histosTH1F.end(); ++it_histo)(it_histo).second->Write();
for(map<string,TH2F
>::iterator it_histo = histosTH2F.begin();it_histo != histosTH2F.end(); ++it_histo)(*it_histo).second->Write();
output->Close();
}

What is the right method of writing histograms into a file when using CMSSW/python?
histosTH1F[“hpt”] = new TH1F(“hpt”,“p_{T}”,100,0,5);
histosTH1F[“hm4”] = new TH1F(“hm4”, "M_{#pi^{+}#pi^{+}#pi^{-}#pi^{-}}
histosTH1F[“hpt”]->Fill(pt);
histosTH1F[“hm4”]->Fill(mrec4);

Is this ROOT method below valid with Python modules?
void PromptAnalyzer::endJob()
{
// Output file
TFile* output = new TFile(“output.root”,“RECREATE”);
output->cd();
for(map<string,TH1F*>::iterator it_histo = histosTH1F.begin();it_histo != histosTH1F.end(); ++it_histo)(it_histo).second->Write();
for(map<string,TH2F
>::iterator it_histo = histosTH2F.begin();it_histo != histosTH2F.end(); ++it_histo)(*it_histo).second->Write();
output->Close();
}

Would you please advise?

I appreciate,
Luiz
ps: detailed information of the code and python config file is in the attachment.


Please read tips for efficient and successful posting and posting code

ROOT Version: 6.14/09
Platform: LXPLUS7
Compiler: gcc (GCC) 8.3.1 20190225
___output-please.txt (7.1 KB)

Hello,

You report the following code is segfaulting:

 TFile* output = new TFile(“output.root”,“RECREATE”);
output->cd();
for(map<string,TH1F*>::iterator it_histo = histosTH1F.begin();it_histo != histosTH1F.end(); ++it_histo)
    (*it_histo).second->Write();
output->Close();

Can you check that the histograms you get from the map are not null? What happens if you just write a new histogram you create (not one from the map)?

@pcanal can perhaps have a look too.

Hi etejedor,

Ok, this is a separate histogram:

void PromptAnalyzer::beginJob()

{

TH1F* h1 = new TH1F(“h1”, “h1 title”, 100, 0.0, 4.0);
h1->FillRandom(“gaus”,10000);
}

There is no output.root either. I think the culprit is the demo module itself.

There is no compilation error but the module is broken for some reason that I could not figure out.

See the error:

[lregisem@lxplus739 submit-t200]$ more job_999.err
++ starting: constructing source: PoolSource
++++ starting: open input file: lfn = root://eostotem//eos/totem/data/cmstotem/2018/90m/RECO_copy/TOTEM20/7000
1/682DFA26-F03B-E911-85AB-A0369FC5DF98.root
15-Dec-2020 16:09:59 CET Initiating request to open file root://eostotem//eos/totem/data/cmstotem/2018/90m/RE
CO_copy/TOTEM20/70001/682DFA26-F03B-E911-85AB-A0369FC5DF98.root
%MSG-w XrdAdaptor: file_open 15-Dec-2020 16:10:03 CET pre-events
Data is served from cern.ch instead of original site eoscms
%MSG
15-Dec-2020 16:11:01 CET Successfully opened file root://eostotem//eos/totem/data/cmstotem/2018/90m/RECO_copy
/TOTEM20/70001/682DFA26-F03B-E911-85AB-A0369FC5DF98.root
++++ finished: open input file: lfn = root://eostotem//eos/totem/data/cmstotem/2018/90m/RECO_copy/TOTEM20/7000
1/682DFA26-F03B-E911-85AB-A0369FC5DF98.root
++ finished: constructing source: PoolSource
++++ starting: constructing module with label ‘TriggerResults’ id = 1
++++ finished: constructing module with label ‘TriggerResults’ id = 1
++++ starting: constructing module with label ‘p’ id = 2
++++ finished: constructing module with label ‘p’ id = 2
++++ starting: constructing module with label ‘demo’ id = 3
++++ finished: constructing module with label ‘demo’ id = 3
++ preallocate: 1 concurrent runs, 1 concurrent luminosity sections, 1 streams
++ starting: begin job
++++ starting: begin job for module with label ‘demo’ id = 3
++++ finished: begin job for module with label ‘demo’ id = 3
++++ starting: begin job for module with label ‘TriggerResults’ id = 1
++++ finished: begin job for module with label ‘TriggerResults’ id = 1
++++ starting: begin job for module with label ‘p’ id = 2
++++ finished: begin job for module with label ‘p’ id = 2
++ finished: begin job
++++ starting: begin stream for module: stream = 0 label = ‘demo’ id = 3
++++ finished: begin stream for module: stream = 0 label = ‘demo’ id = 3
++++ starting: begin stream for module: stream = 0 label = ‘TriggerResults’ id = 1
++++ finished: begin stream for module: stream = 0 label = ‘TriggerResults’ id = 1
++++ starting: begin stream for module: stream = 0 label = ‘p’ id = 2
++++ finished: begin stream for module: stream = 0 label = ‘p’ id = 2
++++ starting: source run
++++ finished: source run
++++ starting: global begin run 319270 : time = 6575030674553435136
++++ finished: global begin run 319270 : time = 6575030674553435136
++++ starting: begin run: stream = 0 run = 319270 time = 6575030674553435136
++++ finished: begin run: stream = 0 run = 319270 time = 6575030674553435136
++++ starting: source lumi
++++ finished: source lumi
++++ starting: global begin lumi: run = 319270 lumi = 24 time = 6575030674553435136
++++ finished: global begin lumi: run = 319270 lumi = 24 time = 6575030674553435136
++++ starting: begin lumi: stream = 0 run = 319270 lumi = 24 time = 6575030674553435136
++++ finished: begin lumi: stream = 0 run = 319270 lumi = 24 time = 6575030674553435136
++++ starting: source event
++++ finished: source event
Begin processing the 1st record. Run 319270, Event 34178124, LumiSection 24 on stream 0 at 15-Dec-2020 16:11:1
8.667 CET
++++ starting: processing event : stream = 0 run = 319270 lumi = 24 event = 34178124 time = 657503067884776243
2
++++++ starting: processing path ‘p’ : stream = 0
++++++++ starting: prefetching before processing event for module: stream = 0 label = ‘demo’ id = 3
++++++++++ starting: event delayed read from source: stream = 0 label = ‘demo’ id = 3
++++++++++ finished: event delayed read from source: stream = 0 label = ‘demo’ id = 3
++++++++++ starting: event delayed read from source: stream = 0 label = ‘demo’ id = 3
++++++++++ finished: event delayed read from source: stream = 0 label = ‘demo’ id = 3
++++++++++ starting: event delayed read from source: stream = 0 label = ‘demo’ id = 3
++++++++++ finished: event delayed read from source: stream = 0 label = ‘demo’ id = 3
++++++++++ starting: event delayed read from source: stream = 0 label = ‘demo’ id = 3
++++++++++ finished: event delayed read from source: stream = 0 label = ‘demo’ id = 3
++++++++++ starting: event delayed read from source: stream = 0 label = ‘demo’ id = 3
++++++++++ finished: event delayed read from source: stream = 0 label = ‘demo’ id = 3
++++++++ finished: prefetching before processing event for module: stream = 0 label = ‘demo’ id = 3
++++++++ starting: processing event for module: stream = 0 label = ‘demo’ id = 3

A fatal system signal has occurred: segmentation violation
The following is the call stack containing the origin of the signal.

Current Modules:

Module: PromptAnalyzer:demo (crashed)

A fatal system signal has occurred: segmentation violation
condor_exec.exe: line 7: 72 Segmentation fault cmsRun /afs/cern.ch/user/l/lregisem/CMSSW_10_6_18/src/P
romptLUAna/PromptAnalyzer/python/submit-t200/job_999.py

cp: cannot stat ‘output.root’: No such file or directory

I am also compiling the code with -g option.

Any clue ?

Thanks
Luiz

Hello,

If I understand correctly, in PromptAnalyzer::endJob() is where you are supposed to create a ROOT file and write whatever you want in it. That does not happen even if you just create the file and close it? It would be good to know where the segmentation violation is happening, since that might be preventing the code that generates the file from even starting to run.

The demo module is something provided by CMSSW?

Perhaps @pcanal knows more about this.

Hi etejedor,

Using the TFileService method I do get the root output file:
lregisem@lxplus748 submit-t200]$ ls -las *.root
1 -rw-r–r--. 1 lregisem zh 339 Dec 16 20:02 data_1899.root
1 -rw-r–r--. 1 lregisem zh 339 Dec 16 20:02 output.root

Checking the content of the root file:
[lregisem@lxplus748 submit-t200]$ root -l data_1899.root
root [0]
Attaching file data_1899.root as _file0…
Warning in TFile::Init: file data_1899.root probably not closed, trying to recover
Info in TFile::Recover: data_1899.root, recovered key TDirectoryFile:demo at address 232
Warning in TFile::Init: successfully recovered 1 keys
(TFile ) 0x1f14290
root [1] .ls
TFile
* data_1899.root
TFile* data_1899.root
KEY: TDirectoryFile demo;1 demo
root [2] demo->cd();
root [3] .ls
TDirectoryFile* demo demo
root [4]

It turns out that there is no histogram even not booked.
The job_1899.err file reports the following:

Begin processing the 1st record. Run 319263, Event 443628707, LumiSection 307 on stream 0 at 16-Dec-2020 20:02:
07.482 CET
++++ starting: processing event : stream = 0 run = 319263 lumi = 307 event = 443628707 time = 6574892290707236352
++++++ starting: processing path ‘p’ : stream = 0
++++++++ starting: prefetching before processing event for module: stream = 0 label = ‘demo’ id = 3
++++++++++ starting: event delayed read from source: stream = 0 label = ‘demo’ id = 3
++++++++++ finished: event delayed read from source: stream = 0 label = ‘demo’ id = 3
++++++++++ starting: event delayed read from source: stream = 0 label = ‘demo’ id = 3
++++++++++ finished: event delayed read from source: stream = 0 label = ‘demo’ id = 3
++++++++++ starting: event delayed read from source: stream = 0 label = ‘demo’ id = 3
++++++++++ finished: event delayed read from source: stream = 0 label = ‘demo’ id = 3
++++++++++ starting: event delayed read from source: stream = 0 label = ‘demo’ id = 3
++++++++++ finished: event delayed read from source: stream = 0 label = ‘demo’ id = 3
++++++++++ starting: event delayed read from source: stream = 0 label = ‘demo’ id = 3
++++++++++ finished: event delayed read from source: stream = 0 label = ‘demo’ id = 3
++++++++ finished: prefetching before processing event for module: stream = 0 label = ‘demo’ id = 3
++++++++ starting: processing event for module: stream = 0 label = ‘demo’ id = 3
A fatal system signal has occurred: segmentation violation
The following is the call stack containing the origin of the signal.
Current Modules:
Module: PromptAnalyzer:demo (crashed)
A fatal system signal has occurred: segmentation violation
condor_exec.exe: line 7: 72 Segmentation fault cmsRun /afs/cern.ch/user/l/lregisem/CMSSW_10_6_18/src/Pr
omptLUAna/PromptAnalyzer/python/submit-t200/job_1899.py

The analyze method is supposed to fill the histograms:

for(TrackCollection::const_iterator itTrack = tracks->begin();itTrack != tracks->end();++itTrack) {
double pt = itTrack->pt();

histosTH1F[“hpt”]->Fill(pt); }

The beginJob method creates the histograms:


edm::Service fs;
int nbins_pt = 100;
histosTH1F[“hpt”] = fs->make(“hpt”,“p_{T}”,nbins_pt,0,5);

It seems the demo module crash is at the very beginning of the running process so no booked histograms are seen.
But the root output file has been created.

I guess TFileServie is supposed to close the output.root. But that is not happening.

Do you have a document on the TFileService method?

Thanks
Luiz Emediato

Hello,

I believe TFileService is something provided by CMSSW?

I googled and found this:

Perhaps there is another forum that provides support about that tool?

Hi etejedor,

Ok, thanks for the link.

You can close this subject.

Best,
Luiz

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.