Histogram filling with ~zero when using PROOF, correctly without PROOF

[jsurb]

Hi ROOTers,

I’ve been having intermittent difficulty with a TSelector class that sorts my data and writes an output file containing a number of histograms for subsequent analysis. When using PROOF, I am left with a set of histograms that have a large (probably correct) number of entries, but incorrect values that are all near zero (mean ~1e-307, RMS =0). I say intermittent, as only some of my histograms are giving me difficulty, whereas others are fine. The troublesome histograms are all accessed by the same TH1D pointer. Other arrays of histograms behave as expected. I get no errors while running with the compiled class.

When I remove PROOF and use a limited data-set, these histograms fill with non-zero, reasonable values. When printed to screen, these values too seem reasonable.

I will work to strip down the code to just the troublesome histograms and edit this original post or reply with my code. I wanted to quickly post without the code first, in case I’m missing something easy or obvious to any of you.

Thank you for reading and any help you have to give!
-jsurb

---

ROOT Version: 6.04/02
Platform: Not Provided
Compiler: Not Provided

---

[ganis]

Hello,

It is basically impossible to say anything without looking at the code.
So, please, post the parts of the code where you define the histograms, add them to fOutput, and where you retrieve the histograms in Terminate.

G Ganis

[jsurb]

Hi G Ganis,

Thank you for the fast response. In the header, before the initialization of the TChain and related/branch variables, I define my array of histograms:

TH1D *hGamma[16];

In the .C file, under ::Begin()

for(Int_t segaNo = 0;segaNo<16;segaNo++){
      hGamma[segaNo] = 0;
}

under ::SlaveBegin()

for(Int_t segaNo=0;segaNo<16;segaNo++){
     hGamma[segaNo] = new TH1D(Form("SeGA%d",segaNo),Form("SeGA%dSpectra",segaNo),9000,0,9000);
     hGamma[segaNo]->GetXaxis()->SetTitle("Energy (keV)");
     hGamma[segaNo]->GetYaxis()->SetTitle(Form("counts/%0.1f keV",(hGamma[segaNo]->GetBinWidth(5))));
     fOutput->Add(hGamma[segaNo]);
}

under ::Process(), (SeGAenergy[i] is defined in the Tree)

for(Int_t segaNo=0;segaNo<16;segaNo++){
    if(SeGAenergy[segaNo]){
      eGamma = SeGAcal[segaNo][0]+SeGAcal[segaNo][1]*SeGAenergy[segaNo];
      hGamma[segaNo]->Fill(eGamma);
      //std::cout << eGamma << std::endl;
    }
}

Under ::Terminate(), I write to disk without additional processing.

TFile *myfile = new TFile("./rootfiles/tselector.root","RECREATE");
fOutput->Write();//Write all the histograms to disk
myfile->Close();

[jsurb]

I’m now starting to think it’s related to how I read in the calibration used in ::Process(). I use

std::ifstream infile("./tabularData/SeGA-calibration.txt");
Int_t a;
Double_t b,c;
Double_t SeGAcal[16][2];

while(infile >> a >> b >> c){
    SeGAcal[a][0] = b;
    SeGAcal[a][1] = c;
  }

If I hard code the calibration (SeGAcal) or serial-run as-is, these histograms fill fine. Are there known issues with reading from a text file with a PROOF scheme?

[jsurb]

I’ve found a way around my problem, albeit, with a solution I think is a little contrived. The reasoning I have of my problem is that in my PROOF session, I had many threads trying to access the same calibration file concurrently, and this lead to failure in reading the calibration values, giving me the nonsensical histograms. I don’t know enough to know this is the reasoning for sure.

My workaround is to create a TEnv object containing my calibration values in the PROOF launcher script. I pass this TEnv object to the PROOF session after creation and before having it run my TSelector code on my data chain. This new calibration file has the same numbers, but is formatted differently to allow TEnv functions to use it. I add the following in SlaveBegin()

if(fInput->FindObject("TEnv")){
     // Should be found in PROOF session
     SeGAenv = dynamic_cast<TEnv*>(fInput->FindObject("TEnv"));
   }else{
     // Should only run in serial (non-PROOF)
     SeGAenv = new TEnv("./tabularData/SeGA-Calibration.txt");
   }

This way, if using PROOF, I just lookup the environment and if running in a serial scheme, it defines it here. In process, I access the the values with something like:

cal0 = SeGAenv->GetValue(Form("SeGA.%d.slope",segaNo),-1.0);
cal1 = SeGAenv->GetValue(Form("SeGA.%d.inter",segaNo),-1.0);

With this code, I read the calibration file only once, from a single thread, to avoid any competition that may have been causing the issue before. This gives good looking, calibrated, histograms.

[ganis]

Hello,
Sorry for the late reply.
Yes, the problem was with the input file.
You had a relative path in ::Process, and the worker processes have working directories different from the one were you start ROOT. The solution you have found is a good one, in particular if the file is not so large. Otherwise, if you are using PROOF-Lite, as I think, it should work by replacing the relative path by an absolute path.

G Ganis

[jsurb]

Hi G Ganis,

Thanks for the explanation. I didn’t think about the file paths, but yes, they are/were relative paths.

Regards,
jsurb