TFile Reading Non-linear

[NSos]

---

_ROOT Version: 5
_Platform: Scientific Linux 6.10
_Compiler: g++ 4.4.7

---

Hi everyone! I am trying to read a rather large TFile (50 GB) full of TTrees, reading and processing the TTrees one-by-one. I have been tracking the reading time and it appears to be ~quadratic. I attach the basic code for reading below. I think the quadratic nature of the reading comes from the data->Get(key->GetName()) line, where TFile will loop over everything to find the right TTree. I do not know of an alternative way to loop over all the TFile contents though, so any help with that would be appreciated!

#include <iostream>
#include <cmath>
#include <TFile.h>
#include <TTree.h>
#include <TKey.h>
#include <TStopwatch.h>

using namespace std;

int main(){

	int counter = 0;
	Double_t cumulative_time = 0;

	TStopwatch timer;
	timer.Start();

	TFile* data = TFile::Open("Data.root");
	TKey* key = 0; 
	TTree* event = 0;
	
	if(data->IsOpen()){

		TIter tree_iter(data->GetListOfKeys()); 

		while( (key = (TKey*)tree_iter()) ){ 

			event = (TTree*)data->Get(key->GetName());

			//Do stuff with TTree

			if(remainder(counter, 500) == 0){ //Time each 500 reads
	
				timer.Stop();
				cumulative_time += timer.RealTime();
				cout << cumulative_time << endl;
				timer.Start();
			}
		
			delete event;
		}

		data->Close();
	}

	return 0;
}

[Wile_E_Coyote]

Attach here the output of: data->ls();

[NSos]

It looks like this:

|  KEY: TTree|fisevent_40865_BRGG1_wf;1|fisevent_40865_BRGG1_wf|
|  KEY: TTree|fisevent_40865_BRGG2_wf;1|fisevent_40865_BRGG2_wf|
|  KEY: TTree|fisevent_40865_BRGG3_wf;1|fisevent_40865_BRGG3_wf|
|  KEY: TTree|fisevent_40865_BRGG4_wf;1|fisevent_40865_BRGG4_wf|
|  KEY: TTree|fisevent_40865_BRGG5_wf;1|fisevent_40865_BRGG5_wf|
|  KEY: TTree|fisevent_40865_BRGG6_wf;1|fisevent_40865_BRGG6_wf|
|  KEY: TTree|fisevent_40865_BRGG7_wf;1|fisevent_40865_BRGG7_wf|
|  KEY: TTree|fisevent_40865_BRGG8_wf;1|fisevent_40865_BRGG8_wf|
|  KEY: TTree|fisevent_40865_BRGG9_wf;1|fisevent_40865_BRGG9_wf|
|  KEY: TTree|fisevent_40865_BRGG10_wf;1|fisevent_40865_BRGG10_wf|
|  KEY: TTree|fisevent_40865_BRGG11_wf;1|fisevent_40865_BRGG11_wf|
|  KEY: TTree|fisevent_40865_BRGG12_wf;1|fisevent_40865_BRGG12_wf|
|  KEY: TTree|fisevent_40865_BRGG13_wf;1|fisevent_40865_BRGG13_wf|
|  KEY: TTree|fisevent_40865_BRGG14_wf;1|fisevent_40865_BRGG14_wf|
|  KEY: TTree|fisevent_40865_BRGG15_wf;1|fisevent_40865_BRGG15_wf|
|  KEY: TTree|fisevent_40865_HIPS17_wf;1|fisevent_40865_HIPS17_wf|
|  KEY: TTree|fisevent_40865_HIPS18_wf;1|fisevent_40865_HIPS18_wf|
|  KEY: TTree|fisevent_40865_HIPS19_wf;1|fisevent_40865_HIPS19_wf|
|  KEY: TTree|fisevent_40865_HIPS20_wf;1|fisevent_40865_HIPS20_wf|
|  KEY: TTree|fisevent_40865_HIPS21_wf;1|fisevent_40865_HIPS21_wf|
|  KEY: TTree|fisevent_40865_HIPS22_wf;1|fisevent_40865_HIPS22_wf|
|  KEY: TTree|fisevent_40865_HIPS23_wf;1|fisevent_40865_HIPS23_wf|
|  KEY: TTree|fisevent_40865_HIPS24_wf;1|fisevent_40865_HIPS24_wf|
|  KEY: TTree|fisevent_40865_HIPS25_wf;1|fisevent_40865_HIPS25_wf|
|  KEY: TTree|fisevent_40865_HIPS26_wf;1|fisevent_40865_HIPS26_wf|
|  KEY: TTree|fisevent_40865_HIPS27_wf;1|fisevent_40865_HIPS27_wf|
|  KEY: TTree|fisevent_40865_HIPS28_wf;1|fisevent_40865_HIPS28_wf|
|  KEY: TTree|fisevent_40865_HIPS29_wf;1|fisevent_40865_HIPS29_wf|
|  KEY: TTree|fisevent_40865_HIPS30_wf;1|fisevent_40865_HIPS30_wf|
|  KEY: TTree|fisevent_40865_BRGF_wf;1|fisevent_40865_BRGF_wf|
|  KEY: TTree|fisevent_40865_HIPF_wf;1|fisevent_40865_HIPF_wf|
|  KEY: TTree|fisevent_40865_FIFB_wf;1|fisevent_40865_FIFB_wf|
|  KEY: TTree|fisevent_40865_FIAB_wf;1|fisevent_40865_FIAB_wf|
|  KEY: TTree|fisevent_40866;1|fisevent_40866|
|  KEY: TTree|fisevent_40866_BRGG1_wf;1|fisevent_40866_BRGG1_wf|
|  KEY: TTree|fisevent_40866_BRGG2_wf;1|fisevent_40866_BRGG2_wf|
|  KEY: TTree|fisevent_40866_BRGG3_wf;1|fisevent_40866_BRGG3_wf|
|  KEY: TTree|fisevent_40866_BRGG4_wf;1|fisevent_40866_BRGG4_wf|
|  KEY: TTree|fisevent_40866_BRGG5_wf;1|fisevent_40866_BRGG5_wf|
|  KEY: TTree|fisevent_40866_BRGG6_wf;1|fisevent_40866_BRGG6_wf|
|  KEY: TTree|fisevent_40866_BRGG7_wf;1|fisevent_40866_BRGG7_wf|
|  KEY: TTree|fisevent_40866_BRGG8_wf;1|fisevent_40866_BRGG8_wf|
|  KEY: TTree|fisevent_40866_BRGG9_wf;1|fisevent_40866_BRGG9_wf|
|  KEY: TTree|fisevent_40866_BRGG10_wf;1|fisevent_40866_BRGG10_wf|
|  KEY: TTree|fisevent_40866_BRGG11_wf;1|fisevent_40866_BRGG11_wf|
|  KEY: TTree|fisevent_40866_BRGG12_wf;1|fisevent_40866_BRGG12_wf|
|  KEY: TTree|fisevent_40866_BRGG13_wf;1|fisevent_40866_BRGG13_wf|
|  KEY: TTree|fisevent_40866_BRGG14_wf;1|fisevent_40866_BRGG14_wf|
|  KEY: TTree|fisevent_40866_BRGG15_wf;1|fisevent_40866_BRGG15_wf|
|  KEY: TTree|fisevent_40866_HIPS17_wf;1|fisevent_40866_HIPS17_wf|

[Wile_E_Coyote]

I do not really see why you have a problem (unless you cut the data->ls(); output).

Instead of:

event = (TTree*)data->Get(key->GetName());

you could probably simply use:

event = (TTree*)key->ReadObj();

[NSos]

I did cut it, as it is enormous. I attached a dropbox link to the full log below. It is just more of the same though.

[NSos]

@Wile_E_Coyote

event = (TTree*)key->ReadObj();

I have tried ReadObj() and the entries-vs-time plot is now straight as an arrow! Completely linear. Thanks for the help!