Performace problems in updating a ROOT file

ROOT
1

Dear all,
I am trying to write a kind of acquisition system and I would like to directly use ROOT files to save the acquired data. I would like the system to produce “big” files (say 500MB each) but with some intermediate “small” update (say every 100MB), in order to enable another data monitoring program (ROOT-based) to read the intermediate data and show some plots.
I have tried the following (let me know if you need a working example):

TFile f("datafile.root","RECREATE");

TTree *tree1=new TTree(...........)
TTree *tree2=new TTree(...........)

// allocate all the needed branches
tree1->Branch(........)
tree2->Branch(........)

// acquisition loop
for(int iev=0;iev<MAX_EVENTS;iev++)
{
   // acquire data to tree variables
   // ...

   //tree fill
   tree1->Fill();
   tree2->Fill();

   if(iev%NEVENTS_UPDATE==0)
	{
		// update file for data monitoring program
		// no cycles
		tree1->Write(0, TObject::kOverwrite);
		tree2->Write(0, TObject::kOverwrite);
	}
}
f.Write();
f.Close();

I have noticed that the update step (i.e. the loop on all the trees to perform a write with the kOverwrite option) is really time-consuming and has a major impact on the overall performances… :frowning: The importance of this factor depends also on the used numbers of trees (linearly?).

Do you have any advice? Am I missing something?

Thank you!
Luigi

2

See the documenation at root.cern.ch/root/html/TTree.html#TTree:AutoSave

Cheers,
Philippe

3

Dear Philippe,
thank you for the link. Looking at the source code I have noticed that TTree::AutoSave in practice performs a Write() of the TTree, possibly with a kOverwrite option, i.e. exactly the same approach of my small example 8).
So the question remains: is this the fastest method for updating a TFile?

For example, if one writes the same data in a plain ugly binary file :unamused:, the update process (i.e. writing buffers on disk) is not such a big problem (for a given fast disk). So, where does ROOT “loose” some speed? Maybe we are simply paying the great flexibility of the format (that would be fine with me)?

Regards!
Luigi

4

When calling AutoSave ROOT writes all branch buffers and the tree header to the file.
If you have many branches and large buffers, this can take a lot of time.
It is better to use AutoSave in an automatic way (say after having written 200 MBytes, ie the default value). This is better that calling yourself AutoSave after N events.
When Autosave is called, it is guaranteed that you can always read the events written so far. You will not get incomplete events.

Rene

5

possibly with a kOverwrite option,this means that the list of keys does not increase due to the kept ‘cycles’…

Philippe