TFile::Close Slow

[kkrizka]

Hello,

I am seeing that the TFile::Close function takes a long time when closing a TFile with many histograms. The TFile::Write function, which I call first, finishes within a second.

My workflow consists of running over an ntuple once and making several versions of the same histogram for events passing different selections (ie: changing pT cuts, different b-tagging working points). The output is stored in a ROOT file with the following structure:

selection1/histogram1
selection1/histogram2
selection2/histogram1
selection2/histogram2
selection3/histogram1
selection3/histogram2
...

It allows me to quickly compare histograms between alternate selections to understand their effect.
I would like to understand why the close operation is taking so long. I am increasing the number of selections (testing different b-tagging working points) and I am seeing the closing of the TFile take hours.

I’ve attached an example demonstrating the problem. It creates 200 directories (my selections also include binning in different variables, which is why this number is so large) with 200 histograms each. The result is one second to TFile::Write and over a minute to TFile::Close. If I double the number of directories, the closing takes 6 minutes.

closeFileTest.cpp (925 Bytes)

The example code can be run with can be run with

g++ closeFileTest.cpp -o closeFileTest
 $(root-config --cflags --libs )
./closeFileTest

[pcanal]

Hi,

It is plausible that the slowdown will be reduced if you use:

gROOT->GetListOfFiles()->Remove(file);

However, you also need to make absolutely sure that you have no objects/pointers shared amongst any of the files (this should be the case if you only have histograms) and you need to explicitly delete the file objects.

Cheers,
Philippe.

PS. The slowdown is likely coming from the fact that to avoid double delete in a file, each object being deleted inform all the objects directories and objects of its deletion (so there is an N^2 problem).

[kkrizka]

Hi Philippe,

Thank you for the quick response.

Sadly I don’t think this solution will work for me, because the TFile::Write + TFile::Close sequence happens in a common framework used by ATLAS. So the actual closing of the file is out of my control.

Should I just interpret this as an effective limit on the number of objects a TFile can store being ~100k?

–
Karol Krizka

[pcanal]

Hi Karol,

Since the Write/Close is done by Athena, I expect that they also take cares of the delete (instead of relying on ROOT to do it).

In this case (and if your file only contains histograms or the like), it is safe for you to do the gROOT->GetListOfFiles()->Remove(file); at anytime.

Cheers,
Philippe.