Pyroot hangs on TFile.Close() method

I’m trying to loop over all objects inside a 2 root files and close them in the end by using TFile.Close() and the program hangs on Close() method of the first file. So these are the steps of my program:

  • Open file1.root and file2.root
  • Loop through every histogram recursively inside both files by using key.ReadObj()
  • Create two new root files and copy some of the histograms from both initial files. To fetch the histograms from initial files I use: object.FindObject(dir) and then I call histogram.Write() for every histogram I want to be written to result file.
  • Close both result files which works fine
  • Try to close both input files but program hangs on the first Close() call

Based on some debugging that I did I can see that whenever I stop in GDB the bottom most call is usually a RecursiveRemove(). is this normal behaviour or is there something wrong? How long should the process of closing the file take?


_ROOT Version: 6.12/07
_Platform: SLC6
_Compiler: gcc700
_Root comes from: CMSSW_10_4_0


Hi Andrius,

hard to say exactly without a reproducer. Can you share the code and the input files? Since you are dealing with histograms, any ROOT version can be used, irrespective of the CMSSW environment.

Is the content of both new files ok despite the issue at tear down? I ask because some care must be taken if you rely on ROOT’s implicit memory management.

Cheers,
D

Hi dpiparo,

Yes, the output of the produced files is absolutely fine.

I will create a minimal working example with data first thing tomorrow in the morning!

Regards,
Andrius

Hi dpiparo,

I did some more debugging and the Close() call seems to terminate but it takes a while (in the order of hours). The time that it takes for a file to close seems to depend quadratically on the number of histograms read from it. When reading 50k histograms it took ~30m to close the file and when reading 100k histograms the time was ~2h. Initial files were exactly the same. I made a minimal working example which you can find here:

/afs/cern.ch/user/a/akirilov/public/dpiparo

Regards,
Andrius

Hi dpiparo,

Any news on this? What if I just os._exit() after I’m finished with my script?

Regards,
Andrius

You can work around this (if you are sure there is not multiple reference within the file to the same histogram) by doing:

gROOT->GetListOfFiles()->Remove( myfileptr );

Cheers,
Philippe.

Hi pcanal,

Thanks a lot for the recommendation, it worked perfectly. This is a (trivial) python version of the code you provided in case somebody else stumbles upon it: gROOT.GetListOfFiles().Remove(myfile).

Regards,
Andrius

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.