PyRoot hangs on TFile.Close() method again

Andrius · April 8, 2019, 5:05pm

Hi,

I seem to be having a problem that I used to have before but the offered solution no longer works. These are the details: Pyroot hangs on TFile.Close() method

I’m using this (CMSSW) release of root: https://github.com/cms-sw/root/tree/cms/v6-12-00-patches/3f31cef

I should also add that the number of histograms in the files are in the order of ten thousands.

Perhaps there were some recent changes that could have caused this issue?

ROOT Version: 6.12/07
Platform: SLC 7
Compiler: gcc700

Andrius · April 8, 2019, 5:07pm

The code that i’m experiencing the issue with:

github.com

cms-sw/cmssw/blob/d54ae3ab15799634aa5fbfc88c8a7e7c2d207618/DQMServices/FileIO/scripts/compareHistograms.py

#!/bin/env python

from __future__ import print_function
import ROOT
ROOT.PyConfig.IgnoreCommandLineOptions = True
import os
import sys
import argparse
import root_numpy
import numpy as np
from blacklist import get_blacklist

def create_dif(base_file_path, pr_file_path, pr_number, test_number, cmssw_version, output_dir_path):
   base_file = ROOT.TFile(base_file_path, 'read')
   pr_file = ROOT.TFile(pr_file_path, 'read')

   if base_file.IsOpen():
      print('Baseline file successfully opened', file=sys.stderr)
   else:
      print('Unable to open base file', file=sys.stderr)

This file has been truncated. show original

Would have added it in the main message but I can only add two links.

pcanal · April 8, 2019, 6:21pm

It could be that you also need:

ROOT.gROOT.GetListOfFiles().Remove(result_file)

Cheers,
Philippe.

Andrius · April 9, 2019, 9:05am

Hi Philippe,

That’s exactly what I tried but it still takes significantly longer to close the files. This problem used to occur only on files opened for reading and the workaround still works perfectly for that. But now I only experience this issue with files open for writing, and the workaround doesn’t work anymore. It still takes huge amount of time to close the files.
This is the updated code that I’m using:

github.com

cms-sw/cmssw/blob/8db59832a30daecc2b74d8913106ff4dc7347fab/DQMServices/FileIO/scripts/compareHistograms.py

#!/bin/env python

from __future__ import print_function
import ROOT
ROOT.PyConfig.IgnoreCommandLineOptions = True
import os
import sys
import argparse
import root_numpy
import numpy as np
from blacklist import get_blacklist

def create_dif(base_file_path, pr_file_path, pr_number, test_number, cmssw_version, output_dir_path):
   base_file = ROOT.TFile(base_file_path, 'read')
   pr_file = ROOT.TFile(pr_file_path, 'read')

   if base_file.IsOpen():
      print('Baseline file successfully opened', file=sys.stderr)
   else:
      print('Unable to open base file', file=sys.stderr)

This file has been truncated. show original

pcanal · April 17, 2019, 3:39pm

Hi Andrius,

I see the problem. The pr_file is Closed while the other two files (and their many histograms) are still (indirectly) on the list of cleanups.

To solve the problem, just move the GetListOfFiles().Remove right after the TFile creations.

Cheers,
Philippe.

system · May 1, 2019, 3:39pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.