TFile::Append:0: RuntimeWarning: Replacing existing TH1 in PyROOT


ROOT Version: 6.04
Platform: Centos 6.4 (final)
Compiler: Not Provided


Hello,

I can’t seem to get rid of these errors using pyroot:

TFile::Append:0: RuntimeWarning: Replacing existing TH1

I do delete my hist objects, but I suspect that there is some linking problem related to saving a cloned copy of the histogram into a dictionary.

The snippet of code that throws this warning is as follows:

for key in r_file.GetListOfKeys():
    key_obj = key.ReadObj()
    name = key.GetName()

    # This case, key will be a tree like nominal_tZ or nominal_Loose_data etc
    if from_tree != "":

        if (from_tree not in name) or ("TTree" not in key_obj.ClassName()):
            continue

        histname = name.replace(from_tree, "h")

        use_bins = list(self.config["binning"])
        use_bins[bound_shift[0]] += bound_shift[1]
        binEdges = array.array('d', use_bins)

        hist = ROOT.TH1D(histname, histname, len(binEdges) - 1, array.array('d', binEdges))
        for i in range(key_obj.GetEntries()):
            key_obj.GetEntry(i)
            hist.Fill(
                key_obj.GetBranch("LeptonW_pt").GetLeaf("LeptonW_pt").GetValue(),
                key_obj.GetBranch("weight_Total").GetLeaf("weight_Total").GetValue()
            )
        hlist[histname] = hist.Clone(histname)
        del hist
        del key_obj    

All of the solutions I can find on forums apply to C++ based root, and I can’t seem to make them work in pyroot. Is there something that I am missing that I need to do to avoid this behavior?

Thanks,
-Alex

Hello @adsherman,

I believe your warning comes from this line:

hlist[histname] = hist.Clone(histname)

Here you are cloning a histogram into a new one with the same name, so internally ROOT will delete the old histogram and register the new one with the same name. Could you try with:

hlist[histname] = hist

instead?
This will keep a reference to hist and thus make it survive the loop.

Also, please note that you do not need to explicitly delete Python objects, they will be garbage collected when they go out of scope. For example these are not necessary:

        del hist
        del key_obj

Cheers,
Enric

Hi Enric,

I changed to hlist[histname] = hist but with no luck, the error remains the same. I think I need to clear the TH1 somehow in the non-python root memory, but del doesn’t get at it like delete in C++.

Also, it does not throw the error the first pass through. I call this loop on 3 different files and it only flashes the error for the 2nd and 3rd:

INFO (06:31:43) : <ReadRootFile>: Opening r_file /scratch/adsherma/tZqAnalysis/NtupleReader/20180922_scaletest_default//FR-TTT_VR-2j0b_el/FR-TTT_VR-2j0b_el.root
INFO (06:31:43) : <ReadRootFile>: Opening r_file /scratch/adsherma/tZqAnalysis/NtupleReader/20180922_scaletest_default//FR-LTT_VR-2j0b_el/FR-LTT_VR-2j0b_el.root
INFO (06:31:43) : <ReadRootFile>: Opening r_file /scratch/adsherma/tZqAnalysis/NtupleReader/20180922_scaletest_default//FR-ZPlusJets-LTT_VR-2j0b_el/FR-ZPlusJets-LTT_VR-2j0b_el.root
INFO (06:31:43) : <GetHistoList>: Getting files from /scratch/adsherma/tZqAnalysis/NtupleReader/20180922_scaletest_default//FR-TTT_VR-2j0b_el/FR-TTT_VR-2j0b_el.root
INFO (06:31:46) : <GetHistoList>: Getting files from /scratch/adsherma/tZqAnalysis/NtupleReader/20180922_scaletest_default//FR-LTT_VR-2j0b_el/FR-LTT_VR-2j0b_el.root
TFile::Append:0: RuntimeWarning: Replacing existing TH1: h_ttWZH (Potential memory leak).
TFile::Append:0: RuntimeWarning: Replacing existing TH1: h_data (Potential memory leak).
TFile::Append:0: RuntimeWarning: Replacing existing TH1: h_other (Potential memory leak).
TFile::Append:0: RuntimeWarning: Replacing existing TH1: h_tZ (Potential memory leak).
TFile::Append:0: RuntimeWarning: Replacing existing TH1: h_diboson (Potential memory leak).
INFO (06:31:47) : <GetHistoList>: Getting files from /scratch/adsherma/tZqAnalysis/NtupleReader/20180922_scaletest_default//FR-ZPlusJets-LTT_VR-2j0b_el/FR-ZPlusJets-LTT_VR-2j0b_el.root
TFile::Append:0: RuntimeWarning: Replacing existing TH1: h_ttWZH (Potential memory leak).
TFile::Append:0: RuntimeWarning: Replacing existing TH1: h_data (Potential memory leak).
TFile::Append:0: RuntimeWarning: Replacing existing TH1: h_other (Potential memory leak).
TFile::Append:0: RuntimeWarning: Replacing existing TH1: h_tZ (Potential memory leak).

Thanks,
-Alex

Hi @adsherman,

Is it possible that you are creating histograms with the same name in different iterations at this line:

        hist = ROOT.TH1D(histname, histname, len(binEdges) - 1, array.array('d', binEdges))

That would explain why you still get the warning. Since you keep your histograms in a dictionary, they are not deleted by Python. If you create a histogram with the same name as one you already have in the dictionary, ROOT will throw that warning.

Cheers,
Enric

Hi Enric,

Does ROOT have access to the dictionary outside of the scope of that method? I declare the dictionary as a freshly initialized dictionary at the beginning of each call to this method. In native python I am 95% sure it should not have access to this information anymore, and so I think it is a ROOT related behavior entirely. I just would like to clear it somehow between calls so I dont get this warning!

Thanks,
-Alex

Hi @adsherman,

No, ROOT does not have access to your dictionary. What happens here is that histograms get internally registered by ROOT based on their names. This happens on the C++ side. The warning you see comes as a consequence of creating a histogram with some name before another histogram with the same name has been deleted.

The situation you are facing happens e.g. if you write:

hist = ROOT.TH1D(histname, histname, len(binEdges) - 1, array.array('d', binEdges))
hist = ROOT.TH1D(histname, histname, len(binEdges) - 1, array.array('d', binEdges))

This is why I was asking if you are sure no two histograms with the same name are being created without the first one being deleted first.

Cheers,
Enric

Hi Enric,

Yes this is what I meant. So I can not have two histograms with the same names and different contents in different dictionaries at the same time? I need to compare 3 simultaneously and it makes my life much much easier if they have the same name and are kept in 3 different dictionaries.

Thanks,
-Alex

Hi Alex,

You can also do this:

hist = ROOT.TH1D(histname, histname, len(binEdges) - 1, array.array('d', binEdges))
hist.SetDirectory(0)

so that the histograms are not added to the internal registry.

Cheers,
Enric

Hi Enric,

This did it! I will start putting this everywhere!

Thanks,
-Alex

You can also call

ROOT.TH1.AddDirectory(false);

to make this the default behavior.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.