Problem with object copy in python

Hello,

I’m working on a project of ROOT commandline tools for unix box (I code in python) and I’m facing a problem with my command to copy objects from a ROOT file to another.
I actually use the code of Rene Brun, available on the ROOT tutorial here : https://root.cern.ch/root/htmldoc/tutorials/io/copyFiles.C.html which I translate into python.
Both of the code works on ‘light file’, but when I tested them on a ‘too big file’, actually just 100Mo, it happens strange things.
While the C++ code takes 20 minutes to copy this file and 4 minutes to close the 2 files, the python code takes 3 minutes to copy this file and then never leave the TFile.Close() method at the end of the program (actually it leaves the program but after a very long time, i’m sure how much for the moments, but I’m testing this).

the C++ macro

void CopyDir(TDirectory *source) {
   //copy all objects and subdirs of directory source as a subdir of the current directory

   TDirectory *savdir = gDirectory;
   TDirectory *adir = savdir->mkdir(source->GetName());
   adir->cd();

   //loop on all entries of this directory
   TKey *key;
   TIter nextkey(source->GetListOfKeys());
   while ((key = (TKey*)nextkey())) {
      const char *classname = key->GetClassName();
      TClass *cl = gROOT->GetClass(classname);
      if (!cl) continue;
      if (cl->InheritsFrom(TDirectory::Class())) {
         source->cd(key->GetName());
         TDirectory *subdir = gDirectory;
         adir->cd();
         CopyDir(subdir);
         adir->cd();
      } else if (cl->InheritsFrom(TTree::Class())) {
         TTree *T = (TTree*)source->Get(key->GetName());
         adir->cd();
         TTree *newT = T->CloneTree(-1,"fast");
         newT->Write();
      } else {
         source->cd();
         TObject *obj = key->ReadObj();
         adir->cd();
         obj->Write();
         delete obj;
     }
  }
  adir->SaveSelf(kTRUE);
  savdir->cd();
}

// Test
void test_copy()
{
  TFile* source = new TFile("DQM.root");
  TFile* dest = new TFile("file.root","recreate");
  
  dest->cd();
  CopyDir(source);
  
  std::cout << "Before close " << endl;

  source->Close();
  dest->Close();
  
  std::cout << "After close " << endl;
}

the python macro

#!/usr/bin/python

import ROOT

def CopyDir(source):
   adir = ROOT.gDirectory.mkdir(source.GetName())
   adir.cd()

   for key in source.GetListOfKeys():
       classname = key.GetClassName()
       cl = ROOT.gROOT.GetClass(classname)
       if (not cl):
           continue
       if (cl.InheritsFrom(ROOT.TDirectory.Class())):
           subdir = source.Get(key.GetName()) # Get the TDirectory...
           adir.cd()
           CopyDir(subdir)
           adir.cd()
           #del subdir # A tester
       elif (cl.InheritsFrom(ROOT.TTree.Class())):
           T = source.Get(key.GetName())
           adir.cd()
           newT = T.CloneTree(-1,"fast")
           newT.Write()
       else:
           source.cd()
           obj = key.ReadObj()
           adir.cd()
           obj.Write()
           del obj

   adir.SaveSelf(ROOT.kTRUE)
   adir.GetMotherDir().cd()

source = ROOT.TFile("DQM.root")
dest = ROOT.TFile("file.root","recreate")

dest.cd()
CopyDir(source)

print("Before close")

source.Close()
dest.Close()

print("After close")

I don’t know what really happen during the TFile.Close() at the end and why there is such a big difference between the C++ code and the python code.

What do you think ?

Thanks,

Julien

Julien,
haven’t gone through all the details, but my guess would be this:[code] obj = key.ReadObj()
adir.cd()
obj.Write()
del obj[code]which does not do the equivalent of “delete obj;”, since Python does not take ownership on naked pointers that are returned (it can’t know the ownership). At the end of the script, all these objects get destroyed by ROOT anyway, causing a lot of notification callbacks.

What version of ROOT are you using? In ROOT6, you can call obj.destruct() directly to force destruction of the C++ object.

Cheers,
Wim

Hi Wim,

Thanks for the reply. About the code : del obj
I actually replace it by obj.Delete() which works.

And about the ownership issue, I fixed it adding the following code :

ROOT.gROOT.GetListOfFiles().Remove(source) ROOT.gROOT.GetListOfFiles().Remove(dest)
The speed of the copy is not decreasing with time any more and there is no more waiting at the end of the code. The time spent in the execution is divided by 10 !

Cheers,

Julien