Problems writing large TFolder hierachy

ROOT
1

Dear ROOT experts,

I encounter problems when trying to write a TFolder-based hierachy which contains several histograms and other information. To be precise, I use some custom classes derived from TFolder and have of the order of ~70 folders (each one about ~10 levels deep) which in total is about 1.4GB of memory.

In order to simplify the situation (and to rule out possible problems in the custom code), I attach a short ROOT script reproducing the problem. This script creates some dummy folder hierachy of 500 TFolders, each one containing 100 TH3D objects which should mimic the number of objects and memory I have to deal with in my use case. In the end I try to write the ROOT folder into a ROOT file and this is the moment were my ROOT session get stuck (nothing happens for hours while the process is running at 100% CPU). There are no error or warning messages.

At the moment I am a bit lost. I am quite sure that there might be options to twaek the writing of large objects a bit. Any hint or suggestion is greatly appreciated.

Thanks
Christian
TFolder_test.C (748 Bytes)

2

Ok, apparently the problem is that the nested TFolder hierachy cannot be written to the file as its size exceeds 1GB which is the maximum size which is allowed to be assoicated with a single TKey which I took from here:

[url]Program hangs when trying to save very long TClonesArray
[url]<TBufferFile::CheckCount>: Warning

However, I do not even see an error or warning message. The program is just stuck in some kind of loop. Unfortunately, this limitation is neither documented in the user guide nor on the reference documentation online (both TKey and TFolder do not mention anything about size limitations). Maybe this information could be added somewhere?

Luckily, there is a possibility to write a large, nested TFolder structure using TTree’s. However, this also barely documented (though it is very briefly mentioned in the user guide for TTree’s but without any helpful example). The most helpful information I found on this topic is the following reply from Rene:

http://root.cern.ch/root/roottalk/roottalk01/4626.html

For reference and other users who might stumble over the same problem in the future, I attach a modified version of the script which allows to write the folder structure.

Nonetheless, my specific problem persists as I do not use TFolder objects but a custom class which is derived from TFolder. In this case, when creating a TTree from my folder, the splitting does not work as the TTree only creates subbranches for objects which are explicitly of type TFolder::Class

http://root.cern.ch/gitweb?p=root.git;a=blob;f=tree/tree/src/TTree.cxx;h=d82fe6fad5059996eb655d778c21ad2fadeb790f;hb=HEAD#l1552

if (ob->IsA() != TFolder::Class())
  return 0;

In order to forward this nice splitting feature to classes derived from TFolder, one would need to change this to

if (!ob->InheritsFrom(TFolder::Class()))
  return 0;

an similarly in line TTree.cxx:1563 http://root.cern.ch/gitweb?p=root.git;a=blob;f=tree/tree/src/TTree.cxx;h=d82fe6fad5059996eb655d778c21ad2fadeb790f;hb=HEAD#l1563.

Would this change have any side effects one should be aware of? Otherwise, I would like to request this change. Should I open a JIRA ticket for this feature request?

Thanks
Christian
TFolder_test.C (909 Bytes)