I am trying to merge 400 files each with around 7MB using hadd in ROOT version 5.34/07.
The first 20 files are done within 10 mins, but then it slows down quickly and takes hours to merge 1 or 2 more files when the file size of the target file reaches around 100~200 MB.
I opend 3 files at a time and used 3GB of memory to run the job. And using 6GB memory does not improve the situation.
Any help or information will be greatly appreciated.
The file structure of each file is like this:
KEY: AttributeListLayout Schema;1 KEY: TDirectoryFile physicsMeta;1 physicsMeta KEY: TDirectoryFile Lumi;1 Lumi KEY: TTree CollectionTree;1 CollectionTree KEY: TTree physics;1 physics
And I am not sure if the following warning is related to the fact that the merging is always stucked at one of the directories, but not the trees:
Warning in <TClass::TClass>: no dictionary for class AttributeListLayout is available Warning in <TClass::TClass>: no dictionary for class pair<string,string> is available
The compression level is set to be matched with:
hadd -f6 -n 3 targetfile sourcefile1 sourcefile2 sourcefile3 ...