I could neither find a similar topic on this board nor on Savannah.
I work with
OS: Ubuntu 10.10 64bit
Root 5.32 and 5.32.01
I have more than 1000 inputfiles which contain a tree in a subfolder, each with 10k entries and only one cycle number (= 1) (output from Geant4 simulations on a cluster)
To merge these files I do in the Shell
so I specify all input files.
What happens: (with for example 1331 input files)
923 files are processed in one step and then the rest (408).
hadd splits the jobs itself automatically in two steps
An output file with a tree with two cycle numbers is created:
- cycle number 1 contains 9.23M entries (= 923 *10e3 )
so far so good
- cycle number 2 contains only 4.08M entries
not 13.31M entries.
So the second cycle does not contain the events from the first cycle.
When you use the ‘-n XY’ option this gets even worse, then each cycle contains only its part of the data.
The only way I see to handle this at the moment is to keep the number of input file below the number, where hadd does the automatic splitting and do the recursive ‘merging’ by hand.
In 5.30.06 the merging of that many files is not possible, and has to be done recursively by hand (or a script)
This is quite uncomfortably if one likes to process the file afterwards automatically without taking care of cycle numbers.
Can anyone comment on that? Does it also occur on other systems?