Problems with hadd in lxplus

If i merge only the first 10 files it seems to work:

$ hadd -O merged.root data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0001.3.root data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0002.3.root data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0003.3.root data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0004.3.root data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0005.3.root data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0006.3.root data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0007.3.root data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0008.3.root data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0009.3.root data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0010.3.root
hadd Target file: merged.root
hadd compression setting for all output: 101
hadd Source file 1: data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0001.3.root
hadd Source file 2: data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0002.3.root
hadd Source file 3: data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0003.3.root
hadd Source file 4: data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0004.3.root
hadd Source file 5: data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0005.3.root
hadd Source file 6: data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0006.3.root
hadd Source file 7: data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0007.3.root
hadd Source file 8: data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0008.3.root
hadd Source file 9: data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0009.3.root
hadd Source file 10: data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0010.3.root
hadd Target path: merged.root:/

Sorry just to say that the -f0 is also required to prevent truncations of entries in the files.

I don’t think I see an obvious reason for this to not work for the following 3 files?

It works also with -f0 with only 10 files:

$ hadd -f0 -O merged.root data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0001.3.root data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0002.3.root data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0003.3.root data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0004.3.root data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0005.3.root data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0006.3.root data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0007.3.root data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0008.3.root data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0009.3.root data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0010.3.root
hadd Target file: merged.root
hadd compression setting for all output: 0
hadd Source file 1: data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0001.3.root
hadd Source file 2: data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0002.3.root
hadd Source file 3: data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0003.3.root
hadd Source file 4: data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0004.3.root
hadd Source file 5: data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0005.3.root
hadd Source file 6: data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0006.3.root
hadd Source file 7: data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0007.3.root
hadd Source file 8: data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0008.3.root
hadd Source file 9: data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0009.3.root
hadd Source file 10: data24_13p6TeV.00473317.debugrec_hlt.reproc.HIST_DEBUGSTREAMMON.g69._0010.3.root
hadd Target path: merged.root:/

Yes it also seems to work with the addition of the 13th file but not the 11th or 12th and I don’t see an obvious reason for this?

May be they are corrupted somehow ?

Hi,
check this out: Check the structure of root files before hadd

What happens if you put your 11th (or 12th) file in the beginning of the list of files to be merged?

Hi @yus ,

Thanks for the info, the failure occurs for different ordering as can be seen in one of the previous replies. It is also not consistent when failing in general or across different operating systems.

Hi @yus and @couet ,

The files should not be corrupted and can be merged but it is not consistent when they fail.

The merging works when the 11th or 12th or 13th files are placed at the start, but the only major difference I can see would be one of the string lengths (or one unfilled branch but there are many others that are fine during merging).

I already have to account for this by ordering for a different branch so I cannot simultaneously order all files based on many conditions (these are just one set of examples of many that need to be merged without input every time).

Is the string length in a branch preallocated based on the first file when running hadd (even if no compression is used)?

I am not sure may be @pcanal knows ?