Root 6.04.14 hadd 100Gb and rootlogon

Dear experts,

I am currently trying to use hadd to make an output file with a tree with size > 100 Gb.

I read:
https://root-forum.cern.ch/t/root-6-04-14-hadd-100gb-tfilemerger-recursiveremove/24117/4
and:

It seems that in my case the easiest solution is to create in the directory I execute “hadd” a rootlogon.C file with:
{
TTree::SetMaxTreeSize( 1000000000000LL ); // 1 TB
}

However, it seems that when I run “hadd” this new option is not taken into account. I tried two commands:

  • hadd output.root input_*

  • root -l
    root [0] TTree::GetMaxTreeSize()
    (Long64_t) 1000000000000
    root [1] gSystem->Exec(“hadd output.root input_*”);

I guess this is because hadd is not recompiled. Please could you tell me what the easiest way to take the new MaxTreeSize number into account in hadd?

I would prefer not to write a separate macro replicating the hadd behavior if possible.

Thanks for your help
Best wishes
Matthias

Indeed hadd does not load the rootlogon file. However you can still manager using the following pattern.

Use a source file (let’s called it startup.C) like

#include "TTree.h"

int startup() {
  TTree::SetMaxTreeSize( 1000000000000LL ); // 1 TB
  return 0;
}

namespace {
  static int i = startup();
}

compile it into a shared library, for example with:

root.exe -b -l -q startup.C+

and then use LD_PRELOAD (DYLD_INSERT_LIBRARIES on MacOS) to preload that library:

 DYLD_INSERT_LIBRARIES=startup_C.so hadd  output.root input_*
 LD_PRELOAD=startup_C.so hadd  output.root input_*

Cheers,
Philippe.

Dear Philippe,

thank you very much for your message, this is very helpful!

In addition, let me copy/paste here a little python script a colleague passed me which does the job also in case people visit this thread (*).

Thanks again for your help
Best wishes
Matthias

(*)

import ROOT
import os, sys

print 'Merging %s' % sys.argv[1]

print "Max tree size",ROOT.TTree.GetMaxTreeSize()
ROOT.TTree.SetMaxTreeSize(200000000000) # 200 Gb
print "Updated tree size",ROOT.TTree.GetMaxTreeSize()

rm = ROOT.TFileMerger(False)
rm.SetFastMethod(True)


path = 'mypath'
file_output = '%s.root' % sys.argv[1]
file_list = []
for path, dirs, files in os.walk(path):
  for filename in files:
    if ('%s_part' % sys.argv[1]) in filename: file_list.append(path+filename)

print "Input file list:",file_list
print "Output file:",file_output

for F in file_list:

    print "Adding ->",F
    rm.AddFile(F)

rm.OutputFile(file_output)
rm.Merge()

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.