Executable to combine files with TFileMerger

ROOT
1

Hi,

I would like to know if someone has written an executable to combines ROOT files (like hadd) which use TFileMerger. I need it to combine large files in CERN CASTOR and put the output back in CASTOR, without using local CERN AFS disk. The instructions given for TFileMerger uses CINT/macro interpreter but I would prefer an executable like hadd.

If nobody has done it then I will write one and ask the root developers to include it in the next release cycle.

Haryo

2

Hi,

hadd should already be able to do it, just pass the fylly qualified name of the file as argument to hadd (rfio://server/path/file.root).

Cheers,
PHilippe.

3

Hi Philippe,

No, hadd can’t do it. I should have been clearer in my original message.

I typed this into the lxplus command line

hadd rfio:/castor/cern.ch/user/h/haryo/test.root [list of files here]

and the only thing it does is producing an empty root file

lxplus prompt>rfdir /castor/cern.ch/user/h/haryo/

-rw-r–r-- 1 haryo zh 470 Dec 07 20:55 test.root

Anyway, I have written the executable and am only willing to share it to the community.

Haryo

4

[quote]and the only thing it does is producing an empty root file[/quote]Strange.

[quote]Anyway, I have written the executable and am only willing to share it to the community.[/quote]Do not hesitate to upload it here.

Cheers,
Philippe.

5

Here is the executable. I don’t put the compilation instructions as it varies from environment to environment. Parts of the code are taken from hadd. I haven’t implemented the “-T” option a la hadd. I can also send the source file directly to one of the developers.

Let me know if this code will be part of ROOT release or not.

Haryo

[code]/**
@file TFileMerger_x.cc

@brief Executable to merge file with TFileMerger class.
Can combine file in CASTOR or dCache, with output in CASTOR or dCache.

Usage:
\verbatim
TFileMerger_x [-f] <outputfile> <inputfile1> <inputfile2> ...
\endverbatim

@param outputfile Name of outputfile. Use the appropriate <tt>rfio</tt>
or <tt>dcap</tt> for output in CASTOR or dCache.

@param inputfile1
Input files. Use the appropriate <tt>rfio</tt> or <tt>dcap</tt> for inputs
in CASTOR or dCache.  One can also use <tt>\@</tt> to indicate indirect
input like <tt>hadd</tt>.

@date Sun Dec  5 20:19:52 UTC 2010

@author Haryo Sumowidagdo <Suharyo.Sumowidagdo@cern.ch>

@version $Id: TFileMerger_x.cc,v 1.3 2010/12/05 23:08:19 haryo Exp $

*/

#include
#include
#include
#include
#include
#include <TFileMerger.h>
#include <TFile.h>

/**
@brief Helper function to add input files.

@param sourcelist Array of string which holds the list of input files.

@param entry An element in the command line arguments.

*/
int AddFile(std::vectorstd::string& sourcelist,
std::string entry);

int main(int argc, char** argv)
{
bool force = strcmp(argv[1],"-f") == 0 ? true : false ;

size_t target = 1 ;
if (force) {
    ++target;
}

if (force) {
    std::cout << "Option -f is used, will force recreating the target file."
              << std::endl;
}

std::string              targetFile(argv[target]);
std::vector<std::string> sourceFile;

for (size_t i = target+1 ; i < (size_t) argc ; ++i) {
    if (AddFile(sourceFile,argv[i]) != 0) {
        return 1;
    }
}

std::cout << "Target file:" << targetFile << std::endl;
for (size_t f = 0 ; f != sourceFile.size() ; ++f) {
    std::cout << "sourceFile[" << f << "]: [" <<sourceFile[f] << "]"
              << std::endl;
}

TFileMerger merger((Bool_t) false);
merger.SetFastMethod((Bool_t) false);

merger.OutputFile(targetFile.c_str());

std::cout << "\n" << "Merging file" << std::endl;
for (size_t f = 0 ; f != sourceFile.size() ; ++f) {
    std::cout << "Adding source file:[" << sourceFile[f] << "]"
              << std::endl;
    merger.AddFile(sourceFile[f].c_str());
}

merger.Merge();

std::cout << "Finish merging files." << std::endl;
return 0;

}

int AddFile(std::vectorstd::string& sourcelist,
std::string entry)
{
if( entry.empty() ) {
return 0;
}

size_t j =entry.find_first_not_of(’ ');

if( j==std::string::npos ) {
return 0;
}

entry = entry.substr(j);

if( entry.substr(0,1)=="@") {

  std::ifstream indirect_file(entry.substr(1).c_str() );

  if( ! indirect_file.is_open() ) {
     std::cerr<< "Could not open indirect file " << entry.substr(1) << std::endl;
     return 1;
  }

  while( indirect_file ){
      std::string line;
      std::getline(indirect_file, line);
      if( AddFile(sourcelist, line)!=0 ) {
          return 1;
      }
  }
  return 0;

}

sourcelist.push_back(entry);
return 0;
}[/code]

6

Hi Haryo,

Thanks for input. Please note that in v5.30/00 hadd has been upgraded (along the line of your example plus additional update for backward compatibility) to use the TFileMerger.

Cheers,
Philippe.