TTree::MergeList

Hello everyone,

I’m trying to merge 2 differents (TTree*) objects, with similar structure (branch1, branch2, branch3, branch4).
After merging using TTree::MergeList(treelist), I got a root file and when I open it, I got the following structure:

  • branch1;31
  • branch1;32
  • branch2;1
  • branch3;1
  • branch4;1

Actually I have a huge amount of events for branch1 (12M), do you know if there is a limitation of size to merge TTree ? I cannot easily show an example here, as long as i’m using big files to be able to reproduce de problem…
When I use lower number of events everything is fine.

[code]#include <Riostream.h>
#include <TApplication.h>
#include <TCanvas.h>
#include <TH1F.h>
#include <TSystem.h>
#include <TSystemDirectory.h>
#include <TTree.h>
#include <TList.h>
#include <TFile.h>

using namespace std;

int main(int argc, char **argv)
{
if(argc != 2) {

            cout << "[USAGE] " << argv[0] << " <path (e.g. path to rootfiles)>" << endl;
            return 1;
    }

TString path = argv[1];
    if(! TString(gSystem->BaseName(path)).Length()) {

            path = path.Remove(path.Length()-1);
    }

TSystemDirectory dir(path, path);

    vector<TFile*> flist;
    TList *list0 = new TList, *list1 = new TList, *list2 = new TList, *list3 = new TList;
    TList *files = dir.GetListOfFiles();
    if (files) {                TSystemFile *file;
            TString fname;
            TIter next(files);
            while ((file=(TSystemFile*)next())) {

                    fname = file->GetName();
                    if (!file->IsDirectory() && fname.EndsWith("root")) {

                            TFile *frun = new TFile(path+"/"+fname, "READ");
                            if(frun == NULL || frun->IsZombie()) continue;

                            TTree *t0 = ((TTree*) frun->Get("branch1"));
                            TTree *t1 = ((TTree*) frun->Get("branch2"));
                            TTree *t2 = ((TTree*) frun->Get("branch3"));
                            TTree *t3 = ((TTree*) frun->Get("branch4"));
                                                            if(t0 != NULL) list0->Add(t0);
                            if(t1 != NULL) list1->Add(t1);
                            if(t2 != NULL) list2->Add(t2);
                            if(t3 != NULL) list3->Add(t3);

                            if(t0 == NULL || t1 == NULL || t2 == NULL || t3 == NULL)cout << "?";
                            else cout << ".";

                            cout.flush();
                            flist.push_back(frun);
                    }
            }
    }

cout << endl << "saving.. in " << path << ".root ";
    cout.flush();
    TFile *output = new TFile(path + ".root", "RECREATE");
    if(list0->First() != NULL)
    {
            TTree  *new_t0 = TTree::MergeTrees(list0);
                    new_t0->SetName("branch1");
                    new_t0->Write();
                    cout << ".";
                    cout.flush();
    }
if(list1->First() != NULL)
    {
            TTree  *new_t1 = TTree::MergeTrees(list1);
                    new_t1->SetName("branch2");
                    new_t1->Write();
                    cout << ".";
                    cout.flush();
    }
if(list2->First() != NULL)
    {
            TTree  *new_t2 = TTree::MergeTrees(list2);
                    new_t2->SetName("branch3");
                    new_t2->Write();
                    cout << ".";
                    cout.flush();
    }

if(list3->First() != NULL)
    {
            TTree  *new_t3 = TTree::MergeTrees(list3);
                    new_t3->SetName("branch4");
                    new_t3->Write();
                    cout << ".";
                    cout.flush();
    }
cout << endl;

    output->Close();

    while( ! flist.empty() ){

            flist.back()->Close();
            flist.pop_back();
    }

return 0;

}

[/code]

Btw how should I interprete these branch1;31 and branch2;32 ? Where are the ;1;2;3;4…;31 previous numbers ?

Hi,

Btw how should I interprete these branch1;31 and branch2;32 ?

Those are cycles, i.e. backup of the meta data of the TTree. I assume that this means that you 30-ish files for those 2 trees and only one for the other two.

do you know if there is a limitation of size to merge TTree ? I

There is no limitation on the size of the TTree. In the way you are doing it, there is a limitation on the number of files opened (this is an operating system limitation, usually around a 1000 files).

Cheers,
Philippe.

Thank you very much for explanations

Are you sure the limitation is around 1000 files ?
I am merging only 329 files.

I’m going to look more carefully to understand what is going on…

Hi,

Are you sure the limitation is around 1000 files ?

No. The number itself is completely operating system dependent. In addition I have no idea what the problem really is in your case in this limitation.

As a side note, did you try to just use the ‘hadd’ command line utilities to solve your problem?

Cheers,
Philippe.

I tried, but I guess hadd don’t like that my trees are themselves in a directory.
But I will try to adapt hadd.C and maybe it will work better… :slight_smile:

Thanks for your help !
Marco

Concernant mon problème d’arbre, j’ai laissé tombé mon programme utilisant MergeList().

Autrement, il y avait une erreur dans hadd.C, quand un arbre se trouve dans un dossier cela ne marche pas.

Voici ce que j’ai fait pour corriger le problème:

[code]else if ( obj->IsA()->InheritsFrom( TTree::Class() ) ) {

     // loop over all source files create a chain of Trees "globChain"
    TString obj_name;
    if (path.Length()) obj_name = path + "/" + obj->GetName();
    else obj_name = obj->GetName();
[...]

}
[/code]

J’ai relancé l’exécution de mes 300 fichiers et je n’ai plus le problème ! Merci pour votre aide.
J’espère que mon bout de code pourra en aider d’autres ou que cela pourra être implémenté dans hadd.C ! Avis à un admin !