Crash when trying to merge large trees

Hello,

I have a few trees I want to merge, so I wrote the script:

void merge()
{
	TChain chain("tree");
	for (int i = 0 ; i < 1000 ; i++)
	{
		chain.Add("treefile" + TString::Format( "%i" , i) + ".root");
	}
	chain.Merge("output.root");	
}

Which crashes mid-run, can anyone please guide me through that?

Thank you.

Hi,

what is the error? Are you sure that the files exist? Do you know that you can specify globs? E.g.

TChain chain("tree");
chain.Add("treefile*.root");
chain.Merge("output.root");	

Do you know that you can use the hadd executable too?

Cheers,
D

the error I get is

Exception: bad allocation
Error: Symbol G__exception is not defined in current scope  (tmpfile)(1)
Error: type G__exception not defined FILE:(tmpfile) LINE:1
(Long64_t)0
*** Interpreter error recovered ***

Also, Im not sure what’s hadd executable.

Hi gdule, according to my limited ROOT experience, “bad allocation” error usually means you’re running out of memories allowed per process in you system. My suggestion would be do not combine those trees and analyze them separately.

The hadd executable is a bash command comes with ROOT installation. You may use it like this: $ hadd output_file_name input_filename_1 input_filename_2 …

Hope this helps.

That’s rather weird, since I’ve already merged larger trees than now.

The problem is I need all of the data, which is stored in multiple .root files.

I’v tried $ hadd command, and it simply created some .root file with none of the data (though it weight about 2/3 of each of my .root files).

Help!

Hi,

this is not an expected behaviour for hadd which is used in production since many years. I suspect you are using some ROOT 5 version: is this correct?

Cheers,
D

Yes, I am using root 5 (since it is on Windows)

Hi,

just to come back to my previous question: are all the files you are listing physically on disk and not corrupted? Can you identify acting on the max number of iteration the file at which the issue appears and inspect the file which is linked to the problem?

D

All the files are on disk and not corrupted (each containing a fully functional TTree).
Im not sure I understand what you mean by the second part.

Start with 2 files, 4, 8, 16 and identify if the crash occurs with a particular file.

It does not occur on any particular file.
I have 5 files, and each one weighs about 300mb, and it fails to merge any two of them.

Could you share two of these files? Your original post has a loop on 1000 files: is this the same problem?

I have since merged these 1000 files to 5, so It will be easier for me to work with, and also maybe as a fix to the problem, but it wasn’t.

How could I send you these file? (keep in mind that they are quite heavy)

I am not sure what is the infrastructure you have at disposal: CERNBox?

Yes, I’ll upload these files and send them to you once they are uploaded.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.