# Hadd error

**URL:** <https://root-forum.cern.ch/t/hadd-error/8806>\
**Category:** ROOT\
**Created:** [August 17, 2009, 12:55am UTC](https://root-forum.cern.ch/t/hadd-error/8806 "2009-08-17T00:55:29Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![zhiyiliu](https://root-forum.cern.ch/letter_avatar_proxy/v4/letter/z/85e7bf/32.png) [@zhiyiliu](https://root-forum.cern.ch/u/zhiyiliu)\
**Post date:** [August 17, 2009, 12:55am UTC](https://root-forum.cern.ch/t/hadd-error/8806/1 "2009-08-17T00:55:29Z")

</div>

Hi,  
I am trying to combine lots of root files together by ‘hadd’ in root. However, I got this error:

> [@](#):
>
> terminate called after throwing an instance of ‘std::bad\_alloc’  
> what(): St9bad\_alloc

(1) each file is small, but the total size is like 1 GB  
(2) the computer memory is 2 GB  
(3) I checked each file, they are fine (created without error)

It seems that it is due to memory consumption. I really need to combine all those files to one file. Any alternative approach, please?

Thank you,  
Zhiyi.  
Linux: 2.4  
ROOT: v4.04

---

<div class="post-metadata">

**Author:** ![brun](https://root-forum.cern.ch/letter_avatar_proxy/v4/letter/b/b9e5f3/32.png) [@brun](https://root-forum.cern.ch/u/brun)\
**Post date:** [August 17, 2009, 5:32am UTC](https://root-forum.cern.ch/t/hadd-error/8806/2 "2009-08-17T05:32:19Z")

</div>

It is likely that your Tree is a memory resident Tree, you should fix this problem first, please.

Moving to 5.24 you could connect each Tree file and do

` TFile f(file1.root,"update");
   TTree *T = (TTree*)f.Get("name of your tree");
   T->FlushBaskets();
   T->Write();
`  
and repeat this opetartion for each file, then you can use hadd.

Rene

---

<div class="post-metadata">

**Author:** ![zhiyiliu](https://root-forum.cern.ch/letter_avatar_proxy/v4/letter/z/85e7bf/32.png) [@zhiyiliu](https://root-forum.cern.ch/u/zhiyiliu)\
**Post date:** [August 17, 2009, 6:20am UTC](https://root-forum.cern.ch/t/hadd-error/8806/3 "2009-08-17T06:20:56Z")

</div>

Thanks for your message.

You mean this operation is only available for “5.24”? How about other versions like v5.22?

[quote=“brun”]It is likely that your Tree is a memory resident Tree, you should fix this problem first, please.

Moving to 5.24 you could connect each Tree file and do

` TFile f(file1.root,"update");
   TTree *T = (TTree*)f.Get("name of your tree");
   T->FlushBaskets();
   T->Write();
`  
and repeat this opetartion for each file, then you can use hadd.

Rene[/quote]

---

<div class="post-metadata">

**Author:** ![pcanal](https://root-forum.cern.ch/user_avatar/root-forum.cern.ch/pcanal/32/12439_2.png) [@pcanal](https://root-forum.cern.ch/u/pcanal)\
**Post date:** [August 18, 2009, 2:59pm UTC](https://root-forum.cern.ch/t/hadd-error/8806/4 "2009-08-18T14:59:53Z")

</div>

Hi,

FlushBaskets was introduced in v5.20 and since then is called by default in TTree::Write.

Cheers,  
Philippe

---

<div class="post-metadata">

**Author:** ![vsavinov](https://root-forum.cern.ch/letter_avatar_proxy/v4/letter/v/439d5e/32.png) [@vsavinov](https://root-forum.cern.ch/u/vsavinov)\
**Post date:** [December 7, 2010, 6:06am UTC](https://root-forum.cern.ch/t/hadd-error/8806/5 "2010-12-07T06:06:19Z")

</div>

Dear ROOT Wizards,

I am having the same problem when trying to merge even just two ROOT files (but the output file is large):

hadd md3pdm2x-v5.1-sa4t\_data\_lq\_periodI.root md3pdm2x-v5.1-sa4t\_data\_lq\_periodI1.root md3pdm2x-v5.1-sa4t\_data\_lq\_periodI2.root

Target file: md3pdm2x-v5.1-sa4t\_data\_lq\_periodI.root

Source file 1: md3pdm2x-v5.1-sa4t\_data\_lq\_periodI1.root  
Source file 2: md3pdm2x-v5.1-sa4t\_data\_lq\_periodI2.root

Target path: md3pdm2x-v5.1-sa4t\_data\_lq\_periodI.root:/  
Found subdirectory MakerD3PDm2x  
Target path: md3pdm2x-v5.1-sa4t\_data\_lq\_periodI.root:/MakerD3PDm2x

md3pdm2x-v5.1-sa4t\_data\_lq\_periodI1.root tree:MakerD3PDm2x/anaTree entries=1234567890  
md3pdm2x-v5.1-sa4t\_data\_lq\_periodI2.root tree:MakerD3PDm2x/anaTree entries=1234567890

terminate called after throwing an instance of 'std::bad\_alloc’  
what(): std::bad\_alloc  
Abort (core dumped)

The sizes of input files:

3403712 md3pdm2x-v5.1-sa4t\_data\_lq\_periodI1.root  
2289104 md3pdm2x-v5.1-sa4t\_data\_lq\_periodI2.root

SLC5 / 64bit 24GB of real memory

root\_v5.27.06.Linux-slc5-gcc4.3

Applying this fix:

TFile f(file,“update”);  
TTree _T = (TTree_)f.Get(“MakerD3PDm2x/anaTree”);  
T-\>FlushBaskets();  
T-\>Write();  
f.Close();

to each of the two input files didn’t help,

could you please help?

thanks!  
Vladimir

---

<div class="post-metadata">

**Author:** ![vsavinov](https://root-forum.cern.ch/letter_avatar_proxy/v4/letter/v/439d5e/32.png) [@vsavinov](https://root-forum.cern.ch/u/vsavinov)\
**Post date:** [December 7, 2010, 7:13am UTC](https://root-forum.cern.ch/t/hadd-error/8806/6 "2010-12-07T07:13:28Z")

</div>

p.s. using root\_v5.27.06.Linux-slc5\_amd64-gcc4.3 solved my immediate problem

---

<div class="post-metadata">

**Author:** ![pcanal](https://root-forum.cern.ch/user_avatar/root-forum.cern.ch/pcanal/32/12439_2.png) [@pcanal](https://root-forum.cern.ch/u/pcanal)\
**Post date:** [December 7, 2010, 7:04pm UTC](https://root-forum.cern.ch/t/hadd-error/8806/7 "2010-12-07T19:04:51Z")

</div>

Hi,

If you tell me where to find those 2 files, I will attempt to reproduce the problem.

Cheers,  
Philippe.

---

<div class="post-metadata">

**Author:** ![vsavinov](https://root-forum.cern.ch/letter_avatar_proxy/v4/letter/v/439d5e/32.png) [@vsavinov](https://root-forum.cern.ch/u/vsavinov)\
**Post date:** [December 7, 2010, 7:54pm UTC](https://root-forum.cern.ch/t/hadd-error/8806/8 "2010-12-07T19:54:11Z")

</div>

Philippe,

these files are on a local computer in my office -  
I would need to copy them to elsewhere,  
and I will do so, but it’s not clear to me there is  
enough room on afs at CERN for that.

There is one more piece of info:

when I was (successfully) merging my huge TTrees using hadd  
(64bit ROOT) I noticed that the trees were definitely being  
merged in memory: my machine has 24 real GB, and, when the  
total size of files to be merged approached approx. 20-22 GB,  
the machine started swappping like crazy and, after an hour of  
not even being able to log in, I was able to recover by Ctrl/C  
ROOT session that was doing the merging. My interpretation is  
that ROOT was trying to merge my TTrees in RAM, though I tried  
the prescription suggested by Rene earlier in this thread -

as soon as the files are at CERN, I will let you know -

thanks,  
v

[quote=“pcanal”]Hi,

If you tell me where to find those 2 files, I will attempt to reproduce the problem.

Cheers,  
Philippe.[/quote]

---

<div class="post-metadata">

**Author:** ![pcanal](https://root-forum.cern.ch/user_avatar/root-forum.cern.ch/pcanal/32/12439_2.png) [@pcanal](https://root-forum.cern.ch/u/pcanal)\
**Post date:** [December 7, 2010, 7:57pm UTC](https://root-forum.cern.ch/t/hadd-error/8806/9 "2010-12-07T19:57:31Z")

</div>

Hi,

[quote]My interpretation is that ROOT was trying to merge my TTrees in RAM,  
[/quote]This should not really happen …

Can you send me the result of tree-\>Print(); and myfile-\>Map(); for one of the two files …

Thanks,  
Philippe.

---

<div class="post-metadata">

**Author:** ![vsavinov](https://root-forum.cern.ch/letter_avatar_proxy/v4/letter/v/439d5e/32.png) [@vsavinov](https://root-forum.cern.ch/u/vsavinov)\
**Post date:** [December 7, 2010, 8:35pm UTC](https://root-forum.cern.ch/t/hadd-error/8806/10 "2010-12-07T20:35:52Z")

</div>

[quote=“pcanal”]Hi,

[quote]My interpretation is that ROOT was trying to merge my TTrees in RAM,  
[/quote]This should not really happen …

Can you send me the result of tree-\>Print(); and myfile-\>Map(); for one of the two files …

Thanks,  
Philippe.[/quote]

Philippe, the files are huge, but I can let you log in to this computer, if you could help that way - please let me know.

now, the result of Print and Map:

everything is in [proxy.phyast.pitt.edu/~savinov/tmp/ROOT/](http://proxy.phyast.pitt.edu/~savinov/tmp/ROOT/) ,

more specifically:

1. tree-\>Print():

[proxy.phyast.pitt.edu/~savinov/t … \_print.log](http://proxy.phyast.pitt.edu/~savinov/tmp/ROOT/ttree_print.log)

1. file-\>Map():

this is an ENORMOUS file, I had to ctrl/c!:

[proxy.phyast.pitt.edu/~savinov/t … le\_map.log](http://proxy.phyast.pitt.edu/~savinov/tmp/ROOT/file_map.log)

btw, now, this is how I “fixed” them before trying to hadd:

void fix\_all\_files() {

fix\_one\_file("./md3pdm2x-v5.1-sa4t\_data\_lq\_periodI1.root");  
fix\_one\_file("./md3pdm2x-v5.1-sa4t\_data\_lq\_periodI2.root");

}

void fix\_one\_file ( Char\_t\* file, Char\_t\* source = “MakerD3PDm2x/anaTree”) {

cout \<\< "Will try to fix TTree " \<\< source \<\< " from file " \<\< file \<\< endl;

TFile f(file,“update”);  
TTree _T = (TTree_)f.Get(source);  
T-\>FlushBaskets();  
T-\>Write();  
f.Close();

return;

}

thanks,  
v

---

<div class="post-metadata">

**Author:** ![pcanal](https://root-forum.cern.ch/user_avatar/root-forum.cern.ch/pcanal/32/12439_2.png) [@pcanal](https://root-forum.cern.ch/u/pcanal)\
**Post date:** [December 7, 2010, 8:49pm UTC](https://root-forum.cern.ch/t/hadd-error/8806/11 "2010-12-07T20:49:46Z")

</div>

Hi,

[quote]btw, now, this is how I “fixed” them before trying to hadd:[/quote]Is the problem happening before or after you did the fix? Are the log before or after you did the fix?

Philippe.

---

<div class="post-metadata">

**Author:** ![vsavinov](https://root-forum.cern.ch/letter_avatar_proxy/v4/letter/v/439d5e/32.png) [@vsavinov](https://root-forum.cern.ch/u/vsavinov)\
**Post date:** [December 7, 2010, 8:56pm UTC](https://root-forum.cern.ch/t/hadd-error/8806/12 "2010-12-07T20:56:09Z")

</div>

[quote=“pcanal”]Hi,

[quote]btw, now, this is how I “fixed” them before trying to hadd:[/quote]Is the problem happening before or after you did the fix? Are the log before or after you did the fix?

Philippe.[/quote]

the problem happens regardless of me “fixing” or not “fixing” the files.  
(I am “fixing” copies, not the originals) -

the results  
of

map and print are shown after “fixing” the files -

btw, I expected this “fixing” to take a lot of time, but “fixing” each file takes just a few seconds…

thanks!  
v

---

<div class="post-metadata">

**Author:** ![pcanal](https://root-forum.cern.ch/user_avatar/root-forum.cern.ch/pcanal/32/12439_2.png) [@pcanal](https://root-forum.cern.ch/u/pcanal)\
**Post date:** [December 7, 2010, 8:58pm UTC](https://root-forum.cern.ch/t/hadd-error/8806/13 "2010-12-07T20:58:39Z")

</div>

[quote]Philippe, the files are huge, but I can let you log in to this computer, if you could help that way - please let me know[/quote]So this seems like the only way to make progress. My email is [pcanal@fnal.gov](mailto:pcanal@fnal.gov).

Cheers,  
Philippe.

---

<div class="post-metadata">

**Author:** ![pcanal](https://root-forum.cern.ch/user_avatar/root-forum.cern.ch/pcanal/32/12439_2.png) [@pcanal](https://root-forum.cern.ch/u/pcanal)\
**Post date:** [December 8, 2010, 4:28pm UTC](https://root-forum.cern.ch/t/hadd-error/8806/14 "2010-12-08T16:28:11Z")

</div>

Hi Vladimir,

Thank you very much for reporting this problem (a significant memory in the TTreeClone in the development release v5.27/06) which is now fixed by revision 37405 of the trunk.

Cheers,  
Philippe.

---

<div class="post-metadata">

**Author:** ![vsavinov](https://root-forum.cern.ch/letter_avatar_proxy/v4/letter/v/439d5e/32.png) [@vsavinov](https://root-forum.cern.ch/u/vsavinov)\
**Post date:** [December 8, 2010, 11:36pm UTC](https://root-forum.cern.ch/t/hadd-error/8806/15 "2010-12-08T23:36:13Z")

</div>

Philippe,  
thank you very much for fixing this bug!  
Just for fun, earlier today I used the new hadd  
to merge two sets of files and successfully did  
so for output file size exceeding 30GB!  
thanks!  
v
