I notice some unfortunate memory consumption when iterating over a tree in a root file containing an ntuple: The memory consumption increases with each new entry opened, causing root to crash in the end (on the node I am using it reaches about 1GB before crashing).
- using the MakeClass infrastructure
- problem occurs even when only looping over the entries with GetEntry() and not doing anything else
- root v5.18
- gObjectTable->Print() does not show anything unusual (as far as I can tell…), no increasing consumption indicated
I could produce a small working example but it won’t be that small as it is spread out over several files…
Could you post a data file somewhere? if the leak reaches gigabytes, one file should be sufficient to spot it.
Yes, one file should be enough… but that’s actually also in the GB-range. I am currently copying it over to lxplus, it should be there in about 20min, size 1.8GB
My tests seem to indicate that the single large file is not the problem, splitting it up would lead to the same problem (when opening a small file first and then a larger one afterwards in a TChain, root crashes after the same amount of total events).
Seems like the problem is still unsolved… as the file is in a tmp directory it might disappear soon. If anyone needs it, post here or send me an email (cern mail account, can be inferred from my username) and I will put it there again.
I was away and back to this problem only now when your file has disappeared.
Could you post the right setup reproducing this problem?
Thanks for your reply…
The file is still on lxplus253.cern.ch:/tmp/mflechl/ntuple/matchig.D3PD.aan.root
You can get a minimal version (only loops over entries without doing anything) of my code from
For me, it crashes after about 18000 events, and in fact by then consumes 3.3 GB of RAM.
To run the code:
- edit Analyzer.C and change the path to the file in line 55
- root -l Analyzer.C; click on “Load File”, then click on “Run”
I cannot reproduce your problem. I do not see any leak at all.
I modified your script Analyzer.c (only function Analyzer) to run it easily on my machine (see attachement) and the output is
code  root -l Analyzer.C
File: matchig.D3PD.aan.root loaded
Running over events…
Warning in TClass::TClass: no dictionary for class AttributeListLayout is available
Warning in TClass::TClass: no dictionary for class pair<string,string> is available
The input chain contains: 30687 events.
…|10000|…|20000|…|30000|Done running over events.
root  .q
Analyzer.C (1.79 KB)
Thanks - when using your modified file (in fact I only took over the part where the shared libraries are created from within the macro, instead of “make”), the memory problem disappears for me as well.
While I am happy now and have code that works for me - I still wonder what the problem was. Was there anything clearly wrong/disfavoured in my approach? Or a compiler problem (or does .L also go via the default gcc?)?
As it works now it is not worth spending a lot of time on this… but in case it is something obvious I would be interested to find out.
I suspect that your shared libs were not in phase with your code, because the result should be identical. Anyhow, I recommend the way I rewrote your main function
-you do not need any makefile
-it is portable across all supported architectures (linux(es), mac, Windows)
-if you change one of teh source files, the system ACLIC will automatically recompile
what has changed (like make but more secure)