I notice some unfortunate memory consumption when iterating over a tree in a root file containing an ntuple: The memory consumption increases with each new entry opened, causing root to crash in the end (on the node I am using it reaches about 1GB before crashing).
Details:
using the MakeClass infrastructure
problem occurs even when only looping over the entries with GetEntry() and not doing anything else
root v5.18
gObjectTable->Print() does not show anything unusual (as far as I can tell…), no increasing consumption indicated
I could produce a small working example but it won’t be that small as it is spread out over several files…
Yes, one file should be enough… but that’s actually also in the GB-range. I am currently copying it over to lxplus, it should be there in about 20min, size 1.8GB lxplus253.cern.ch:/tmp/mflechl/ntuple/matchig.D3PD.aan.root
My tests seem to indicate that the single large file is not the problem, splitting it up would lead to the same problem (when opening a small file first and then a larger one afterwards in a TChain, root crashes after the same amount of total events).
Seems like the problem is still unsolved… as the file is in a tmp directory it might disappear soon. If anyone needs it, post here or send me an email (cern mail account, can be inferred from my username) and I will put it there again.
Martin
Thanks for your reply…
The file is still on lxplus253.cern.ch:/tmp/mflechl/ntuple/matchig.D3PD.aan.root
You can get a minimal version (only loops over entries without doing anything) of my code from www3.tsl.uu.se/~flechl/mincode.tgz
For me, it crashes after about 18000 events, and in fact by then consumes 3.3 GB of RAM.
To run the code:
edit Analyzer.C and change the path to the file in line 55
make
root -l Analyzer.C; click on “Load File”, then click on “Run”
I cannot reproduce your problem. I do not see any leak at all.
I modified your script Analyzer.c (only function Analyzer) to run it easily on my machine (see attachement) and the output is
File: matchig.D3PD.aan.root loaded
Running over events…
Warning in TClass::TClass: no dictionary for class AttributeListLayout is available
Warning in TClass::TClass: no dictionary for class pair<string,string> is available
The input chain contains: 30687 events.
…|10000|…|20000|…|30000|Done running over events.
root [1] .q
[/code]
Thanks - when using your modified file (in fact I only took over the part where the shared libraries are created from within the macro, instead of “make”), the memory problem disappears for me as well.
While I am happy now and have code that works for me - I still wonder what the problem was. Was there anything clearly wrong/disfavoured in my approach? Or a compiler problem (or does .L also go via the default gcc?)?
As it works now it is not worth spending a lot of time on this… but in case it is something obvious I would be interested to find out.
I suspect that your shared libs were not in phase with your code, because the result should be identical. Anyhow, I recommend the way I rewrote your main function
-you do not need any makefile
-it is portable across all supported architectures (linux(es), mac, Windows)
-if you change one of teh source files, the system ACLIC will automatically recompile
what has changed (like make but more secure)