I have a basic question about ROOT TTree IO. I store i my TTree a TClonesArray of very simple TObject as usual and the program works. BUT, looking at the memory consumption during the process, i have to confess that something is not clear to me .
I made 2 case tests:
1) Case using standard settings of the TTree , it means :
1) Using AutoFlush set to 30 MB
2) Using the Learning Phase algorithm that adapt after 10 events the IO according to
some statistical report
2) forcing IO and the caching and removing the learning mechanism
outTree->SetAutoFlush(-10000000);
outTree->SetCacheSize(-1);
outTree->StopCacheLearningPhase();
Surprising to me is the huge consumption of Virtual Memory in case 1 ( see in attachment lphase.ps) compare to the more reasonable one of the case 2 ( see in attachment no_phase.ps ).
Can someone comment on that?
Is there some explanation to this odd behaviour of the TTree IO ?
Hi Philippe
I am actually measuring during the writing to the root file.
To send an simple example is not direct since i use my own program but i can send the
file with the output tree, will that be OK for you?
For convenience, i modified the ROOT examples from the tutorial.
There also this effect is visible, but i had to enlarge the memory needed of course.
So i send you the modified files and you could immediately test.
In your tree4 example, there is 4 huge branches. 3 where each entry is 50Mb, 1 where each entry is 150Mb.
And I do see a fluctuation of 600Mb. The 1st 300 Mb is solely for copy in the data in streamed fashion and the 2nd 300Mb is to zipped. The 1st 300Mb should (and will in v5.28) stay constantly allocated but is released after every cluster fill which happens after each tree fill (because the basket and the cluster are over-full after entry) … hence explaining the changed. In v5.28 the change will be not as wild … but the average will be the same of higher.
Hi Philippe,
Thanks for your answer,
In my first post, i was comparing two different setup of the same TREE leading to two completely
different behaviour from the IO allocated memory. One is using the standard Root TTree setup ( with
learning phase included ) and the other used a customized one ( without learning phase ).
The customized setup of TTree seems according to the attached histogram more stable in terms of IO memory allocation.
So since i need a practical solution for my application, could you advice me what optimal setup i can use to
reduce the amplitude of TTree IO memory oscillation?
[quote]and the other used a customized one ( without learning phase ).[/quote]Humm, this is odd since the ‘learning phase’ of the TTreeCache is not really used in writing.
Changing the value of the AutoFlush change the size (and frequency) of the cluster and if a smaller value works better for you that is fine (as long as the value is not too small, where too small is likely to be anything under 10Mb … too small of a value would make the reading much less efficient).