Hi,
At a quick look it seems to me that SCycleOutput and SCycleBaseHist do what they have to do.
This said, merging of objects other then histograms - e.g. SCycleOutput - has always been done one-by-one (I mean, in all ROOT versions).
The object merged is deleted by PROOF right-after.
What happens to the object cloned in SCycleBaseHist::Book ?
I should perhaps setup a simple SFrame job just doing histogram booking and merging …
G
Hi Gerardo,
would be helpful if i prepare for you a simple setup in my public area to run a test job?
best,
Max
Certainly it would.
Let me know if you do it.
G
Hi Gerardo,
please find in attachment a tar file with a release of SFrame where i added some more histograms
to the example code. Setup ROOT, then from Sframe/ do “make”. To run do “sframe_main user/config/FirstCycle_config.xml”. By default it runs PROOF-lite, to run on a PROOF cluster change
in the xml file where you find “lite” with “username@proof_master:port”.
The code runs on ~12000 events from a single file and does 1000 TH1F. This is done inside a simple loop
so you can increase this number easily for testing.
best,
Max
p.s the input file is on cern afs.
SFrame.tgz (310 KB)
Hi,
i just noticed that i forgot to tell you that before compiling SFrame you need to source setup.sh from SFrame/ dir. Please let me know in case you have problems trying this example code.
best,
Max
Hi,
I’m not too responsive this week, should become next week when I get back to CERN.
The idea with the Book(…) function is that the user calls it with an object that is created on the stack. So, with something like this:
TH1* myHist = Book( TH1D( “myHist”, “My super histogram”, 100, 0.0, 100.0 ) );
This way it’s obvious to the user that they don’t have to care about memory management here. The Clone() function is used to put the histograms into the right directory.
Cheers,
Attila
Hi Attila,
anyway the memory problem persists even in 5.28.00f, where merging improvements were expected.
If also the SCycleOutput are merged 1-by-1 and deleted then i don’t understand why during merging
the proofserv.exe application grows in memory up to more than 2.5gb giving then the usual errors.
Any idea? Gerardo, did you find the time to look at the example code?
best,
Max
Hi all,
i finally managed to test the code with ROOT 5.32.02 and the i found again the same behavior.
The proofserv.exe job starts to eat a lot of memory (more that 20 GB!) in the merging phase when
i setup many histograms (order of 60000 TH1D).
Since, afaiu, Gerardo checked with many TH3 and didn’t observe this effect, i must conclude
this is a “feature” of SFrame, anyway i don’t know how to test this. Any suggestion is more that welcome!
best,
Max
Dear all,
looking at release notes for latest ROOT version 5.34, i saw:
“In TProofOutputFile, add switch to control the way histograms are merged by TFileMerger, i.e. one-by-one or all-in-one-go. The default is one-by-one which requires much less memory. Merging in one-go (the previous default) can be activated by passing ‘H’ in the constructor options.”
I was a bit surprised, since i thought one-by-one was already the default, but ok i probably misunderstood. Anyway i then tested my code in PROOF with this ROOT version but again i found a huge memory consumption
when merging (~30 GB).
I tried to get a feeling to how much is the memory filled my histograms. I took a file made running in steps
and loaded in ROOT:
Footprint of file content (55402 objects loaded): 601104 kB (res), 600552 KB (virt)
I’ve 55402 histograms and they fill about 600 MB of memory. So if the merging would keep in memory only the sum and the next-to-be-merged histograms then it should require about 1.2 GB. So i’m a bit puzzled about the effective way the merging is working… or is working within SFrame that i know it’s not a ROOT-only related question.
I posted here again also to keep a track of the tests i’m doing but if you (Gerri, Attila) prefer we can continue the discussion offline, of course it would be fine.
To me it’s just important we continue the discussion…
best,
Max