Fast close method

ROOT
1

Hi Experts,

I’m PROOF-lite user.
My code produce TFile which have huge amount of TH1(30,000 TH1F, ~2GB).
It take very long time(few hours) when it delete created TFile and many TH1Fs.
Currently I’m just using delete like

But can I speed up this process ?
Could you tell me fast close technique ?

Thanks,
Kenji

2

Hi,

Do you mean that each worker creates 2GB of histograms in memory?
How much memory do you have per worker or core?
My suspicion is that your machine start swapping towards the end because of memory issues.
If this is the reason I do not see an easy way out, if not running with less workers (half, for example).
If you do not need the histograms until the end you may flush the not required histograms to file to free memory.

Anyhow, to be more concrete, I need
1: to see you code or a simplified version of it
2: to know the version of ROOT you are using
3: to know more about the machine on which you are running (cores, memory, …)

G. Ganis

3

Hi,

I will put simplified code later.
My root version is 5.34.09 with GCC4.3 on SLC6.
My machine has 6core(12thread) cpu and 16GB memory.
Basically I process my code with 6workers, and they using 12GB memory.

Cheers,
Kenji

4

Hi,

I misunderstood what make the problem.
Proof code shows the progress of jobs like

But it overestimate the number of processed events around twice.
e.g.)
Proof shows 100% of events are processed but only 50% events are processed.

If I use GCC4.6, proof shows correct values.

Do you know what make this problem ?

Regards,
Kenji

5

Hi,
Sorry for the late follow-up.
I do not know if the problem persists.
Anyhow, we never saw problems with the progress bar. However, gcc 4.3 was never supported compiler for SLC6, so unless it happens in other configurations, I would just ignore.
For the file closing issue, you may give a try with later version of ROOT (e.g. 5.34/14); a few optimizations have been done in parts of ROOT which may affect this.

G. Ganis