PROOF Lite drop in performance

Hello out there,

I am processing a tree with PROOF-lite and I keep running into an intermittent drop in performance. This tends to happen more frequently when processing larger files with larger individual events but I don’t know if that is cause of the problem. See plot below:


When this occurs and I look at top I have many instance of proofserv that aren’t using any CPU.

I have tried changing around the tree cache size with
TProof * p =TProof::Open();
p->SetParameter(“PROOF_UseTreeCache”,1);
p->SetParameter(“PROOF_CacheSize”,Larger Number here);

But that doesn’t seem to help consistently.

I am using ROOT 5.34.09 and I am on a:
uname -a
Linux pike 3.2.0-4-amd64 #1 SMP Debian 3.2.63-2 x86_64 GNU/Linux

Thanks,

Hello,

How many workers do you have?

Could you produce the performance tree (see http://root.cern.ch/drupal/content/creating-and-saving-performance-tree#NewCase) for one run and make it available some how?

G. Ganis

Thanks, for getting back to me,
I have tried it with a variety of different number of workers. Typically between 10-32. I made a performance tree for a run with 20 workers. You can get it from the link below hopefully.

dropbox.com/s/sdegja7upwjqk … .root?dl=0

Hi,
Thanks for the file and sorry if it took so long.

The file corresponds to a run which looks rather ‘normal’ I would say. The initial measurement of the performance is not realistic and it stabilise after a few packets.

In this run, all 20 workers are working until the end as show in the plot (perf.png; the finishing time has a spread of < 1 s) and each worker gets reasonably the same events and packets (evtdist.png).

It is true that a couple of files get fragmented in more packets that the others.
Can you try again by setting

proof->SetParameter("PROOF_Packetizer", "TPacketizer")
proof->SetParameter("PROOF_MaxSlavesPerNode", (Long_t)20)

?
This packetizer maybe more appropriate for these runs (it is the default in current PROOF).

G. Ganis