I am using root 5.06/00 on Windows XP, built from sources using MSVC++.NET 2003. We have a data acquisition program that writes TTree data to disk (we have large buffers to to hold data when the OS decides to do its non-RTOS tasks) We are running into a limit of how fast we can write data. Our TTree averages 313 bytes/record divided accross about 25 branche - many are non-changing. I use default compression (compression factor is about 20) The limit we are hitting is about 50000 TTree::Fill()'s per second. This is on a server with 2.8 GHz P4 , 512 MBytes ram and a 7200 rpm SATA drive. Profiling shows that about 85% of the time is in TTree::Fill() and methods it calls (TBuffer::ResetMap, R__zip etc…)
One thing that is particularly puzzling is that when I use the Windows Task Maneger, the process shows ~5000 page faults per second. Again, this is caused by the TTRee::Fill() or methods it calls (the faults stop when I comment out call to TTree::Fill()
Can you provide any guidelines on how I might improve the throughput? Would a later version of root help? Are there any optimizations I might consider when building root (I used the make from the sources tar file)
Could you send the output of mytree.Print() at the end of the Fill process?
Do I understand correctly that you write about 15 MBytes/s of uncompressed data, ie about 1MByte/s of compressed data?
Are you streaming only basic types, objects, objects with maps?
The ideal would be to have a small file with your Tree.
Rene,
Yes, you are correct. I am only streaming basic type - no objects, etc. I’ve attached a small file with my TTree - 1st 5000 entries - The overall compression in the file is 2.86 - this is actually more typical for a compression factor.
Thanks! test.root (221 KB)
Just as a very short test, I have taken your file and created a small script
filling a Tree with a similar structure with about twice the number of branches).
see the code in attachement.
This programs executes in 17s on my Linux machine (P I$ 2.8 Ghz).
It creates a Tree of 168 MBytes of uncompressed data and 33 MBytes
when compressed.
This performance is more or less expected when mixing many data types.
On Windows (the worst possible environment to do bulk I/O) it could be
that the performance is less than half of Linux.
Let me know how long it takes to do
root > .L nt.C+
root > nt();
On my desktop (not acquisition machine) running windows XP (p4 3 GHz) it takes 14 seconds - I guess I’m doing as well as can be expected. One difference between the test and the actual app is that I call TTree::AutoSave() every 30 seconds. Thanks for your help.
Ed