I have a piece of code that is currently putting heavy strain on the IO of my institutes LUSTRE file system. The code loads data from about 15-20 different files using TChains, merges them using AddFriend, does some simple selection and then dumps the resulting TChain out to a new file. I’m curious if there is anything about the AddFriend method that might be causing significant IO load? Any thoughts?
Edit: I should specify that I am using ROOT/5.34.26/gcc-4.8
The one thing that could goes trouble is if the event in each of the friends are not in the same ‘order’. When this is the case, even-though you read sequentially the main TTree, you would be reading the friend TTree out of order (and thus possibly reading much more than you need).
Another behavior is that (especially if all TTree has data that about the same size), all TTrees might be requesting their data-cluster (typically around 32MB) one right after the other … followed by some quiet time while you process the entries in that data-cluster (and then it repeats).
Since you are using v5.34/26, it is also likely that you are not actually using the TTreeCache, in which case your example would be doing a lots of read request about 32K. If this is the problem, you can solve it by enabling the TTreeCache for each of the chains : tree->SetCacheSize(-1);
Thanks Philippe! My admin tells me it did cut back on the reading load and things are looking much better. I just compiled ROOT6 on it so now might be a good time to migrate.