Philippe, no change: I increased it to 4e+9, yet the speed is exactly the same, and the total usage is still only 218MBytes.
p.s. So the problem has lost its urgency: I changed my code to simply read in the desired variables into very long arrays, and then I run TMath::Sort on those arrays. This allows me to run through the data in 10seconds (instead of the 10hrs I projected ). But of course it would be great if we could resolve this problem, because a random
access of the tree is a highly useful capability…
What is the result in you case of doing:TFile *_file0 = TFile::Open("/raid/home/pnpfuser/data_vx511/root_all/DU_data/2012_04_03_uncompressed.root");
ptree->SetMaxVirtualSize(4e+9);
b = ptree->GetBranch("event");
b->LoadBaskets();
cout << b->GetListOfBaskets()->GetEntries() << '\n';
gROOT->ProcessLine("ptree->Dump(); > ptree.dump");
gROOT->ProcessLine(".! grep TotalBuffers ptree.dump");
b = ptree->GetBranch("tlv_header");
b->LoadBaskets();
cout << b->GetListOfBaskets()->GetEntries() << '\n';
gROOT->ProcessLine("ptree->Dump(); > ptree.dump");
gROOT->ProcessLine(".! grep TotalBuffers ptree.dump");
Ahaaaaa!! Now I see my memory usage skyrocket to 1.6GBytes (the actual file size)! I bet my code will run faster now – I’ll check and let you know in 10min.
So what was the difference? Rather than do LoadBaskets() on ptree, I did it on the actual branch – is that it?
And, to answer your questions, here’s the output:
root [4] cout << b->GetListOfBaskets()->GetEntries() << '\n';
479
root [5] gROOT->ProcessLine("ptree->Dump(); > ptree.dump");
root [6] gROOT->ProcessLine(".! grep TotalBuffers ptree.dump");
fTotalBuffers 2625134592 ! Total number of bytes in branch buffers
root [8] b = ptree->GetBranch("tlv_header");
root [9] b->LoadBaskets();
root [10] cout << b->GetListOfBaskets()->GetEntries() << '\n';
135
root [11] gROOT->ProcessLine("ptree->Dump(); > ptree.dump");
root [12] gROOT->ProcessLine(".! grep TotalBuffers ptree.dump");
fTotalBuffers 3364995072 ! Total number of bytes in branch buffers
It also appears that the total number of bytes in branch buffers is 2x of what it should be…
Yep, it’s running super fast now! So here’s a summary for getting fast random tree access to work:
a) use an uncompressed .root file
b) use TTree::SetMaxVirtualSize(X) where X is twice your file size in bytes.
c) get the particular branch you are interested in from the tree, and do LoadBaskets() on that. Check root’s memory usage, and confirm that it’s close to your file size.
d) do your randomized branch access, it should be very fast at this point.
[quote]It also appears that the total number of bytes in branch buffers is 2x of what it should be…[/quote]Yes this is exactly the problem that I fixed .
[quote]So here’s a summary for getting fast random tree access to work:[/quote]humm …
[quote]a) use an uncompressed .root file[/quote]A priori this is not needed (i.e. I thought it was the original cause of your slow down but it isn’t and the buffer when loaded via LoadBasket are decompressed only once (during LoadBaskets).
[quote]c) get the particular branch you are interested in from the tree, and do LoadBaskets() on that. Check root’s memory usage, and confirm that it’s close to your file size. [/quote]What I don’t yet understand is why in you case, it still fails when you do the TTree::GetEntry after setting the VirtualSize large enough!
[quote]So what was the difference? Rather than do LoadBaskets() on ptree, I did it on the actual branch – is that it?[/quote]No, it should not make a different. However when you do the LoadBasket individually the memory usage (i.e. fTotalBuffers grows ‘slower’).
The next thing to try is:TFile *_file0 = TFile::Open("/raid/home/pnpfuser/data_vx511/root_all/DU_data/2012_04_03_uncompressed.root");
ptree->SetMaxVirtualSize(4e+9);
ptree->LoadBaskets();
b = ptree->GetBranch("event");
cout << b->GetListOfBaskets()->GetEntries() << '\n';
b = ptree->GetBranch("tlv_header");
cout << b->GetListOfBaskets()->GetEntries() << '\n';
gROOT->ProcessLine("ptree->Dump(); > ptree.dump");
gROOT->ProcessLine(".! grep TotalBuffers ptree.dump");
a) one where I do LoadBaskets() on the branch (rather than the tree) form an UNcompressed file and
b) one where I do LoadBaskets() on the branch (rather than the tree) form an compressed file
test A:
Attaching file /raid/home/pnpfuser/data_vx511/root_all/DU_data/2012_04_03_uncompressed.root as _file0...
root [1] ptree->SetMaxVirtualSize(4e+9);
root [2] TBranch* b = ptree->GetBranch("event");
root [3] b->LoadBaskets()
(Int_t)479
root [4] cout << b->GetListOfBaskets()->GetEntries() << '\n';
479
root [5] b = ptree->GetBranch("tlv_header");
root [6] cout << b->GetListOfBaskets()->GetEntries() << '\n';
0
root [7] gROOT->ProcessLine("ptree->Dump(); > ptree.dump");
root [8] gROOT->ProcessLine(".! grep TotalBuffers ptree.dump");
fTotalBuffers 2625134592 ! Total number of bytes in branch buffers
In fact ROOT’s memory usage is 1272856 kBytes.
test B:
Attaching file /raid/home/pnpfuser/data_vx511/root_all/DU_data/compressed/2012_04_03_compressed.root as _file0...
root [1] ptree->SetMaxVirtualSize(4e+9);
root [2] TBranch* b = ptree->GetBranch("event");
root [3] b->LoadBaskets()
(Int_t)29344
root [4] cout << b->GetListOfBaskets()->GetEntries() << '\n';
3
root [5] b = ptree->GetBranch("tlv_header");
root [6] cout << b->GetListOfBaskets()->GetEntries() << '\n';
1
root [7] gROOT->ProcessLine("ptree->Dump(); > ptree.dump");
root [8] gROOT->ProcessLine(".! grep TotalBuffers ptree.dump");
fTotalBuffers 80417385728 ! Total number of bytes in branch buffers
Again, crazy numbers: ROOT’s memusage is only 37996 kBytes.
Okay, this might be another manifestation of the problem. Is there a way for you to send me your compressed file so I can try to reproduce (and fix) the behavior you see (and/or verify the fix I have fixes it ).
Thanks. I was able to download the file and understood the last issue.
Turns out that TTree::LoadBaskets use the value given in a previous call to SetMaxVirtualSize only if being passed a value of 0 or less. The default value used when none is explicitly provided is 2e+9. So to make it work in you case you just need to do:ptree->LoadBaskets(4e+9);.
Due to the data distribution in your files (some/many of the basket are not full), the deficiency lead to more than double counting the actual content. To get the good behavior with the deficent version of ROOT, for your files, you need to use a very large number (i.e. greater than what fTotalBuffers reports). For example:ptree->LoadBaskets(2328673567232LL);
). But of course it would be great if we could resolve this problem, because a random