I am having issues using TTreeFormulas in looping over a tree using PyROOT.
I choose to use TTreeFormula as there are arrays in the tree and I wish to use them to form new variables. I see no issue when accessing certain elements but for some reason the TTreeFormula returns 0.0 for variables indexed [0][2] yet returns the correct value for the same variables indexed [0][1]. I have used the exact same expressions in the TTreeFormula to draw histograms using TTree::Draw and found that this gives the correct distribution of non-zero values.
Can anyone shed some light on why this might be? Am I doing anything wrong?
I attach the python script (PyROOT_testtree.py), test tree in a root file (testtree.root), and an example output with the histograms in (PyROOT_newtree.root). The script should be executable in the same directory as you have testtree.root. PyROOT_newtree.root (6.64 KB) testtree.root (428 KB) PyROOT_testtree.py (6.64 KB)
I’m not understanding what I’m looking at, but in any case the script does not run: first, testree.root is expected (missing a ‘t’), which is easily fixed if I make a guess, but then still:
Beyond that, all values that I see printed are 0.0.
Do you have a simple script that shows the problem and only the problem? Reason being, that in cases like this, I prefer to first rewrite it to a .C to find out whether it’s a python issue, or an issue in the use of TTreeFormula.
Sorry I uploaded the wrong script - I have stripped it down to the bare guts. I only select to histogram a small subset of variables to prove the issue I seem to be having with the 2D arrays in TTreeFormula. I only print out those variables that resolve to 0.0 as this is flagging errors. The thing I have been comparing is in the output file where the histograms filled from the TTreeFormula::EvalInstance() are subscripted with “he_” and histograms obtained from TTree::Draw which are subscripted “hd_” for a number of variables
I attach the script again with example output which should work with the original testtree.root
I have transferred all of of my code to C and have found that one needs to call TTreePlayer:: UpdateFormulaLeaves() after loading a TTree’s entry via TTree::GetEntry.
When I add this line into the python script before the TTreeFormula::EvalInstance() calls it still does not fix the issue I am seeing.
sorry, but I’m still getting these:Error in <TFile::TFile>: file /data/still/SK_MVA/v0r2/neut/13a/allweight/PyROOT_weights_nue_sig.root does not exist
which are coming from the call to GetEntry() as that file apparently contains a reference to a friend:#0 0x00007ffff615b17c in TFile::TFile(char const*, char const*, char const*, int) () from /home/wlav/rootdev/root/lib/libRIO.so
#1 0x00007ffff6165683 in TFile::Open(char const*, char const*, char const*, int, int) () from /home/wlav/rootdev/root/lib/libRIO.so
#2 0x00007ffff5c37589 in TFriendElement::GetFile() () from /home/wlav/rootdev/root/lib/libTree.so
#3 0x00007ffff5c37409 in TFriendElement::GetTree() () from /home/wlav/rootdev/root/lib/libTree.so
#4 0x00007ffff5c1721e in TTree::GetEntry(long long, int) () from /home/wlav/rootdev/root/lib/libTree.soSo, if GetEntry() does not succeed properly, I’m not sure what to expect from any output.
Really, this is what I get on your testtree.root: % root -l -b testtree.root
root [0]
Attaching file testtree.root as _file0...
root [1] h1->GetEntry(0)
Error in <TFile::TFile>: file /data/still/SK_MVA/v0r2/neut/13a/allweight/PyROOT_weights_nue_sig.root does not exist
(Int_t)8888
root [2]after that then, I have to mistrust any and all operations on the tree …
I also don’t understand why you add a PyROOT_newtree.root, since the first thing that happens in the script, is that it gets replaced.
This:t_in=t_in.CopyTree('')definitely seems wrong (or pointless, anyway) as it deletes the TChain you just created.
TTreePlayer:: UpdateFormulaLeaves() is a no-op as the list of formulas that the tree player (t_in.GetPlayer() anyway) has is empty (size 0).
So again, in order to help (and as I said, this really isn’t my cup of tea), I need a script that shows the problem, and only the problem, and does not have any other issues. Plus be as minimal as possible.
I attach a stripped version of the tree which should have no friends. The script uses three methods of accessing values of variables and saves them to histograms in the output file for comparison:
The event histograms, prefixed with ‘he_’ in the output, use the TTreeFormula::EvalInstance method which I mentioned I was having issues with.
The tree histograms, prefixed with ‘ht_’ in the output, use the values from the TTreePlayer instance
The draw histograms, prefixed with ‘hd_’ in the output, are a control sample and use the TTree::Draw method.
sorry for the late reply, but I’m traveling (and will be for quite some time yet).
Thanks, this worked. Next step, I added ‘k.GetTree().GetReadEntry()’ to your prints, so that I can see which entries are problematic. I get:[quote]4 INFINITY fqenll with equation fq1rnll[0][1] has a value of inf
4 NaN fqpi0enllratio with equation fqpi0nll[0]/fq1rnll[0][1] has a value of nan
38 INFINITY fqenll with equation fq1rnll[0][1] has a value of inf
38 NaN fqpi0enllratio with equation fqpi0nll[0]/fq1rnll[0][1] has a value of nan
51 NaN fqpi0enllratio with equation fqpi0nll[0]/fq1rnll[0][1] has a value of nan
63 INFINITY fqenll with equation fq1rnll[0][1] has a value of inf
63 NaN fqpi0enllratio with equation fqpi0nll[0]/fq1rnll[0][1] has a value of nan[/quote]
Next, I look at them individually:[code]>>> t_in.Scan(“fq1rnll[0][1]”)
…snip…
2 * 6603.0488 *
3 * 19844.998 *
4 * inf *
5 * 8930.3759 *
6 * 1495.4970 *
…snip…
37 * 14235.050 *
38 * inf *
39 * 732.83825 *
…snip…
62 * 303.80163 *
63 * inf *
64 * 13948.049 *
65 * 16989.755 *[/code]and[code]>>> t_in.Scan("")
…snip…
50 * 13589.851 *
51 * -nan *
52 * 43516.921 *[/code]
So yes, those entry contain infs and nans in the tree on file.
Next, if I “print fqenll, fqmunll”, then I see those infs as well. Python allows calculating with them, though: value/inf becomes 0., for example, and inf/inf becomes nan. Which seems reasonable.
Before you were talking about values being 0, now inf and nan. But AFAICS, this is all correct. What am I missing?
I’ll have to build ROOT on my laptop to look at the histos. Will take a bit … What should I expect to see?
the TTreeFormulaManager of the TTreeFormula needs to have its GetNdata() called for the TTreeFormula. Since each formula has its own manager, I did: #k.UpdateFormulaLeaves()
if j == 0: k.GetNdata()
I found this call in the TTreePlayer code. Beats me why this has to be so, however.
Anyway, with that line, your ‘he_’ histograms now look the same as the other ones.
For the benefit of those who will google to this thread in the future, searching for “how to evaluate an expression from a tree”, “how to use a TTreeFormula in pyroot” and the like, here’s a quick simplified summary of the solution Ben and Wim achieved:
import ROOT as R
form = R.TTreeFormula( 'name', 'expression_using_leaves_of_the_tree', input_tree )
form.GetNdata()
for event in input_tree:
val = form.EvalInstance()
For example, I have a tree called “g” with leaves “event” and an array called “ke”, so this works:
import ROOT as R
form = R.TTreeFormula( 'form', 'event+ke[0]', g )
form.GetNdata()
for ev in g:
print form.EvalInstance()