Best Way to import several arrays of values from ROOT file with cuts using PyROOT

Giorgio_Del_Castello · July 28, 2022, 8:24pm

Hi,
I am having an issue when reading a root file with PyROOT. What i want to do is set some conditions on the tree that i read from the file and then extract all the values of the leaves that pass the selection. For now I am doing this:

import ROOT
import numpy as np
import copy
def read_var(myvar:str,datafile:ROOT.TTree,cuts:str =''):
    Nentries = datafile.Draw(myvar,cuts,"goff")
    if Nentries==0:
        print("Empty buffer")
        return False
    res= np.frombuffer(datafile.GetV1(),count=Nentries)
    return copy.deepcopy(res)

This not only is extremely slow, but if the file is particularly big it takes so much time that on jupyter notebook it stops executing the cell.
Thank you for your help,
Giorgio

couet · July 29, 2022, 7:08am

I am sure @etejedor can help you with this python question.

etejedor · July 29, 2022, 8:09am

Hello,

I understand the expensive part here is the copy?

You can consider using RDataFrame:

https://root.cern/doc/master/classROOT_1_1RDataFrame.html

With RDataFrame you could do the filters you need based on the values of your dataset columns (with Filter) and then, if you want a NumPy array out of a given column, you can call RDataFrame.AsNumpy.

etejedor · July 29, 2022, 8:10am

Here’s a tutorial for AsNumpy:

https://root.cern/doc/master/df026__AsNumpyArrays_8py_source.html

Giorgio_Del_Castello · July 29, 2022, 8:59am

Thank you a lot, for the reply! I will have a look at both suggestions !

Giorgio_Del_Castello · July 29, 2022, 9:03pm

Hi, I encountered another problem while trying out the RDataFrame, my root files are filled by custom made classes (that are usable in the root shell and in pyroot) but when i do AsNumpy it gives a series of errors due to the lack of the classes. The framework that I am using is called Diana, and it contains all the said classes and in pyroot it appears as a namespace inside the ROOT package.

How can i solve this issue ?
Here is the series of errors it gives.

df = ROOT.RDataFrame('qtree','Production_290011_T_p001.root')

IncrementalExecutor::executeFunction: symbol ‘ZN9__gnu_cxx13new_allocatorIN5Diana7QObjectEE9constructIS2_JRS2_EEEvPT_DpOT0’ unresolved while linking symbol ‘__cf_20’!
You are probably missing the definition of void __gnu_cxx::new_allocatorDiana::QObject::construct<Diana::QObject, Diana::QObject&>(Diana::QObject*, Diana::QObject&)
Maybe you need to load the corresponding shared library?
IncrementalExecutor::executeFunction: symbol ‘ZSt10_ConstructIN5Diana7QObjectEJS1_EEvPT_DpOT0’ unresolved while linking symbol ‘__cf_20’!
You are probably missing the definition of void std::_Construct<Diana::QObject, Diana::QObject>(Diana::QObject*, Diana::QObject&&)
Maybe you need to load the corresponding shared library?
IncrementalExecutor::executeFunction: symbol ‘ZSt10_ConstructIN5Diana7QObjectEJRS1_EEvPT_DpOT0’ unresolved while linking symbol ‘__cf_20’!
You are probably missing the definition of void std::_Construct<Diana::QObject, Diana::QObject&>(Diana::QObject*, Diana::QObject&)
Maybe you need to load the corresponding shared library?
IncrementalExecutor::executeFunction: symbol ‘ZN9__gnu_cxx13new_allocatorIN5Diana7QObjectEE9constructIS2_JRS2_EEEvPT_DpOT0’ unresolved while linking symbol ‘__cf_21’!
You are probably missing the definition of void __gnu_cxx::new_allocatorDiana::QObject::construct<Diana::QObject, Diana::QObject&>(Diana::QObject*, Diana::QObject&)
Maybe you need to load the corresponding shared library?
IncrementalExecutor::executeFunction: symbol ‘ZSt10_ConstructIN5Diana7QObjectEJS1_EEvPT_DpOT0’ unresolved while linking symbol ‘__cf_21’!
You are probably missing the definition of void std::_Construct<Diana::QObject, Diana::QObject>(Diana::QObject*, Diana::QObject&&)
Maybe you need to load the corresponding shared library?
IncrementalExecutor::executeFunction: symbol ‘ZSt10_ConstructIN5Diana7QObjectEJRS1_EEvPT_DpOT0’ unresolved while linking symbol ‘__cf_21’!
You are probably missing the definition of void std::_Construct<Diana::QObject, Diana::QObject&>(Diana::QObject*, Diana::QObject&)
Maybe you need to load the corresponding shared library?
IncrementalExecutor::executeFunction: symbol ‘ZN9__gnu_cxx13new_allocatorIN5Diana7QObjectEE9constructIS2_JRS2_EEEvPT_DpOT0’ unresolved while linking symbol ‘__cf_22’!
You are probably missing the definition of void __gnu_cxx::new_allocatorDiana::QObject::construct<Diana::QObject, Diana::QObject&>(Diana::QObject*, Diana::QObject&)
Maybe you need to load the corresponding shared library?
IncrementalExecutor::executeFunction: symbol ‘ZSt10_ConstructIN5Diana7QObjectEJS1_EEvPT_DpOT0’ unresolved while linking symbol ‘__cf_22’!
You are probably missing the definition of void std::_Construct<Diana::QObject, Diana::QObject>(Diana::QObject*, Diana::QObject&&)
Maybe you need to load the corresponding shared library?
IncrementalExecutor::executeFunction: symbol ‘ZSt10_ConstructIN5Diana7QObjectEJRS1_EEvPT_DpOT0’ unresolved while linking symbol ‘__cf_22’!
You are probably missing the definition of void std::_Construct<Diana::QObject, Diana::QObject&>(Diana::QObject*, Diana::QObject&)
Maybe you need to load the corresponding shared library?
IncrementalExecutor::executeFunction: symbol ‘ZN9__gnu_cxx13new_allocatorIN5Diana7QObjectEE9constructIS2_JRS2_EEEvPT_DpOT0’ unresolved while linking symbol ‘__cf_23’!
You are probably missing the definition of void __gnu_cxx::new_allocatorDiana::QObject::construct<Diana::QObject, Diana::QObject&>(Diana::QObject*, Diana::QObject&)
Maybe you need to load the corresponding shared library?
IncrementalExecutor::executeFunction: symbol ‘ZSt10_ConstructIN5Diana7QObjectEJS1_EEvPT_DpOT0’ unresolved while linking symbol ‘__cf_23’!
You are probably missing the definition of void std::_Construct<Diana::QObject, Diana::QObject>(Diana::QObject*, Diana::QObject&&)
Maybe you need to load the corresponding shared library?
IncrementalExecutor::executeFunction: symbol ‘ZSt10_ConstructIN5Diana7QObjectEJRS1_EEvPT_DpOT0’ unresolved while linking symbol ‘__cf_23’!
You are probably missing the definition of void std::_Construct<Diana::QObject, Diana::QObject&>(Diana::QObject*, Diana::QObject&)
Maybe you need to load the corresponding shared library?
IncrementalExecutor::executeFunction: symbol ‘ZN9__gnu_cxx13new_allocatorIN5Diana7QObjectEE9constructIS2_JRS2_EEEvPT_DpOT0’ unresolved while linking symbol ‘__cf_24’!
You are probably missing the definition of void __gnu_cxx::new_allocatorDiana::QObject::construct<Diana::QObject, Diana::QObject&>(Diana::QObject*, Diana::QObject&)
Maybe you need to load the corresponding shared library?
IncrementalExecutor::executeFunction: symbol ‘ZSt10_ConstructIN5Diana7QObjectEJS1_EEvPT_DpOT0’ unresolved while linking symbol ‘__cf_24’!
You are probably missing the definition of void std::_Construct<Diana::QObject, Diana::QObject>(Diana::QObject*, Diana::QObject&&)
Maybe you need to load the corresponding shared library?
IncrementalExecutor::executeFunction: symbol ‘ZSt10_ConstructIN5Diana7QObjectEJRS1_EEvPT_DpOT0’ unresolved while linking symbol ‘__cf_24’!
You are probably missing the definition of void std::_Construct<Diana::QObject, Diana::QObject&>(Diana::QObject*, Diana::QObject&)
Maybe you need to load the corresponding shared library?
Traceback (most recent call last):
File “”, line 1, in
File “/usr/local/root6.22/install/lib/ROOT/pythonization/_rdataframe.py”, line 72, in RDataFrameAsNumpy
result_ptrs[column] = df.Takecolumn_type
ValueError: Could not instantiate TakeDiana::QObject:
none of the 2 overloaded methods succeeded. Full details:
ROOT::RDF::RResultPtr<vectorDiana::QObject > ROOT::RDF::RInterfaceROOT::Detail::RDF::RLoopManager,void::Take(experimental::basic_string_view<char,char_traits > column = “”) =>
ValueError: nullptr result where temporary expected
ROOT::RDF::RResultPtr<vectorDiana::QObject > ROOT::RDF::RInterfaceROOT::Detail::RDF::RLoopManager,void::Take(experimental::basic_string_view<char,char_traits > column = “”) =>
ValueError: nullptr result where temporary expected
none of the 3 overloaded methods succeeded. Full details:
ROOT::RDF::RResultPtr<vectorDiana::QObject > ROOT::RDF::RInterfaceROOT::Detail::RDF::RLoopManager,void::Take(experimental::basic_string_view<char,char_traits > column = “”) =>
ValueError: nullptr result where temporary expected
ROOT::RDF::RResultPtr<vectorDiana::QObject > ROOT::RDF::RInterfaceROOT::Detail::RDF::RLoopManager,void::Take(experimental::basic_string_view<char,char_traits > column = “”) =>
ValueError: nullptr result where temporary expected
ROOT::RDF::RResultPtr<vectorDiana::QObject > ROOT::RDF::RInterfaceROOT::Detail::RDF::RLoopManager,void::Take(experimental::basic_string_view<char,char_traits > column = “”) =>
ValueError: nullptr result where temporary expected

etejedor · August 1, 2022, 2:33pm

Hello,

It’s strange that you get these messages if you can use the classes normally from Python and you did not have the same problem when you used TTree::Draw.

@pcanal any ideas?

Giorgio_Del_Castello · August 3, 2022, 4:53pm

Yes, I am also quite puzzled. Is there any way i can investigate the problem ?

pcanal · August 4, 2022, 8:08pm

What was the actual RDataFrame calls? You may need (at least of now) to explicit load the diana library that inplements Diana::QObject.

system · August 18, 2022, 8:08pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.