Problems with TProof::Load()

ROOT
1

Hi,

We’ve been trying to do an analysis with other classes so we use proof->Load(“*.C++”).

Our proof cluster is composed of nodes running two workers each. When we do this load it seems to only load on the first of the worker on each node. So the first worker works just fine, but the second one can’t link the selector because it has undefined symbols. It seems not to be trying to load the library at all. Is there some way to tell it to load on both or am I doing something else wrong.

This is the script we used:
{
//Setup include
TString base_tools = “/homes/gaia/mschram/analysis/tools/”;
gSystem->AddIncludePath(" -I"+base_tools);

//Setup PROOF
TProof *proof = TProof::Open(“rabidec@gaia040”);
proof->Load(base_tools+“ParticleObj.C+”);
proof->SetParallel(16);
proof->Print();

//Setup chain
TChain *chain = new TChain(“ntp1”);

chain->SetProof();
TString data_base =“root://gaia040:1093//exports/data/pool/proofpool/”;
//chain->Add(data_base+“BpBm/MergedRaw/r22d_BpBm_Run1_Merged_Aug8_2008.root”);
chain->Add(data_base+“r22d_BpBm_Run1_Merged_Aug8_2008.root”);

//Process data
TStopwatch this_sw;
chain->Process(“/homes/gaia/mschram/analysis_proof/selectors/BabarSelectorKtau
tau.C+”,“”);
cout << this_sw->CpuTime() << “/” << this_sw->RealTime() << endl;
}

I did an ls in the directories holding this session on one of the nodes and this is what I saw, note the lack of ParticleObj_C.so for the second worker:

gaia048:/exports/data/pool/proofbox/rabidec$ ls session-gaia040-1218640049-22017/worker-0.6-gaia048-1218640051-31056
BabarSelectorKtautau.C BabarSelectorKtautau.h ParticleObj_C.so
BabarSelectorKtautau_C.so ParticleObj.C ParticleObj.h
gaia048:/exports/data/pool/proofbox/rabidec$ ls session-gaia040-1218640049-22017/worker-0.7-gaia048-1218640051-31060
BabarSelectorKtautau.C BabarSelectorKtautau.h ParticleObj.h
BabarSelectorKtautau_C.so ParticleObj.C

And this is the sample output of a log:
// --------- Start of element log -----------------

// Ordinal: 0.13 (role: worker)

// Path:rabidec@gaia051.beowulf.com:1093//exports/data/pool/proofbox/rabidec/ses
a040-1218641552-23434/worker-0.13-gaia051-1218641554-11326.log

// # of retrieved lines: 19

// ------------------------------------------------

11:32:37 11326 Wrk-0.13 | Info in TUnixSystem::ACLiC: creating shared
library/exports/data/pool/proofbox/rabidec/session-gaia040-1218641552-23434/work
-gaia051-1218641554-11326/./BabarSelectorKtautau_C.so

dlopen error:/exports/data/pool/proofbox/rabidec/session-gaia040-1218641552-2343
-gaia051-1218641554-11326/./BabarSelectorKtautau_C.so: undefined symbol:
_ZTV11ParticleObj

Load Error: Failed to load Dynamic link library/exports/data/pool/proofbox/rabid
-gaia051-1218641554-11326/./BabarSelectorKtautau_C.so

/usr/lib/gcc/x86_64-redhat-linux/3.4.6/…/…/…/…/lib64/crt1.o(.text+0x21):
In function `_start’:

: undefined reference to `main’

/exports/data/pool/proofbox/rabidec/session-gaia040-1218641552-23434/worker-0.13
-gaia051-1218641554-11326/./filegCzG0F.o(.text+0x394a): In function
`BabarSelectorKtautau::Process(long long)':

: undefined reference to `ParticleObj::ParticleObj()’

/exports/data/pool/proofbox/rabidec/session-gaia040-1218641552-23434/worker-0.13
-gaia051-1218641554-11326/./filegCzG0F.o(.gnu.linkonce.t._ZN11ParticleObjD1Ev+0x
13): In function `ParticleObj::~ParticleObj()':

: undefined reference to `vtable for ParticleObj’

/exports/data/pool/proofbox/rabidec/session-gaia040-1218641552-23434/worker-0.13
-gaia051-1218641554-11326/./filegCzG0F.o(.gnu.linkonce.t._ZNSt6vectorI11Particle
ObjSaIS0_EEC1Em+0x76): In function `std::vector<ParticleObj,
std::allocator >::vector(unsigned long)':

: undefined reference to `ParticleObj::ParticleObj()’

/exports/data/pool/proofbox/rabidec/session-gaia040-1218641552-23434/worker-0.13
-gaia051-1218641554-11326/./filegCzG0F.o(.gnu.linkonce.t.ZSt10_ConstructI11Part
icleObjS0_EvPT_RKT0+0x33): In function `void
std::_Construct<ParticleObj, ParticleObj>(ParticleObj*, ParticleObj
const&)':

: undefined reference to `ParticleObj::ParticleObj(ParticleObj const&)’

/exports/data/pool/proofbox/rabidec/session-gaia040-1218641552-23434/worker-0.13
-gaia051-1218641554-11326/./filegCzG0F.o(.gnu.linkonce.t.ZNSt6vectorI11Particle
ObjSaIS0_EE13_M_insert_auxEN9__gnu_cxx17__normal_iteratorIPS0_S2_EERKS0+0x5c):
In function `std::vector<ParticleObj, std::allocator

::_M_insert_aux(__gnu_cxx::__normal_iterator<ParticleObj*,
std::vector<ParticleObj, std::allocator > >, ParticleObj
const&)':

: undefined reference to `ParticleObj::ParticleObj(ParticleObj const&)’

collect2: ld returned 1 exit status

*** Interpreter error recovered ***

11:32:45 11326 Wrk-0.13 | Error in TSelector::GetSelector: file
BabarSelectorKtautau.C+ does not have a valid class deriving from
TSelector

11:32:45 11326 Wrk-0.13 | Error in TProofPlayerSlave::Process: cannot
load: BabarSelectorKtautau.C+

// --------- End of element log -------------------

Thanks,
Charles

2

Hi Charles,

So you have 8 machines and want to have 16 workers?
You call:

proof->SetParallel(16);

only after

 proof->Load(base_tools+"ParticleObj.C+");

It should rather be the reverse. Anyway SetParallel is a function to limit the number of workers. If you don’t want to limit it you don’t need to call it.
How many workers do you get after simply calling TProof::Open
(one can check it with gProof->GetParallel() ) ?

Cheers,
Jan

3

Hi all

I have the same problem with TProof::Load(), trying to load a macro, - when there is more than 1 worker per machine, it seems to load only for the first worker. My current workaround is to call gProof->Exec(gSystem->Load(“Whatever.so”)), as this one does propagate to all the workers.

We have 8 cores and 8 workers per machine, and there is no call to SetParallel().

Anna

4

Hi Anna and Rabidec,

Which version of ROOT are you using?
I tired TProof::Load with the trunk and it loaded my macro successfully.

Cheers,
Jan

5

Jan,

It’s the 5-21-01 tag alice version

6

Hi,

This problem should be now fixed in the trunk.
Please let me know if you try.

Sorry for the inconvenience.

Gerri