TSelector with PROOF number selected

ROOT
1

I’m having trouble with a TSelector; both one I’ve written, and an automated one use TTree::MakeSelector.

In essence I wish to simply count the number of events that pass the selection. I have created an instance variable num_events within the TSelector, and ++num_events in the TSelector::Process method.
I then print out the value in TSelector::Terminate.

Simple enough; and it works fine without PROOF. The moment I start a PROOF-lite instance TProof::Open("") and set my TChain to use PROOF …things fall through.
The value is printed a 0.

Now granted, I have no clue how PROOF handles synchronisation, I am aware that you need to add things like histograms to fOutput or something; but it doesn’t like it if I try and add num_events as it’s not a TObject.
Now if I was using thread based programming I would use a worker variable, then unlock a mutex or something before the worker was done, and use the worker variable to update the shared state variable.

I do not see how to do that.

And from the examples in the “tree” and “proof” directories, they have not needed to; they just increment fStatus or fProcessed, and apparently that should work for PROOF too… but I’m not convinced.

Frankly, I’m not even sure that the files are being opened in PROOF mode.

Any thoughts?

2

Hi, I cannot help with PROOF (probably @ganis is the right person for that, so let’s ping him :slight_smile: ) but I can suggest an alternative solution if you have access to ROOT v6.10 or later: TDataFrame.

Example usage: given a function (your function) bool IsGoodEvent(double x, double y) that takes the value of two branches x and y and returns true if the event passes the selection, this is the full code that counts the number of events passing the selection – running over events in parallel on all your cores:

#include "ROOT/TDataFrame.hxx"
using namespace ROOT::Experimental; // TDF lives here

int main() {
  ROOT::EnableImplicitMT(); // enables implicit multi-threading
  TDataFrame d("treeName", {"file1.root", "file2.root"});
  auto n_good_events = d.Filter(IsGoodEvent, {"x","y"}).Count();
  std::cout << *n_good_events << std::endl;
  return 0;
}

Cheers,
Enrico

1 Like
4

I’ve done some testing today; I’ve set the log level for PROOF to 1, and tried to follow through the source code to see where the log messages are generated to see if it seems to be following the right path; to no avail.

I littered each method with std::clog statements and discovered that TSelector::Begin is executed, but TSelector::SlaveBegin (and TSelector::Notify, and TSelector::SlaveTerminate) do not. To confirm this, rather than just printing messages to a terminal (in case of any redirection), I made TSelector::SlaveBegin create files with random names. These files were not created.

In contrast, without PROOF, works fine, all methods are excutes, log messages printed, and random file created.

5

Hi, thanks for the suggestion.

Unfortunately I am restricted at the moment to 5.34/36 but that might change in the future.

Though that’s interesting, I would require more than just counting; like histograms and things. TSelector seems to be build specifically for that purpose. If I could use ROOT 6, I’ll swap out the SetBranchAddress stuff for TTreeReader

But the issue will likely remain.

6

Hi,

TDataFrame allows to do much more than counting. For example:

Cheers,
D

7

Dear parnmatt,

The simplest way you can do that with Proof in 5.34 is shown in the attached example.
Hope it helps until you can move to >= v6.10 aand TDataFrame.

G Ganis

ProofCounter.C (3.2 KB)
ProofCounter.h (993 Bytes)

1 Like
8

Thanks; it’s interesting to use a TParameter when the examples in the proof and tree directories do not.
However, this does not help matters; I have even tried filling a histogram, as per the examples, and it still doesn’t do anything.

As stated before, ::Begin is called, but ::SlaveBegin is not; I believe the slaves are not being created, or something like that.

The output printed to my terminal when using PROOF-Lite is the following:

 +++ Starting PROOF-Lite with 16 workers +++
Opening connections to workers: OK (16 workers)
Setting up worker servers: OK (16 workers)
PROOF set to parallel mode (16 workers)

Info in <TProofLite::SetQueryRunning>: starting query: 1
Info in <TProofQueryResult::SetRunning>: nwrks: 16
Looking up for exact location of files: OK (33 files)
Looking up for exact location of files: OK (33 files)
Info in <TPacketizer::TPacketizer>: Initial number of workers: 16
Validating files: OK (33 files)
Lite-0: all output objects have been merged

My 33 test files take 1.727s according to time; and does nothing.

The same analysis with PROOF disabled (thus sequential, single threaded) takes 9.506s, but actually does something.

Any thoughts as to why ::SlaveBegin might not be being called?

9

Dear parnmatt,

Your PROOF session looks as started correctly and also the dataset (TChain) is correctly validated.
It is weird that SlaveBegin is not called. Where did you created the random files? In /tmp or in the workers sandboxes?
Also, printouts in SlaveBegin, Process and SlaveTerminate go in the workers log files, the workers being separated processes.
You can get the workers logs files with the command:

  root[] TProofLog *pl = TProof::Mgr("lite://")->GetSessionLogs()

and then

  root[] pl->Display("*")

I suggest that you start only 2 workers while trying to understand what is going on:

  root[] TProof *proof = TProof::Open("workers=2")

After that, you should give a bit more details of what you are doing in your TSelector, and also how you run the all thing.

G Ganis

Ok that the documentation is far from complete, but TParameter is used in many examples under tutorials/proof .

10

Sorry, a few things came up that required my attention. I now have a little more time to address issues with this part of the analysis.

My “random files” are a small subset of the monte carlo I’ll be running over. These ROOT files are real, stored on a physical disk which my user (and group) have access to.

If the PROOF-Lite workers are spawned as process threads using my user and/or group permissions, then the workers should be able to see the files. I understand that a full PROOF system may not do that; but PROOF-Lite runs locally (slaves on master).

I still have to check reducing the workers to 2, and reading the sessions logs.
I have tried both Info and std::clog; I can understand the former being remapped, but the second not so much.
I have std::clog pointing to an std::ofstream when testing. The only reason I can see it not keeping that is if the workers are created in a completely separate session, and do not share that state with the master; and using the default std::clog::rdbuf which should point to stderr or stdout depending on the compiler implementation.

I am creating a standalone execcutable, compiled against ROOT libraries (ensuring -lProof, as root-config --libs does not include that library.

I’m executing PROOF in the following way:

TChain chain(file_args.tree.c_str());
fill_chain(file_args.fileset, chain);

struct selection const selection = { 5500.0, { 5.4, 9.0 }, 0.8 };
selector select(&selection);

if (file_args.proof.use) {
    TProof::Open(file_args.proof.url.c_str());
    chain.SetProof(true);
}

chain.Process(&select);

if (file_args.proof.use)
    chain.SetProof(false);

Where selector is my class inheriting from TSelector, which takes a pointer to a struct selection; it’s a struct over a class as I do not need methods (and a struct is just a public class in C++), and it’s not just the values themselves for quick ease, grouping together, and I intend to extend to generating them.

I have a set of parsed command line arguments, which are used containing the tree name, set of files, etc.
The file_args.proof.url is by default an empty string, therefore TProof::Open(file_args.proof.url.c_str()); is equivelent to TProof::Open(""); which runs PROOF-Lite, locally, but allows a user to give a PROOF url on the command line if they wish.

I don’t know what you need to know about the selector itself, though I wrote it from scratch using the documentation, I used the pregenerated TTree::MakeSelector code to check against (which itself didn’t work with PROOF).

I’m using 5.34/36; but that shouldn’t make too much difference to how this works to my understanding.
(I’m still in the process of making our codebase a little more compatable with C++11 and ROOT 6, so I can either use TTreeReader or this niceTDataFrame interface)

11

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.