File buffered dynamic array

[gfronze]

Dear Fellow ROOTers,

I’m writing you to discuss the best option to solve an issue I am facing in my code.

For analysis purposes I have to download a huge number of lectures of HV and current of a detector of ALICE. Both those kinds of values have to be saved in the same “vector”, sorted with respect to their timestamp, then processed.

At first I have used some TLists and TSortedLists, then I started experiencing some performance problems when looping over huge lists. I moved then to TObjArrays. I excluded TClonesArray because the HV and current measurements are stored as objects of two classes respectively (both classes have the same parent class).

When everything was working as expected with a small data sample, I tried to download a huge amount of data. At that point I started having some problems with RAM occupancy and noticed that after a long download session the RAM pressure was enough to make the system kill the process.

To complete my project I need for a vector-like type, sortable, which is buffered to disk and kept in the RAM only in buckets (a sort of TTree?).

Thank you so much for your help!

Gabriele

[pcanal]

You might be able to use a TTree with a TTreeIndex (see TTree::BuildIndex) based on your timestamp.

[gfronze]

Hello pcanal,

Indeed this is the road I am trying to follow.
Currently I have 72 arrays of data, one for each detector element.
In you opinion should I use 72 TTrees or a single TTree with multiple branches set to contain a TObjArray each?
If you suggest to use several TTrees am I allowed put in the TTree objects of different classes belonging to the same parent class?

[pcanal]

am I allowed put in the TTree objects of different classes belonging to the same parent class?

Yes.

In you opinion should I use 72 TTrees or a single TTree with multiple branches set to contain a TObjArray each?

If all the array have the same timestamp then yes, I would put them in a single TTree so that they can share the index. If they all have different timestamp then you will need to put them in a separate TTree.

Cheers,
Philippe.

[gfronze]

Dear Philippe,

Thank you for the information!
Indeed the 72 detector elements are not read in a synchronous way, so I suppose I should go with 72 TTrees.

I have managed to test BuildIndex to order the TTree filled with integers.
Have you any clue on what should I use as major and minor if I want to use a data member of the object?

Many thanks!

Gabriele

[pcanal]

Have you any clue on what should I use as major and minor if I want to use a data member of the object?

You need to be able to make one integer out of the timestamp.

[gfronze]

Roger that. Do you mean another branch pointing merely the object timestamp, right?

[pcanal]

Not really. You can have in the index expression anything that works for a TTreeFormula including a function for the timestamp object if any.

[gfronze]

Great!

Given that my parent class has a method:

inline ULong64_t GetTimeStamp() const { return fTimeStamp; };

How should I pass it to the TTree::BuildIndex method?

[pcanal]

Easy

fTimeStamp

You might need to prefix with the branchname.

Cheers,
Philippe.

[gfronze]

Awesome, so I suppose TTree knows the internal structure of the objects!
Thank you so much for your help! Good night!