Recommended Container for Multidimensional Data

Hi, I’d like to know what is the recommended ROOT container type for storing multi-dimensional data in a TTree. Can you advise for both fixed-sized and variable-sized containers?

In my case, I have 3 fixed dimensions and 1 variable-sized: The three fixed dimensions are as such:
(some fixed list of floats) x (some fixed list of integers) x ( indexes from 0 to 27)
and for each combination of those, the data point is an std::vector with a variable number of elements.

I’ve several times tried to use arrays of std::vector, or arrays of TVectorD, or other combinations, but they never feel satisfactory. I’d like to store this in a big multidimensional container because later I’d like to iterate over these values. What do experts recommend here?

Jean-François

So I had the brilliant idea of using a TMap, but the things stored therein need to inherit from TObject, so that doesn’t allow floats and ints and such.

My solution was to use a TVectorD as the key, with the elements being (first dimension, second dimension, third dimension). It’s a bit wasteful, but I can’t use TArray or anything simpler because they don’t inherit from TObject.

The code that creates the TTree compiles and runs (very slowly now) and the output TTree seems to contain a TMap with actual stuff in it. I try to make a TMap * and use SetBranchAddress to get it, and indeed this TMap gets filled with something when I do TTree::GetEntry(foo). The problem is now when I make a new TVectorD key with values that I know should be present, TMap::FindObject returns a pointer to 0x0 as if the key was not present. If I try TMap::Print() I get values that look like what I expect, but I can’t get to the data itself.

There are many parts where this operation can go wrong, so I am hoping someone will tell me there is a better/recommended way to store multidimensional data in a TTree.

Here’s what I do (this is inside the Loop method of an analysis class created from MakeClass):

  // The values that will go into the keys
  Double_t algo0_smoothings[] = {0,1,2,3,4,5,6,7,8,9};
  Double_t algo0_thresholds[] = {0,50,100,150,200,250,300,350,400,450};
  int N = 10;

  TMap clu_algo0(2800); // 10x10 algo0 values and up to 28 hit channels
  clu_algo0.SetOwnerKeyValue(kTRUE,kTRUE);

  // I omit the code where I create the TTree * ...
  outtree->Branch("clu_algo0","TMap",&clu_algo0,32000,0);

  for(int jentry = 0; jentry < nentries; jentry++) {
    input_tree->GetEntry(jentry);
    // Now we are inside the for loop over all the entries in the input TTree:
    clu_algo0.DeleteAll();
    for(int i : hit_channels){
      // Now we are inside a loop over a variable-number of hits in the range (0,27)
	      for(Int_t j = 0;j < N ;j++)
		{
		  Double_t smooth = algo0_smoothings[j];
		  for(Int_t k = 0;k < N;k++)
		    {
		      Double_t thresh = algo0_thresholds[k];
		      TVectorD * key = new TVectorD(3);
		      (*key)[0] = hit_channel;
		      (*key)[1] = smooth;
		      (*key)[2] = thresh;

                      // I omit the code where I fill the clusters_tv with numbers.
		      TVectorD * clusters_tv = new TVectorD(clusters.size());
		      clu_algo0.Add(key,clusters_tv);
		    }
		}
    } // Ends the loop over the hit channels
  outtree->Fill();
  } // Ends the loop over all the entries in the input tree.

In trying to read the TMap from the output tree, I am using the interpreter, and I am trying the following:

 root -l outfiles/20140507/run00121/correction0/Proto2Analyzed.root 
*  ROOT v5.34/18  *

root [0] 
Attaching file outfiles/20140507/run00121/correction0/Proto2Analyzed.root as _file0...
Warning in <TClass::TClass>: no dictionary for class Output_Vars is available
Warning in <TClass::TClass>: no dictionary for class Hot is available
root [1] TMap * clu_algo0 = new TMap(2800)
root [2] Proto2Analyzed->SetBranchAddress("clu_algo0",&clu_algo0)
(const Int_t)0
root [3] Proto2Analyzed->GetEntry(35)
(Int_t)154547
root [4] TVectorD * key = new TVectorD(3)
root [5] (*key)[0] = 12
(const int)12
root [6] (*key)[1] = 5
(const int)5
root [7] (*key)[2] = 100
(const int)100
root [8] clu_algo0->FindObject(key)
(const class TObject*)0x0
root [9] clu_algo0->GetEntries()
(const Int_t)800

I can independently check that the TTree entry #35 does have a valid hit_channel for channel 12, so the key (12,5,100) should be present.

Thanks for any eyes on my code. I can try to explain better if someone asks, but I am worried that my now-rambly post will be ignored.

Jean-François

I’ve composed a minimal example showing how the written TMap’s contents cannot be retrieved from the TTree with a seemingly-identical TVectorD key. It is attached to this post. The result is the same in compiled mode or interpreted mode.

Is this something to do with the hash algorithm used for the keys? Do two different TVectorDs with the same values have different hashes or something? What are some other value containers that I could use for the TMap key?

Jean-François
maptest.C.txt (1.39 KB)
maptest.C (1.39 KB)

So I’ve confirmed that two different TVectorDs with the same content can have different Hash values, so when they go into a TMap, there is no way to recover them “by their content”. Actually I wasn’t able to super-confirm this, because I couldn’t call TVectorD::Hash() on a TVectorD in the interpreter…

Anyways, to fix this, I decided to make my own class that inherits from TObject, uses the ClassDef macro, and also inherits from TArrayD to store the values. I then overload the Hash() method to make something unique-but-only-based-on-the-contents, so that two different MyKey objects with the same values will have the same Hashes. The code is attached once again, but now it only works in compiled mode because of the class.

As you can see from the output, the Hash values for the keys at writing and reading are identical, but I still don’t get a valid value from the TMap. Am I not using TMap::GetValue properly?

Jean-François
maptest.C (2 KB)

I’ve been able to get a TMap working now with a container of multiple numerical values. A problem is that I don’t know of any suitable container of numerical values that inherits from TObject that compares (using .IsEqual) by comparing the contents. A TVectorD compares .IsEqual using the address in memory, so two vectors with the same contents can’t be used to store & retrieve a value from a TMap.

My solution was to create my own class MyKey that inherits from TObject, uses ClassDef, and also inherits from TArrayD in order to store values. I overloaded Hash() so that MyKeys with identical contents have identical hashes and IsEqual() so that they compare the hashes. Here is the class:

  class MyKey : public TObject, public TArrayD
  {
    ClassDef(MyKey, 1);
  public:
    MyKey(TArrayD arr) : TArrayD(arr) {};
    MyKey(void) : TArrayD(0) {};
    MyKey(Double_t chan, Double_t smooth, Double_t thresh)
    {
      fArray = 0;
      const Double_t arr[] = {chan,smooth,thresh};
      Set(3,arr);
    };
    virtual ~MyKey() {};
    ULong_t Hash() const
    {
      TString values;
      for(int i = 0;i < this->GetSize();i++)
	{
	  values.Append(TString::Format("%e,",this->operator[](i)));
	}
      return values.Hash();
    };
    Bool_t IsEqual(const TObject* obj) const
    {
      return this->Hash() == obj->Hash();
    }
  };

Hopefully I’ve done this right, and I still think that I have overlooked some existing simple solution. Is it really the case that are are no containers for numerical values that inherit from TObject that can be used as keys in TMap such that only their contents count for retrieval?

Jean-François

Hi Jean-François,

I think you indeed did the right thing. There is indeed no such container currently in libCore.

Cheers,
Philippe.