Problems with std::vector<std::array> branches

Hi,

We are having issues scanning and drawing branches of std::vector<std::vector<std::array<float, 2>>>.

In trying to create a simpler minimal working example of the problem, I see this error:

Error in <TBranchElement::InitializeOffsets>: Could not find the real data member '_M_elems[2]' when constructing the branch 'vector_array' [Likely missing ShowMember].

with the following short macro:

void mwe_no_dict() {
  gInterpreter->GenerateDictionary("vector<array<float,2>>", "array;vector");

  std::vector<std::array<float,2>> vector_array;

  TTree* tree = new TTree();
  tree->Branch("vector_array", &vector_array);
}

Does anyone know what I am doing wrong here?

Thanks,
Andy


ROOT Version: 6.30/04
Platform: linuxx8664gcc
Compiler: g++ (Spack GCC) 13.1.0


Hi Andy,

Thanks for the interesting post and welcome to the ROOT Community!

Sorry to read it does not work out of the box for you. Could you share with us the file, or a few events, so that we can reproduce?

Cheers,
D

Hi Danilo,

Thanks for the reply. I’ve attached a small file that we are having trouble with. After writing out this message, I found that I can’t post links as a new user so let me know if there’s another way for me to get this file to you.

As a bit of explanation, we should have (and do) have twice as many etimes as hits. The etimes are in pairs corresponding to two ends of a straw. So the first two lines (68217.414 and 68215.187) are from the same hit on a straw but at different ends. These pairs are represented by the inner-most std::array<float, 2>.

trkana->Scan("kl.nhits:kltsh.etime", "kl.nhits==12")
***********************************************
*    Row   * Instance *  kl.nhits * kltsh.eti *
***********************************************
*        7 *        0 *        12 * 68217.414 *
*        7 *        1 *        12 * 68215.187 *
*        7 *        2 *        12 * 68225.953 *
*        7 *        3 *        12 * 68224.867 *
*        7 *        4 *        12 *  68236.25 *
*        7 *        5 *        12 * 68235.132 *
*        7 *        6 *        12 * 68209.578 *
*        7 *        7 *        12 * 68207.632 *
*        7 *        8 *        12 * 68235.179 *
*        7 *        9 *        12 * 68232.054 *
*        7 *       10 *        12 * 68230.843 *
*        7 *       11 *        12 * 68228.171 *
*        7 *       12 *        12 * 68205.796 *
*        7 *       13 *        12 * 68206.093 *
*        7 *       14 *        12 * 68221.406 *
*        7 *       15 *        12 * 68222.757 *
*        7 *       16 *        12 * 68224.203 *
*        7 *       17 *        12 * 68226.234 *
*        7 *       18 *        12 * 68217.671 *
*        7 *       19 *        12 * 68221.585 *
*        7 *       20 *        12 * 68224.203 *
*        7 *       21 *        12 * 68226.234 *
*        7 *       22 *        12 * 68217.671 *
*        7 *       23 *        12 * 68221.585 *
***********************************************

However, when we try to Scan each end separately, we get the same results (i.e. in the first row we get 68217.414 twice when we expect 68215.187 in the final column)

 trkana->Scan("kl.nhits:kltsh.etime[][][0]:kltsh.etime[][][1]", "kl.nhits==12")
***********************************************************
*    Row   * Instance *  kl.nhits * kltsh.eti * kltsh.eti *
***********************************************************
*        7 *        0 *        12 * 68217.414 * 68217.414 *
*        7 *        1 *        12 * 68225.953 * 68225.953 *
*        7 *        2 *        12 *  68236.25 *  68236.25 *
*        7 *        3 *        12 * 68209.578 * 68209.578 *
*        7 *        4 *        12 * 68235.179 * 68235.179 *
*        7 *        5 *        12 * 68230.843 * 68230.843 *
*        7 *        6 *        12 * 68205.796 * 68205.796 *
*        7 *        7 *        12 * 68221.406 * 68221.406 *
*        7 *        8 *        12 * 68224.203 * 68224.203 *
*        7 *        9 *        12 * 68217.671 * 68217.671 *
*        7 *       10 *        12 * 68224.203 * 68224.203 *
*        7 *       11 *        12 * 68217.671 * 68217.671 *
***********************************************************

There are a couple more details on our experiment’s GitHub page (looks like I can’t post a direct link as a new user but it’s Mu2e/TrkAna issue #168). For example, we don’t have a problem when reading it in via python+uproot.

Anyway, if it would be easier that I break the problem down into a non-experiment specific example, then I’m happy to try. I would just have to get past the error in my iniital post.

Thanks,
Andy

Hi Danilo,

Looks like I still can’t post a link to my small file that contains the issue. Is there another way I can provide it?

Thanks,
Andy

1 Like

Hi Danilo,

It looks like I can now post files. Here is a file with the issue: small-file.root (113.7 KB)

To briefly recap the problem: the two etimes in trkana->Scan("kl.nhits:kltsh.etime[][][0]:kltsh.etime[][][1]", "kl.nhits==12") are identical:

**********************************************************
*    Row   * Instance *  kl.nhits * kltsh.eti * kltsh.eti *
***********************************************************
*        7 *        0 *        12 * 68217.414 * 68217.414 *
*        7 *        1 *        12 * 68225.953 * 68225.953 *
...

even though the second elements can be seen with trkana->Scan("kl.nhits:kltsh.etime", "kl.nhits==12"):

***********************************************
*    Row   * Instance *  kl.nhits * kltsh.eti *
***********************************************
*        7 *        0 *        12 * 68217.414 *
*        7 *        1 *        12 * 68215.187 *
*        7 *        2 *        12 * 68225.953 *
*        7 *        3 *        12 * 68224.867 *
...

Thanks,
Andy

@pcanal Maybe someone could have a look at the problem reported in the first post here (the newest ROOT 6.32.02 is also affected),
Moreover, this line breaks ACLiC (it really dies hard, not just issues some harmless warning):
gInterpreter->GenerateDictionary("array<float,2>", "array");

1 Like

std::array does not need a dictionary.

However, std::vector of an std::array is not yet supported. You can however use the following pattern (the class ArrayWrapper and std::vector<ArrayWrapper> need a dictionary):

struct ArrayWrapper {
   std::array<float,2> _values;
};
....
  std::vector<ArrayWrapper> vector_array;

  TTree* tree = new TTree("tree", "");
  tree->Branch("vector_array", &vector_array);

I’m afraid I’m still having trouble. If I do the following on the command line:

struct ArrayWrapper { std::array<float,2> _values; };
gInterpreter->GenerateDictionary("ArrayWrapper");
gInterpreter->GenerateDictionary("vector<ArrayWrapper>", "vector");
std::vector<ArrayWrapper> vector_array;
TTree* tree = new TTree("tree", "");
tree->Branch("vector_array", &vector_array);

I get the following error:

Error in <TTree::Branch>: The class requested (vector<ArrayWrapper>) for the branch "vector_array" is an instance of an stl collection and does not have a compiled CollectionProxy. Please generate the dictionary for this collection (vector<ArrayWrapper>) to avoid to write corrupted data.

Is there a way to tell whether the dictionary was generated correctly?

You will need to have a separate header file with the declaration of ArrayWrapper (and pass its name to both invocation of GenerateDictionary).

Thanks but I still can’t get it to work…

In ArrayWrapper.h:

#include <array>

struct ArrayWrapper {
  ArrayWrapper() { };
   std::array<float,2> _values;
};

and then on the command line:

.L ArrayWrapper.h
gInterpreter->GenerateDictionary("ArrayWrapper", "ArrayWrapper.h");
gInterpreter->GenerateDictionary("vector<ArrayWrapper>", "ArrayWrapper.h;vector");
std::vector<ArrayWrapper> vector_array;
TTree* tree = new TTree("tree", "");
tree->Branch("vector_array", &vector_array);

and I get the same error:

Error in <TTree::Branch>: The class requested (vector<ArrayWrapper>) for the branch "vector_array" is an instance of an stl collection and does not have a compiled CollectionProxy. Please generate the dictionary for this collection (vector<ArrayWrapper>) to avoid to write corrupted data.

humm … something is odd in your setup … with your exact file I get:

root [0] .L ArrayWrapper.h 
root [1] gInterpreter->GenerateDictionary("ArrayWrapper", "ArrayWrapper.h");
In file included from AutoDict_ArrayWrapper_cxx_ACLiC_dict dictionary payload:8:
In file included from ./AutoDict_ArrayWrapper.cxx:1:
/private/var/tmp/arr/ArrayWrapper.h:5:8: error: redefinition of 'ArrayWrapper'
struct ArrayWrapper {
       ^
input_line_8:1:10: note: './ArrayWrapper.h' included multiple times, additional include site here
...

Using:

#pragma once

#include <array>

struct ArrayWrapper {
  ArrayWrapper() { };
   std::array<float,2> _values;
};

it works for me (do not execute .L ArrayWrapper.h)

$ root.exe -b -l
root [0] gInterpreter->GenerateDictionary("ArrayWrapper", "ArrayWrapper.h");
root [1] gInterpreter->GenerateDictionary("vector<ArrayWrapper>", "ArrayWrapper.h;vector");
root [2] std::vector<ArrayWrapper> vector_array;
root [3] TTree* tree = new TTree("tree", "");
root [4] tree->Branch("vector_array", &vector_array);
root [5] .q

Thanks - it looks like there is something wrong with the setup. Your instructions work on a clean install on my laptop but not on the common machines we use for Mu2e…

Thanks again for your help :slight_smile:

So I’ve solved the problem with my area (some previous attempts were polluting my cwd) and can now show the initial issue I’m trying to solve.

With the attached ArrayWrapper.h and mwe.C I get the following output:

Entry #1:
        vec_arr = [ (1,2) ]
        vec_vec_arr = { [ (1,2) ], [ (3,4) ] }
Entry #2:
        vec_arr = [ (5,7) ]
        vec_vec_arr = { [ (20,25) ], [ (30,35) ] }

TTree::Scan of the vec_arr branch works as expected...
  printing all values:
******************************************
*    Row   * Instance *  vec_arr._values *
******************************************
*        0 *        0 *                1 *
*        0 *        1 *                2 *
*        1 *        0 *                5 *
*        1 *        1 *                7 *
******************************************

  printing just first and second value of pair:
*******************************************************************************
*    Row   * Instance *      vec_arr._values[][0] *      vec_arr._values[][1] *
*******************************************************************************
*        0 *        0 *                         1 *                         2 *
*        1 *        0 *                         5 *                         7 *
*******************************************************************************

  printing the difference between the first and second value of pair:
******************************************************************
*    Row   * Instance * vec_arr._values[][0]-vec_arr._values[][1 *
******************************************************************
*        0 *        0 *                                       -1 *
*        1 *        0 *                                       -2 *
******************************************************************


TTree:Scan of the vec_vec_arr branch *does* works as expected for...
  printing all values:
********************************************************
*    Row   * Instance *          vec_vec_arr._values[] *
********************************************************
*        0 *        0 *                              1 *
*        0 *        1 *                              2 *
*        0 *        2 *                              3 *
*        0 *        3 *                              4 *
*        1 *        0 *                             20 *
*        1 *        1 *                             25 *
*        1 *        2 *                             30 *
*        1 *        3 *                             35 *
********************************************************

  and printing each array separately:
*****************************************************************************************
*    Row   * Instance *         vec_vec_arr._values[0] *         vec_vec_arr._values[1] *
*****************************************************************************************
*        0 *        0 *                              1 *                              3 *
*        0 *        1 *                              2 *                              4 *
*        1 *        0 *                             20 *                             30 *
*        1 *        1 *                             25 *                             35 *
*****************************************************************************************


But does *not* work as expected for...

  printing the first and second elements of each entry (trying a few different ways):
*****************************************************************************************
*    Row   * Instance *       vec_vec_arr._values[][0] *       vec_vec_arr._values[][1] *
*****************************************************************************************
*        0 *        0 *                              1 *                                *
*        0 *        1 *                              1 *                                *
*        0 *        2 *                              3 *                                *
*        0 *        3 *                              3 *                                *
*        1 *        0 *                             20 *                                *
*        1 *        1 *                             20 *                                *
*        1 *        2 *                             30 *                                *
*        1 *        3 *                             30 *                                *
*****************************************************************************************
*****************************************************************************************
*    Row   * Instance *     vec_vec_arr._values[][][0] *     vec_vec_arr._values[][][1] *
*****************************************************************************************
*        0 *        0 *                              1 *                              1 *
*        0 *        1 *                              3 *                              3 *
*        1 *        0 *                             20 *                             20 *
*        1 *        1 *                             30 *                             30 *
*****************************************************************************************
*****************************************************************************************
*    Row   * Instance *     vec_vec_arr[]._values[][0] *     vec_vec_arr[]._values[][1] *
*****************************************************************************************
*        0 *        0 *                              1 *                              1 *
*        0 *        1 *                              3 *                              3 *
*        1 *        0 *                             20 *                             20 *
*        1 *        1 *                             30 *                             30 *
*****************************************************************************************

  or taking the difference between each pair (trying a few different ways):
**************************************************************************************
*    Row   * Instance *            vec_vec_arr._values[][0]-vec_vec_arr._values[][1] *
**************************************************************************************
*        0 *        0 *                                                              *
*        1 *        0 *                                                              *
**************************************************************************************
**************************************************************************************
*    Row   * Instance *        vec_vec_arr._values[][][0]-vec_vec_arr._values[][][1] *
**************************************************************************************
*        0 *        0 *                                                            0 *
*        0 *        1 *                                                            0 *
*        1 *        0 *                                                            0 *
*        1 *        1 *                                                            0 *
**************************************************************************************
**************************************************************************************
*    Row   * Instance *        vec_vec_arr[]._values[][0]-vec_vec_arr[]._values[][1] *
**************************************************************************************
*        0 *        0 *                                                            0 *
*        0 *        1 *                                                            0 *
*        1 *        0 *                                                            0 *
*        1 *        1 *                                                            0 *
**************************************************************************************

I know the array looping is quite complex so am I getting the syntax wrong here, or is it not possible to get what we want with vector<vector<array>>?

Thanks,
Andy

ArrayWrapper.h (111 Bytes)
mwe.C (4.9 KB)

Hi Everyone,

Since vector<array> branches are not directly supported and the ArrayWrapper solution doesn’t do what we want, we’re looking into using a vector<vector<vector>> branch for our three-level branch.

Unfortunately, this seems to be working even less well…

The attached vec_vec_vec.C script produces the following output:

$ root -l vec_vec_vec.C
root [0]
Processing vec_vec_vec.C...
Entry #1:
        vec_vec = [ (1,2) ]
        vec_vec_vec = { [ (1,2) ], [ (3,4) ] }
Entry #2:
        vec_vec = [ (5,7) ]
        vec_vec_vec = { [ (20,25) ], [ (30,35) ] }

TTree::Scan of the vec_vec branch works as expected...
  printing all values:
******************************************
*    Row   * Instance *          vec_vec *
******************************************
*        0 *        0 *                1 *
*        0 *        1 *                2 *
*        1 *        0 *                5 *
*        1 *        1 *                7 *
******************************************

  printing just first and second value of pair:
*******************************************************************************
*    Row   * Instance *              vec_vec[][0] *              vec_vec[][1] *
*******************************************************************************
*        0 *        0 *                         1 *                         2 *
*        1 *        0 *                         5 *                         7 *
*******************************************************************************

  printing the difference between the first and second value of pair:
******************************************************************
*    Row   * Instance *                vec_vec[][0]-vec_vec[][1] *
******************************************************************
*        0 *        0 *                                       -1 *
*        1 *        0 *                                       -2 *
******************************************************************


TTree:Scan of the vec_vec_vec branch *doesn't* work as expected for...
  printing all values:
********************************************************
*    Row   * Instance *                    vec_vec_vec *
********************************************************
*        0 *        0 *                              0 *
*        0 *        1 *                              0 *
*        1 *        0 *                              0 *
*        1 *        1 *                              0 *
********************************************************

Are we doing something wrong?

Thanks,
Andy

vec_vec_vec.C (4.6 KB)

This appears too complex for TTreeFormula, I recommend you try looking at this data with RDataFrame

Thanks, @pcanal. We looked into RDataFrame and it doesn’t work for our branch structure either. We can still analyze with compiled macros and loops but for quick data exploriation, the ROOT command line was very useful. We’ll keep an eye on developments in case RDataFrame or TTreeFormula can do what we want in the future.

Thanks for all your help on this,
Andy

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.