How to stream template classes based on Float16_t correctly?

Triple_S · February 24, 2020, 6:32pm

Hi,

I was recently trying to reduce the sizes of my ROOT files. Since the TLorentzVector class uses Doubles, it eats up lots of memory without actually needing it. To go for smaller classes, I looked at the GenVector package that allows to set the type of the fields by a template parameter. However, I noticed something I don’t really understand when I wrote these classes in trees:

This code reproduces the problem:

#include "TObject.h"
#include "TFile.h"
#include "TTree.h"
#include "TClonesArray.h"

#include "Math/LorentzVector.h"

typedef ROOT::Math::LorentzVector<ROOT::Math::PxPyPzE4D<Float16_t> > LorentzVectorF16;

void test() {
    TFile* f = new TFile("test.root", "RECREATE");
    TTree* t = new TTree("t", "Test Tree");

    LorentzVectorF16* lv  = new LorentzVectorF16();
    t->Branch("lv", lv, 32000, 3);
    t->Fill();
    t->Print();
}

The output I get is:

******************************************************************************
*Tree    :t         : Test Tree                                              *
*Entries :        1 : Total =            4631 bytes  File  Size =          0 *
*        :          : Tree compression factor =   1.00                       *
******************************************************************************
*Branch  :lv                                                                 *
*Entries :        1 : BranchElement (see below)                              *
*............................................................................*
*Br    0 :fCoordinates :                                                     *
*Entries :        1 : Total  Size=       3608 bytes  One basket in memory    *
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     *
*............................................................................*
*Br    1 :fCoordinates.fX : Float_t                                          *
*Entries :        1 : Total  Size=        707 bytes  One basket in memory    *
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     *
*............................................................................*
*Br    2 :fCoordinates.fY : Float_t                                          *
*Entries :        1 : Total  Size=        707 bytes  One basket in memory    *
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     *
*............................................................................*
*Br    3 :fCoordinates.fZ : Float_t                                          *
*Entries :        1 : Total  Size=        707 bytes  One basket in memory    *
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     *
*............................................................................*
*Br    4 :fCoordinates.fT : Float_t                                          *
*Entries :        1 : Total  Size=        707 bytes  One basket in memory    *
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     *
*............................................................................*

As you can see the internal fields fX, fY , fZ and fT that should be Float16_t as declared in the template, are streamed just like regular Float_t and I would like to understand why this happens and whether there is a way to stream them correctly.

Thanks in advance and all the best,
Triple_S

ROOT Version: 6.12.06
Platform: Debian 8
Compiler: GCC 4.9.2

pcanal · February 25, 2020, 9:46pm

Because of the way the C++ type system works and choice we made, TTree can not auto-detect that the type using Float16_t, you need to use:

t->Branch("lv", "ROOT::Math::LorentzVector<ROOT::Math::PxPyPzE4D<Float16_t> >", lv, 32000, 3);

You also need to generate the dictionary for both ROOT::Math::PxPyPzE4D<Float16_t> and ROOT::Math::LorentzVector<ROOT::Math::PxPyPzE4D<Float16_t> >.

Cheers,
Philippe.

Triple_S · February 27, 2020, 6:40am

Thanks for your answer.

Unfortunatly none of these methods work as expected. I generated the required dictonaries in a shared library which is always loaded during the start of ROOT. Furthermore trying to generate the dictonary via gInterpreter like this:

gInterpreter->GenerateDictionary("ROOT::Math::PxPyPzE4D<Float16_t>", "tools.h");
gInterpreter->GenerateDictionary("ROOT::Math::LorentzVector<ROOT::Math::PxPyPzE4D<Float16_t> >", "tools.h");

returns:

Warning in <TClassTable::Add>: class ROOT::Math::PxPyPzE4D<Float16_t> already in TClassTable
Warning in <TClassTable::Add>: class ROOT::Math::LorentzVector<ROOT::Math::PxPyPzE4D<Float16_t> > already in TClassTable

so I think the dictonaries are not the problem.

However, even with your approach the datamembers are still streamed as regular Float_t.

There is another thing I noted: When I run the code on the command line of ROOT, I get the desired output. The problem I described only appears when I compile the macro.
I thought it might be due to our outdated ROOT version so I tried it on my private machine which is running the most recent release 6.18.04, but the problem also appears there.

Thanks for your help,
Triple_S

pcanal · February 27, 2020, 4:01pm

That is intriguing. Do you mean that you get the desired output with or without specifying the name of the class explicitly in the call to TTree::Branch?

However, even with your approach the datamembers are still streamed as regular Float_t.

This might indicate that the dictionary for the Float version and for the Float16 version have been generated in the same dictionary.

So it seems that the next steps are to understand the different between your compiled case and the interpreted case and to figure out which dictionary is being used for the Float16 version in each case.

Triple_S · February 27, 2020, 5:56pm

That is intriguing. Do you mean that you get the desired output with or without specifying the name of the class explicitly in the call to TTree::Branch?

Yes, when I run exactly the code I posted in the beginning, I get:

******************************************************************************
*Tree    :t         : Test Tree                                              *
*Entries :        1 : Total =            4671 bytes  File  Size =          0 *
*        :          : Tree compression factor =   1.00                       *
******************************************************************************
*Branch  :lv                                                                 *
*Entries :        1 : BranchElement (see below)                              *
*............................................................................*
*Br    0 :fCoordinates :                                                     *
*Entries :        1 : Total  Size=       3636 bytes  One basket in memory    *
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     *
*............................................................................*
*Br    1 :fCoordinates.fX : Float16_t                                        *
*Entries :        1 : Total  Size=        706 bytes  One basket in memory    *
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     *
*............................................................................*
*Br    2 :fCoordinates.fY : Float16_t                                        *
*Entries :        1 : Total  Size=        706 bytes  One basket in memory    *
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     *
*............................................................................*
*Br    3 :fCoordinates.fZ : Float16_t                                        *
*Entries :        1 : Total  Size=        706 bytes  One basket in memory    *
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     *
*............................................................................*
*Br    4 :fCoordinates.fT : Float16_t                                        *
*Entries :        1 : Total  Size=        706 bytes  One basket in memory    *
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     *
*............................................................................*

which is exactly what I had expected in the first place. It also does not matter whether I open ROOT and copy paste the code lines there or whether I run the macro without compiling it root test.cc. I only get the wrong behavior when I run root test.cc++.

This might indicate that the dictionary for the Float version and for the Float16 version have been generated in the same dictionary.

Actually that would surprise me, since the dictionary for the regular Float_t version is already being created by ROOT, since it is being linked in the LinkDef_GenVector.h of the Math package. root/math/genvector/inc/Math/LinkDef_GenVector.h at master · root-project/root · GitHub

I think it might have something to do with the #pragma read commands there, especially lines 608/609.

#pragma read sourceClass="ROOT::Math::LorentzVector<ROOT::Math::PxPyPzE4D<Float16_t> >"  \
             targetClass="ROOT::Math::LorentzVector<ROOT::Math::PxPyPzE4D<float> >";

However, I don’t know what these statements do exactly.

I think it could also be that ROOT is confused by the link command itself from line 603, since it uses “float” instead of “Float_t” as the template type and Float16_t is a typedef for float.

#pragma link C++ class    ROOT::Math::LorentzVector<ROOT::Math::PxPyPzE4D<float> >+;

Thanks for your help and all the best,
Triple_S

pcanal · February 27, 2020, 6:28pm

For the original code to work, we must be in a situation where only the Float16 dictionary is loaded. This is because the TTree::Branch version where the type is not specified use the C++ keyword typeid to detect the type. typeid returns a type_info and because both Float_t and Float16_t are a typedef to float they have the exact same type_info … as far as the real C++ compiler is concerned.

However, I don’t know what these statements do exactly.

They register with ROOT I/O that the 2 classes are ‘equivalent’ and the I/O should allow reading one onfile version into the other in-memory version.

Actually that would surprise me, since the dictionary for the regular Float_t version is already being created by ROOT, since it is being linked in the LinkDef_GenVector.h of the Math package.

Yes but it does not contain the Float16_t version (as far as I can tell) and thus:

Warning in <TClassTable::Add>: class ROOT::Math::PxPyPzE4D<Float16_t> already in TClassTable

Indicates that something else in your environment does and may or may not have the conflict.

Cheers,
Philippe.

Triple_S · February 27, 2020, 6:56pm

Indicates that something else in your environment does and may or may not have the conflict.

Yes, as I already said, I have a shared library that generates the dictionary for the Float16_t version. I needed this, because the shared library also contains a class inheriting from the Lorentz Vector. This class by the way is always being streamed correctly.

I just tested what happens when I don’t load the shared library and indeed: Without the shared library loaded, the problem also appears in the interpreter, but the question why it does not work in the compile case with the shared library loaded remains.

Thanks for your help and all the best,
Triple_S

pcanal · February 27, 2020, 7:36pm

How do you load the library in the compiled case?

Triple_S · February 27, 2020, 7:47pm

The Library is loaded by the rootlogon macro via gSystem->Load()

pcanal · February 27, 2020, 8:02pm

Depending how you are setting up your main function, this macro may or may not be executed in the compiled case.

Triple_S · February 27, 2020, 8:16pm

Oh, I was’nt aware of this since I the rootlogon macro has some console output which I also get when I compile the macro. How can I test / make sure the macro is taken into account in the compiled case?

pcanal · February 27, 2020, 8:33pm

since I the rootlogon macro has some console output which I also get when I compile the macro.

That would mean that it is actually executed …

Note that you can tell that the dictionary is loaded properly by looking at the result of:

TClass::GetClass("ROOT::Math::PxPyPzE4D<Float16_t>")->GetStreamerInfo()->ls();

Note from every that we discuss so far, I believe that the result of
the original code would depend of the order in which the libraries are loaded (the one with Float_t and the one with Float16) … and I can think of any reason why the case with the classname explicitly introduced would not work (unless I got the order of parameter wrong).

system · March 12, 2020, 8:48pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.