CheckByteCount and Streamer Error

ROOT
21

The second one seems ok

jgalan@sultan2:~/rest-framework/projects/iaxo/iaxo-simulations/simulations/signal$ restRoot
Warning in <UnknownClass::SetDisplay>: DISPLAY not set, setting it to 192.168.1.2:0.0
= Loading libraries ...
 - /home/jgalan/rest-framework/install/lib/libRestFramework.so
 - /home/jgalan/rest-framework/install/lib/libRestAxion.so
Warning in <TClassTable::Add>: class TRestMetadata already in TClassTable
Warning in <TClassTable::Add>: class TRestEventProcess already in TClassTable

Error in <TGClient::TGClient>: only one instance of TGClient allowed
root [0] TRestAxionGeneratorProcess *axGen = new TRestAxionGeneratorProcess();
root [1] TFile *f = TFile::Open("manual_test.root", "RECREATE");
root [2] axGen->Write();
root [3] delete f;
root [4] delete axGen;
root [5] .q
jgalan@sultan2:~/rest-framework/projects/iaxo/iaxo-simulations/simulations/signal$ restRoot
Warning in <UnknownClass::SetDisplay>: DISPLAY not set, setting it to 192.168.1.2:0.0
= Loading libraries ...
 - /home/jgalan/rest-framework/install/lib/libRestFramework.so
 - /home/jgalan/rest-framework/install/lib/libRestAxion.so
Warning in <TClassTable::Add>: class TRestMetadata already in TClassTable
Warning in <TClassTable::Add>: class TRestEventProcess already in TClassTable

Error in <TGClient::TGClient>: only one instance of TGClient allowed
root [0] TFile *f = TFile::Open("manual_test.root");
root [1] f->ShowStreamerInfo()
OBJ: TList	TList	Doubly linked list : 0

StreamerInfo for class: TNamed, version=1, checksum=0xdfb74a3c
  TObject        BASE            offset=  0 type=66 Basic ROOT object   
  TString        fName           offset=  0 type=65 object identifier   
  TString        fTitle          offset=  0 type=65 object title        

StreamerInfo for class: TRestAxionGeneratorProcess, version=3, checksum=0xa7ff3818
  TRestEventProcess BASE            offset=  0 type= 0                     
  TVector2       fAxionMassRange offset=  0 type=61 <                   
  double         fTargetRadius   offset=  0 type= 8 <                   
  TString        fGeneratorType  offset=  0 type=65 <                   
  int            fSeed           offset=  0 type= 3 <                   
  TVector2       fEnergyRange    offset=  0 type=61 <                   
  double         fTotalFlux      offset=  0 type= 8 <                   

StreamerInfo for class: TRestEventProcess, version=3, checksum=0x2513b303
  TRestMetadata  BASE            offset=  0 type= 0                     

StreamerInfo for class: TRestMetadata, version=9, checksum=0xe298dc45
  TNamed         BASE            offset=  0 type=67 The basis for a named object (name, title)
  TString        fVersion        offset=  0 type=65 <                   
  TString        fCommit         offset=  0 type=65 <                   
  TString        fLibraryVersion offset=  0 type=65 <                   
  bool           fOfficialRelease offset=  0 type=18 <                   
  bool           fCleanState     offset=  0 type=18 <                   
  string         fConfigFileName offset=  0 type=300 ,stl=365, ctype=365,                     
  string         fSectionName    offset=  0 type=300 ,stl=365, ctype=365,                     
  string         configBuffer    offset=  0 type=300 ,stl=365, ctype=365,                     
  string         messageBuffer   offset=  0 type=300 ,stl=365, ctype=365,                     
  bool           fError          offset=  0 type=18                     
  int            fNErrors        offset=  0 type= 3                     
  bool           fWarning        offset=  0 type=18                     
  int            fNWarnings      offset=  0 type= 3                     
  TString        fErrorMessage   offset=  0 type=65                     
  TString        fWarningMessage offset=  0 type=65                     

StreamerInfo for class: TVector2, version=3, checksum=0x89b7f4
  TObject        BASE            offset=  0 type=66 Basic ROOT object   
  double         fX              offset=  0 type= 8 components of the vector
  double         fY              offset=  0 type= 8                     
root [2]
22

The first also looks the same, both contain TRestAxionGeneratorProcess

23

So, when you write a single object, ROOT behaves fine.
But, it seems that when there are multiple different objects written, the logic that stores streamer info misbehaves.

Maybe another test … try to repeat the last tests but use the original “linkdef” file (without manual “enhancements”).

24

But this only happens with classes that inherit from TRestEventProcess. Other classes that inherit directly from TRestMetadata are properly written. Perhaps, the reason why they behave different might be because we write those objects to disk from different classes. We should investigate a bit more on our side.

I was thinking, perhaps the fact that we open another ROOT file (just for reading) in between different writes might affect? Something such we should be doing fOutputFile->Cd(); just before writing?

25

I usually open an output file, write all objects and then delete the output file (like in my examples above).
If you first open an output file, then open other files, you need to use the following:

output_file->cd(); // note: "cd", NOT "Cd"
object->Write();
26

I would suspect from that, but the fact is that objects are after-all being written to the file, even without ->cd(). I will give a try anyway.

27

There is no way to manually force a class dictionary to be on the streamer?

28

It seems to me that you’ve hit something that really needs a close inspection.

If you use the original unmodified “linkdef” files, do the simple tests (writing a single TRestAxionGeneratorProcess object) work o.k.?

29

It seems the objects appear when invoking ShowStreamerInfo().

First test:

jgalan@sultan2:~/rest-framework$ restRoot
Warning in <UnknownClass::SetDisplay>: DISPLAY not set, setting it to 192.168.1.2:0.0
= Loading libraries ...
 - /home/jgalan/rest-framework/install/lib/libRestDetector.so
 - /home/jgalan/rest-framework/install/lib/libRestFramework.so
 - /home/jgalan/rest-framework/install/lib/libRestRaw.so
 - /home/jgalan/rest-framework/install/lib/libRestConnectors.so
 - /home/jgalan/rest-framework/install/lib/libRestAxion.so

Error in <TGClient::TGClient>: only one instance of TGClient allowed
root [0] TRestAxionGeneratorProcess axGen;
root [1] TFile *f = TFile::Open("manual_test.root", "RECREATE");
root [2] axGen.Write();
root [3] delete f;
root [4] .q
jgalan@sultan2:~/rest-framework$ restRoot
Warning in <UnknownClass::SetDisplay>: DISPLAY not set, setting it to 192.168.1.2:0.0
= Loading libraries ...
 - /home/jgalan/rest-framework/install/lib/libRestDetector.so
 - /home/jgalan/rest-framework/install/lib/libRestFramework.so
 - /home/jgalan/rest-framework/install/lib/libRestRaw.so
 - /home/jgalan/rest-framework/install/lib/libRestConnectors.so
 - /home/jgalan/rest-framework/install/lib/libRestAxion.so

Error in <TGClient::TGClient>: only one instance of TGClient allowed
root [0] TFile *f = TFile::Open("manual_test.root");
root [1] f->ShowStreamerInfo()
OBJ: TList	TList	Doubly linked list : 0

StreamerInfo for class: TNamed, version=1, checksum=0xdfb74a3c
  TObject        BASE            offset=  0 type=66 Basic ROOT object   
  TString        fName           offset=  0 type=65 object identifier   
  TString        fTitle          offset=  0 type=65 object title        

StreamerInfo for class: TRestAxionGeneratorProcess, version=3, checksum=0xa7ff3818
  TRestEventProcess BASE            offset=  0 type= 0                     
  TVector2       fAxionMassRange offset=  0 type=61 <                   
  double         fTargetRadius   offset=  0 type= 8 <                   
  TString        fGeneratorType  offset=  0 type=65 <                   
  int            fSeed           offset=  0 type= 3 <                   
  TVector2       fEnergyRange    offset=  0 type=61 <                   
  double         fTotalFlux      offset=  0 type= 8 <                   

StreamerInfo for class: TRestEventProcess, version=3, checksum=0x2513b303
  TRestMetadata  BASE            offset=  0 type= 0                     

StreamerInfo for class: TRestMetadata, version=9, checksum=0xe298dc45
  TNamed         BASE            offset=  0 type=67 The basis for a named object (name, title)
  TString        fVersion        offset=  0 type=65 <                   
  TString        fCommit         offset=  0 type=65 <                   
  TString        fLibraryVersion offset=  0 type=65 <                   
  bool           fOfficialRelease offset=  0 type=18 <                   
  bool           fCleanState     offset=  0 type=18 <                   
  string         fConfigFileName offset=  0 type=300 ,stl=365, ctype=365,                     
  string         fSectionName    offset=  0 type=300 ,stl=365, ctype=365,                     
  string         configBuffer    offset=  0 type=300 ,stl=365, ctype=365,                     
  string         messageBuffer   offset=  0 type=300 ,stl=365, ctype=365,                     
  bool           fError          offset=  0 type=18                     
  int            fNErrors        offset=  0 type= 3                     
  bool           fWarning        offset=  0 type=18                     
  int            fNWarnings      offset=  0 type= 3                     
  TString        fErrorMessage   offset=  0 type=65                     
  TString        fWarningMessage offset=  0 type=65                     

StreamerInfo for class: TVector2, version=3, checksum=0x89b7f4
  TObject        BASE            offset=  0 type=66 Basic ROOT object   
  double         fX              offset=  0 type= 8 components of the vector
  double         fY              offset=  0 type= 8                     
root [2]

Second test:

jgalan@sultan2:~/rest-framework$ restRoot
Warning in <UnknownClass::SetDisplay>: DISPLAY not set, setting it to 192.168.1.2:0.0
= Loading libraries ...
 - /home/jgalan/rest-framework/install/lib/libRestDetector.so
 - /home/jgalan/rest-framework/install/lib/libRestFramework.so
 - /home/jgalan/rest-framework/install/lib/libRestRaw.so
 - /home/jgalan/rest-framework/install/lib/libRestConnectors.so
 - /home/jgalan/rest-framework/install/lib/libRestAxion.so

Error in <TGClient::TGClient>: only one instance of TGClient allowed
root [0] TRestAxionGeneratorProcess *axGen = new TRestAxionGeneratorProcess();
root [1] TFile *f = TFile::Open("manual_test.root", "RECREATE");
root [2] axGen->Write();
root [3] delete f;
root [4] delete axGen;
root [5] .q
jgalan@sultan2:~/rest-framework$ restRoot
Warning in <UnknownClass::SetDisplay>: DISPLAY not set, setting it to 192.168.1.2:0.0
= Loading libraries ...
 - /home/jgalan/rest-framework/install/lib/libRestDetector.so
 - /home/jgalan/rest-framework/install/lib/libRestFramework.so
 - /home/jgalan/rest-framework/install/lib/libRestRaw.so
 - /home/jgalan/rest-framework/install/lib/libRestConnectors.so
 - /home/jgalan/rest-framework/install/lib/libRestAxion.so

Error in <TGClient::TGClient>: only one instance of TGClient allowed
root [0] TFile *f = TFile::Open("manual_test.root");
root [1] f->ShowStreamerInfo()
OBJ: TList	TList	Doubly linked list : 0

StreamerInfo for class: TNamed, version=1, checksum=0xdfb74a3c
  TObject        BASE            offset=  0 type=66 Basic ROOT object   
  TString        fName           offset=  0 type=65 object identifier   
  TString        fTitle          offset=  0 type=65 object title        

StreamerInfo for class: TRestAxionGeneratorProcess, version=3, checksum=0xa7ff3818
  TRestEventProcess BASE            offset=  0 type= 0                     
  TVector2       fAxionMassRange offset=  0 type=61 <                   
  double         fTargetRadius   offset=  0 type= 8 <                   
  TString        fGeneratorType  offset=  0 type=65 <                   
  int            fSeed           offset=  0 type= 3 <                   
  TVector2       fEnergyRange    offset=  0 type=61 <                   
  double         fTotalFlux      offset=  0 type= 8 <                   

StreamerInfo for class: TRestEventProcess, version=3, checksum=0x2513b303
  TRestMetadata  BASE            offset=  0 type= 0                     

StreamerInfo for class: TRestMetadata, version=9, checksum=0xe298dc45
  TNamed         BASE            offset=  0 type=67 The basis for a named object (name, title)
  TString        fVersion        offset=  0 type=65 <                   
  TString        fCommit         offset=  0 type=65 <                   
  TString        fLibraryVersion offset=  0 type=65 <                   
  bool           fOfficialRelease offset=  0 type=18 <                   
  bool           fCleanState     offset=  0 type=18 <                   
  string         fConfigFileName offset=  0 type=300 ,stl=365, ctype=365,                     
  string         fSectionName    offset=  0 type=300 ,stl=365, ctype=365,                     
  string         configBuffer    offset=  0 type=300 ,stl=365, ctype=365,                     
  string         messageBuffer   offset=  0 type=300 ,stl=365, ctype=365,                     
  bool           fError          offset=  0 type=18                     
  int            fNErrors        offset=  0 type= 3                     
  bool           fWarning        offset=  0 type=18                     
  int            fNWarnings      offset=  0 type= 3                     
  TString        fErrorMessage   offset=  0 type=65                     
  TString        fWarningMessage offset=  0 type=65                     

StreamerInfo for class: TVector2, version=3, checksum=0x89b7f4
  TObject        BASE            offset=  0 type=66 Basic ROOT object   
  double         fX              offset=  0 type= 8 components of the vector
  double         fY              offset=  0 type= 8                     
root [2]
30

Can you provide a reproducer?

31

It would require to compile the framework (note: C++17 ROOT needed)

rm -rf rest-framework
git clone git@github.com:rest-for-physics/framework.git rest-framework
cd rest-framework
python3 pull-submodules.py --latest --onlylibs --data
cd source/libraries/axion/data
git checkout master
git pull
cd ../../../../
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_INSTALL_PREFIX=../install -DRESTLIB_AXION=ON -DRESTLIB_RAW=ON -DRESTLIB_DETECTOR=ON -DREST_MPFR=ON .. 2>&1 | tee -a cmake.out.txt
make -j install 2>&1 | tee -a make.out.txt
source ../install/thisREST.sh

The example you may use to produce the file (change the parameter eventsToProcess inside helioscope.rml in line 30 to only 10). Then:

cd rest-framework/source/libraries/axion/examples/full-ray-tracing
sed -i -e '{s/100000/10/}' helioscope.rml
restManager --c helioscope.rml --o output.root 2>&1 | tee -a restManager.out.txt
root -l -q -b output.root -e 'gFile->ls(); gFile->ShowStreamerInfo();'

You will see then how the Process objects do not appear inside ShowStreamerInfo(). To test a change on TRestAxionGeneratorProcess data members you may switch inside the submodule source/libraries/axion to the branch jgalan_axion_mass, recompile, then try to read the previous generated file.

However, if you run another example:

cd rest-framework/pipeline/trex/
wget https://sultan.unizar.es/trexdm-readouts/readouts_v2.3.root
restManager --c 01_raw.rml --f R01928_tagTest_Vm_250_Vd_160_Pr_6_Gain_0x0_Shape_0xF_Clock_0x4-068.aqs 2>&1 | tee -a restManager.out.txt
root -l -q -b RawData_01928.root -e 'gFile->ls(); gFile->ShowStreamerInfo();'

You will see how the generated output RawData_01928.root call to ShowStreamerInfo() contains all the TRestXYZProcess that were used.

I am afraid I cannot create a simplified code that reproduces the problem. I created an issue on our side:

32

The “make” reports various warnings:

(...)
[ 12%] generating: /.../rest-framework/build/rootdict/CINT_TRestPhysics.cxx with /.../rest-framework/source/framework/tools/inc/TRestPhysics.h;/.../rest-framework/build/rootdict/TRestPhysics_Linkdef.h
Warning: Unused class rule: TRestPhysics
(...)
[ 15%] generating: /.../rest-framework/build/rootdict/CINT_TRestReflector.cxx with /.../rest-framework/source/framework/tools/inc/TRestReflector.h;/.../rest-framework/build/rootdict/TRestReflector_Linkdef.h
Warning: Unused class rule: TRestReflector
(...)
[ 16%] generating: /.../rest-framework/build/rootdict/CINT_TRestStringHelper.cxx with /.../rest-framework/source/framework/tools/inc/TRestStringHelper.h;/.../rest-framework/build/rootdict/TRestStringHelper_Linkdef.h
Warning: Unused class rule: TRestStringHelper
(...)
[ 20%] generating: /.../rest-framework/build/rootdict/CINT_TRestSystemOfUnits.cxx with /.../rest-framework/source/framework/core/inc/TRestSystemOfUnits.h;/.../rest-framework/build/rootdict/TRestSystemOfUnits_Linkdef.h
Warning: Unused class rule: TRestSystemOfUnits
(...)
[ 85%] Building CXX object source/libraries/axion/CMakeFiles/RestAxion.dir/src/TRestAxionSolarFlux.cxx.o
/.../rest-framework/source/libraries/axion/src/TRestAxionSolarFlux.cxx: In member function ‘TH1F* TRestAxionSolarFlux::GetFluxHistogram(std::string, Double_t)’:
/.../rest-framework/source/libraries/axion/src/TRestAxionSolarFlux.cxx:575:23: warning: too many arguments for format [-Wformat-extra-args]
  575 |         new TH2F(Form("FluxTable", GetName()), "", 100, 0., 1., (Int_t)(20. / binSize), 0., 20.);
      |                       ^~~~~~~~~~~
(...)
[100%] Built target RestAxion
Install the project...
CMake Warning (dev) at cmake_install.cmake:299:
  Syntax Warning in cmake code at column 5

  Argument not separated from preceding token by whitespace.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at cmake_install.cmake:299:
  Syntax Warning in cmake code at column 58

  Argument not separated from preceding token by whitespace.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at cmake_install.cmake:317:
  Syntax Warning in cmake code at column 5

  Argument not separated from preceding token by whitespace.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at cmake_install.cmake:317:
  Syntax Warning in cmake code at column 58

  Argument not separated from preceding token by whitespace.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at cmake_install.cmake:335:
  Syntax Warning in cmake code at column 5

  Argument not separated from preceding token by whitespace.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at cmake_install.cmake:335:
  Syntax Warning in cmake code at column 58

  Argument not separated from preceding token by whitespace.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at cmake_install.cmake:353:
  Syntax Warning in cmake code at column 5

  Argument not separated from preceding token by whitespace.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at cmake_install.cmake:353:
  Syntax Warning in cmake code at column 58

  Argument not separated from preceding token by whitespace.
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Install configuration: ""
rm: cannot remove '/.../rest-framework/install': No such file or directory
(...)
33

Sorry, I forgot to mention you need to pull also the data, adding --data to the pull-submodules.py script will force downloading the data based repositories.

python3 pull-submodules.py --latest --data

This should download the required fluxes.rml file.

34

Ok, this is because detectorlib was not compiled. I should have provided a more complete cmake command. That example requires raw and detector libraries.

cmake -DRESTLIB_DETECTOR=ON -DRESTLIB_RAW=ON ..
35

Not sure about the origin of those warnings, I suspect that we are generating a dictionary for a class that does not inherit from TObject? Can it be the origin of those warnings?

37

The relevant repository is :

Updating submodule: source/libraries/axion/data[ OK ]

The file will be installed at ../install/data/axion/solarFlux/ after executing make install. Only then, the execution of restManager --c helioscope.rml --o output.root will be able to find the file.

38

It seems there is a problem, it is not retrieving the latest master from the data repository …

cd rest-framework/source/libraries/axion/data
git checkout master
git pull
40

Ok, just updated with all necessary commands. Should be working for monkeys and donkeys. :grinning:

41

Now, the instructions are “theorists-proof”.

@pcanal Can it be that the “threaded” output misbehaved (when saving the “streamer info”)? In the “helioscope.rml” trial’s output, I can see:

(...)
TRestProcessRunner : preparing threads...
(...)
TRestProcessRunner: 1 + 14 processes loaded, 1 threads prepared!
(...)
Configuring output file, writing metadata and tree objects
Merging thread files together
Creating file : ./output.root
(...)
42

Not sure if that is related. But the fact is that the ROOT file contains the objects stored in disk, but not the streamers.

We really worked hard to get the automatic schema evolution working properly in our project. But now I have no clue where the problem might be coming from.