CheckByteCount and Streamer Error

I would suspect from that, but the fact is that objects are after-all being written to the file, even without ->cd(). I will give a try anyway.

There is no way to manually force a class dictionary to be on the streamer?

It seems to me that you’ve hit something that really needs a close inspection.

If you use the original unmodified “linkdef” files, do the simple tests (writing a single TRestAxionGeneratorProcess object) work o.k.?

It seems the objects appear when invoking ShowStreamerInfo().

First test:

jgalan@sultan2:~/rest-framework$ restRoot
Warning in <UnknownClass::SetDisplay>: DISPLAY not set, setting it to 192.168.1.2:0.0
= Loading libraries ...
 - /home/jgalan/rest-framework/install/lib/libRestDetector.so
 - /home/jgalan/rest-framework/install/lib/libRestFramework.so
 - /home/jgalan/rest-framework/install/lib/libRestRaw.so
 - /home/jgalan/rest-framework/install/lib/libRestConnectors.so
 - /home/jgalan/rest-framework/install/lib/libRestAxion.so

Error in <TGClient::TGClient>: only one instance of TGClient allowed
root [0] TRestAxionGeneratorProcess axGen;
root [1] TFile *f = TFile::Open("manual_test.root", "RECREATE");
root [2] axGen.Write();
root [3] delete f;
root [4] .q
jgalan@sultan2:~/rest-framework$ restRoot
Warning in <UnknownClass::SetDisplay>: DISPLAY not set, setting it to 192.168.1.2:0.0
= Loading libraries ...
 - /home/jgalan/rest-framework/install/lib/libRestDetector.so
 - /home/jgalan/rest-framework/install/lib/libRestFramework.so
 - /home/jgalan/rest-framework/install/lib/libRestRaw.so
 - /home/jgalan/rest-framework/install/lib/libRestConnectors.so
 - /home/jgalan/rest-framework/install/lib/libRestAxion.so

Error in <TGClient::TGClient>: only one instance of TGClient allowed
root [0] TFile *f = TFile::Open("manual_test.root");
root [1] f->ShowStreamerInfo()
OBJ: TList	TList	Doubly linked list : 0

StreamerInfo for class: TNamed, version=1, checksum=0xdfb74a3c
  TObject        BASE            offset=  0 type=66 Basic ROOT object   
  TString        fName           offset=  0 type=65 object identifier   
  TString        fTitle          offset=  0 type=65 object title        

StreamerInfo for class: TRestAxionGeneratorProcess, version=3, checksum=0xa7ff3818
  TRestEventProcess BASE            offset=  0 type= 0                     
  TVector2       fAxionMassRange offset=  0 type=61 <                   
  double         fTargetRadius   offset=  0 type= 8 <                   
  TString        fGeneratorType  offset=  0 type=65 <                   
  int            fSeed           offset=  0 type= 3 <                   
  TVector2       fEnergyRange    offset=  0 type=61 <                   
  double         fTotalFlux      offset=  0 type= 8 <                   

StreamerInfo for class: TRestEventProcess, version=3, checksum=0x2513b303
  TRestMetadata  BASE            offset=  0 type= 0                     

StreamerInfo for class: TRestMetadata, version=9, checksum=0xe298dc45
  TNamed         BASE            offset=  0 type=67 The basis for a named object (name, title)
  TString        fVersion        offset=  0 type=65 <                   
  TString        fCommit         offset=  0 type=65 <                   
  TString        fLibraryVersion offset=  0 type=65 <                   
  bool           fOfficialRelease offset=  0 type=18 <                   
  bool           fCleanState     offset=  0 type=18 <                   
  string         fConfigFileName offset=  0 type=300 ,stl=365, ctype=365,                     
  string         fSectionName    offset=  0 type=300 ,stl=365, ctype=365,                     
  string         configBuffer    offset=  0 type=300 ,stl=365, ctype=365,                     
  string         messageBuffer   offset=  0 type=300 ,stl=365, ctype=365,                     
  bool           fError          offset=  0 type=18                     
  int            fNErrors        offset=  0 type= 3                     
  bool           fWarning        offset=  0 type=18                     
  int            fNWarnings      offset=  0 type= 3                     
  TString        fErrorMessage   offset=  0 type=65                     
  TString        fWarningMessage offset=  0 type=65                     

StreamerInfo for class: TVector2, version=3, checksum=0x89b7f4
  TObject        BASE            offset=  0 type=66 Basic ROOT object   
  double         fX              offset=  0 type= 8 components of the vector
  double         fY              offset=  0 type= 8                     
root [2] 

Second test:

jgalan@sultan2:~/rest-framework$ restRoot
Warning in <UnknownClass::SetDisplay>: DISPLAY not set, setting it to 192.168.1.2:0.0
= Loading libraries ...
 - /home/jgalan/rest-framework/install/lib/libRestDetector.so
 - /home/jgalan/rest-framework/install/lib/libRestFramework.so
 - /home/jgalan/rest-framework/install/lib/libRestRaw.so
 - /home/jgalan/rest-framework/install/lib/libRestConnectors.so
 - /home/jgalan/rest-framework/install/lib/libRestAxion.so

Error in <TGClient::TGClient>: only one instance of TGClient allowed
root [0] TRestAxionGeneratorProcess *axGen = new TRestAxionGeneratorProcess();
root [1] TFile *f = TFile::Open("manual_test.root", "RECREATE");
root [2] axGen->Write();
root [3] delete f;
root [4] delete axGen;
root [5] .q
jgalan@sultan2:~/rest-framework$ restRoot
Warning in <UnknownClass::SetDisplay>: DISPLAY not set, setting it to 192.168.1.2:0.0
= Loading libraries ...
 - /home/jgalan/rest-framework/install/lib/libRestDetector.so
 - /home/jgalan/rest-framework/install/lib/libRestFramework.so
 - /home/jgalan/rest-framework/install/lib/libRestRaw.so
 - /home/jgalan/rest-framework/install/lib/libRestConnectors.so
 - /home/jgalan/rest-framework/install/lib/libRestAxion.so

Error in <TGClient::TGClient>: only one instance of TGClient allowed
root [0] TFile *f = TFile::Open("manual_test.root");
root [1] f->ShowStreamerInfo()
OBJ: TList	TList	Doubly linked list : 0

StreamerInfo for class: TNamed, version=1, checksum=0xdfb74a3c
  TObject        BASE            offset=  0 type=66 Basic ROOT object   
  TString        fName           offset=  0 type=65 object identifier   
  TString        fTitle          offset=  0 type=65 object title        

StreamerInfo for class: TRestAxionGeneratorProcess, version=3, checksum=0xa7ff3818
  TRestEventProcess BASE            offset=  0 type= 0                     
  TVector2       fAxionMassRange offset=  0 type=61 <                   
  double         fTargetRadius   offset=  0 type= 8 <                   
  TString        fGeneratorType  offset=  0 type=65 <                   
  int            fSeed           offset=  0 type= 3 <                   
  TVector2       fEnergyRange    offset=  0 type=61 <                   
  double         fTotalFlux      offset=  0 type= 8 <                   

StreamerInfo for class: TRestEventProcess, version=3, checksum=0x2513b303
  TRestMetadata  BASE            offset=  0 type= 0                     

StreamerInfo for class: TRestMetadata, version=9, checksum=0xe298dc45
  TNamed         BASE            offset=  0 type=67 The basis for a named object (name, title)
  TString        fVersion        offset=  0 type=65 <                   
  TString        fCommit         offset=  0 type=65 <                   
  TString        fLibraryVersion offset=  0 type=65 <                   
  bool           fOfficialRelease offset=  0 type=18 <                   
  bool           fCleanState     offset=  0 type=18 <                   
  string         fConfigFileName offset=  0 type=300 ,stl=365, ctype=365,                     
  string         fSectionName    offset=  0 type=300 ,stl=365, ctype=365,                     
  string         configBuffer    offset=  0 type=300 ,stl=365, ctype=365,                     
  string         messageBuffer   offset=  0 type=300 ,stl=365, ctype=365,                     
  bool           fError          offset=  0 type=18                     
  int            fNErrors        offset=  0 type= 3                     
  bool           fWarning        offset=  0 type=18                     
  int            fNWarnings      offset=  0 type= 3                     
  TString        fErrorMessage   offset=  0 type=65                     
  TString        fWarningMessage offset=  0 type=65                     

StreamerInfo for class: TVector2, version=3, checksum=0x89b7f4
  TObject        BASE            offset=  0 type=66 Basic ROOT object   
  double         fX              offset=  0 type= 8 components of the vector
  double         fY              offset=  0 type= 8                     
root [2] 

Can you provide a reproducer?

It would require to compile the framework (note: C++17 ROOT needed)

rm -rf rest-framework
git clone git@github.com:rest-for-physics/framework.git rest-framework
cd rest-framework
python3 pull-submodules.py --latest --onlylibs --data
cd source/libraries/axion/data
git checkout master
git pull
cd ../../../../
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_INSTALL_PREFIX=../install -DRESTLIB_AXION=ON -DRESTLIB_RAW=ON -DRESTLIB_DETECTOR=ON -DREST_MPFR=ON .. 2>&1 | tee -a cmake.out.txt
make -j install 2>&1 | tee -a make.out.txt
source ../install/thisREST.sh

The example you may use to produce the file (change the parameter eventsToProcess inside helioscope.rml in line 30 to only 10). Then:

cd rest-framework/source/libraries/axion/examples/full-ray-tracing
sed -i -e '{s/100000/10/}' helioscope.rml
restManager --c helioscope.rml --o output.root 2>&1 | tee -a restManager.out.txt
root -l -q -b output.root -e 'gFile->ls(); gFile->ShowStreamerInfo();'

You will see then how the Process objects do not appear inside ShowStreamerInfo(). To test a change on TRestAxionGeneratorProcess data members you may switch inside the submodule source/libraries/axion to the branch jgalan_axion_mass, recompile, then try to read the previous generated file.

However, if you run another example:

cd rest-framework/pipeline/trex/
wget https://sultan.unizar.es/trexdm-readouts/readouts_v2.3.root
restManager --c 01_raw.rml --f R01928_tagTest_Vm_250_Vd_160_Pr_6_Gain_0x0_Shape_0xF_Clock_0x4-068.aqs 2>&1 | tee -a restManager.out.txt
root -l -q -b RawData_01928.root -e 'gFile->ls(); gFile->ShowStreamerInfo();'

You will see how the generated output RawData_01928.root call to ShowStreamerInfo() contains all the TRestXYZProcess that were used.

I am afraid I cannot create a simplified code that reproduces the problem. I created an issue on our side:

The “make” reports various warnings:

(...)
[ 12%] generating: /.../rest-framework/build/rootdict/CINT_TRestPhysics.cxx with /.../rest-framework/source/framework/tools/inc/TRestPhysics.h;/.../rest-framework/build/rootdict/TRestPhysics_Linkdef.h
Warning: Unused class rule: TRestPhysics
(...)
[ 15%] generating: /.../rest-framework/build/rootdict/CINT_TRestReflector.cxx with /.../rest-framework/source/framework/tools/inc/TRestReflector.h;/.../rest-framework/build/rootdict/TRestReflector_Linkdef.h
Warning: Unused class rule: TRestReflector
(...)
[ 16%] generating: /.../rest-framework/build/rootdict/CINT_TRestStringHelper.cxx with /.../rest-framework/source/framework/tools/inc/TRestStringHelper.h;/.../rest-framework/build/rootdict/TRestStringHelper_Linkdef.h
Warning: Unused class rule: TRestStringHelper
(...)
[ 20%] generating: /.../rest-framework/build/rootdict/CINT_TRestSystemOfUnits.cxx with /.../rest-framework/source/framework/core/inc/TRestSystemOfUnits.h;/.../rest-framework/build/rootdict/TRestSystemOfUnits_Linkdef.h
Warning: Unused class rule: TRestSystemOfUnits
(...)
[ 85%] Building CXX object source/libraries/axion/CMakeFiles/RestAxion.dir/src/TRestAxionSolarFlux.cxx.o
/.../rest-framework/source/libraries/axion/src/TRestAxionSolarFlux.cxx: In member function ‘TH1F* TRestAxionSolarFlux::GetFluxHistogram(std::string, Double_t)’:
/.../rest-framework/source/libraries/axion/src/TRestAxionSolarFlux.cxx:575:23: warning: too many arguments for format [-Wformat-extra-args]
  575 |         new TH2F(Form("FluxTable", GetName()), "", 100, 0., 1., (Int_t)(20. / binSize), 0., 20.);
      |                       ^~~~~~~~~~~
(...)
[100%] Built target RestAxion
Install the project...
CMake Warning (dev) at cmake_install.cmake:299:
  Syntax Warning in cmake code at column 5

  Argument not separated from preceding token by whitespace.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at cmake_install.cmake:299:
  Syntax Warning in cmake code at column 58

  Argument not separated from preceding token by whitespace.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at cmake_install.cmake:317:
  Syntax Warning in cmake code at column 5

  Argument not separated from preceding token by whitespace.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at cmake_install.cmake:317:
  Syntax Warning in cmake code at column 58

  Argument not separated from preceding token by whitespace.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at cmake_install.cmake:335:
  Syntax Warning in cmake code at column 5

  Argument not separated from preceding token by whitespace.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at cmake_install.cmake:335:
  Syntax Warning in cmake code at column 58

  Argument not separated from preceding token by whitespace.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at cmake_install.cmake:353:
  Syntax Warning in cmake code at column 5

  Argument not separated from preceding token by whitespace.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at cmake_install.cmake:353:
  Syntax Warning in cmake code at column 58

  Argument not separated from preceding token by whitespace.
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Install configuration: ""
rm: cannot remove '/.../rest-framework/install': No such file or directory
(...)

Sorry, I forgot to mention you need to pull also the data, adding --data to the pull-submodules.py script will force downloading the data based repositories.

python3 pull-submodules.py --latest --data

This should download the required fluxes.rml file.

Ok, this is because detectorlib was not compiled. I should have provided a more complete cmake command. That example requires raw and detector libraries.

cmake -DRESTLIB_DETECTOR=ON -DRESTLIB_RAW=ON ..

Not sure about the origin of those warnings, I suspect that we are generating a dictionary for a class that does not inherit from TObject? Can it be the origin of those warnings?

The relevant repository is :

Updating submodule: source/libraries/axion/data[ OK ]

The file will be installed at ../install/data/axion/solarFlux/ after executing make install. Only then, the execution of restManager --c helioscope.rml --o output.root will be able to find the file.

It seems there is a problem, it is not retrieving the latest master from the data repository …

cd rest-framework/source/libraries/axion/data
git checkout master
git pull

Ok, just updated with all necessary commands. Should be working for monkeys and donkeys. :grinning:

Now, the instructions are “theorists-proof”.

@pcanal Can it be that the “threaded” output misbehaved (when saving the “streamer info”)? In the “helioscope.rml” trial’s output, I can see:

(...)
TRestProcessRunner : preparing threads...
(...)
TRestProcessRunner: 1 + 14 processes loaded, 1 threads prepared!
(...)
Configuring output file, writing metadata and tree objects
Merging thread files together
Creating file : ./output.root
(...)

Not sure if that is related. But the fact is that the ROOT file contains the objects stored in disk, but not the streamers.

We really worked hard to get the automatic schema evolution working properly in our project. But now I have no clue where the problem might be coming from.

@pcanal Any news about the “streamer info” problem?

I run out of ideas, but perhaps it is because we are overriding the TObject::Write function?

Int_t TRestMetadata::Write(const char* name, Int_t option, Int_t bufsize) {
    if (fStore) {
        configBuffer = ElementToString(fElement);
        return TNamed::Write(name, option, bufsize); 
    }
    return -1;
}   

I am trying to build the rest framework and it fails for me with:

In file included from /home/pcanal/root_working/test/2022-io/MissingStreamerInfo/rest-framework/source/framework/tools/inc/TRestComplex.h:27:
/home/pcanal/root_working/test/2022-io/MissingStreamerInfo/rest-framework/source/framework/tools/inc/mpreal.h:2150:5: error: use of undeclared identifier 'mpfr_rootn_ui'
    mpfr_rootn_ui(y.mpfr_ptr(), x.mpfr_srcptr(), k, r);
    ^
/home/pcanal/root_working/test/2022-io/MissingStreamerInfo/rest-framework/source/framework/tools/inc/mpreal.h:2653:5: error: use of undeclared identifier 'mpfr_nrandom'
    mpfr_nrandom(x.mpfr_ptr(), state, rnd_mode);
    ^

Maybe a missing: sudo apt install libmpfr-dev

Try: grep mpfr_rootn_ui /usr/include/mpfr.h

Yes, it must be mpfr missing libraries. But sometimes we may get problems if the mpfr library version is not the appropriate one.

I think to investigate this problem is safe to disable MPFR.

MPFR could be disabled by cmake -DREST_MPFR=OFF

Then, perhaps is better to launch the attached file.

restManager --c generator.txt

generator.txt (2.7 KB)