@pcanal Any news about the “streamer info” problem?
I run out of ideas, but perhaps it is because we are overriding the TObject::Write function?
Int_t TRestMetadata::Write(const char* name, Int_t option, Int_t bufsize) {
if (fStore) {
configBuffer = ElementToString(fElement);
return TNamed::Write(name, option, bufsize);
}
return -1;
}
I am trying to build the rest framework and it fails for me with:
In file included from /home/pcanal/root_working/test/2022-io/MissingStreamerInfo/rest-framework/source/framework/tools/inc/TRestComplex.h:27:
/home/pcanal/root_working/test/2022-io/MissingStreamerInfo/rest-framework/source/framework/tools/inc/mpreal.h:2150:5: error: use of undeclared identifier 'mpfr_rootn_ui'
mpfr_rootn_ui(y.mpfr_ptr(), x.mpfr_srcptr(), k, r);
^
/home/pcanal/root_working/test/2022-io/MissingStreamerInfo/rest-framework/source/framework/tools/inc/mpreal.h:2653:5: error: use of undeclared identifier 'mpfr_nrandom'
mpfr_nrandom(x.mpfr_ptr(), state, rnd_mode);
^
Maybe a missing: sudo apt install libmpfr-dev
Try: grep mpfr_rootn_ui /usr/include/mpfr.h
Yes, it must be mpfr missing libraries. But sometimes we may get problems if the mpfr library version is not the appropriate one.
I think to investigate this problem is safe to disable MPFR.
MPFR could be disabled by cmake -DREST_MPFR=OFF
Then, perhaps is better to launch the attached file.
restManager --c generator.txt
generator.txt (2.7 KB)
@Javier_Galan Which version of mpfr is needed:
Package mpfr-devel-3.1.1-4.el7.x86_64 already installed and latest version
Nothing to do
rootsrv1:rest-framework (master) pcanal$ grep mpfr_rootn_ui /usr/include/mpfr.h
It might be an issue with the version. In the following README.md we give instructions for specific mpfr (mpir is not necessary) version.
My REST compilation links against 4.0.2
:/usr/lib/x86_64-linux-gnu$ dpkg -l libmpfr-dev
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-=================-============-============-==============================================================
ii libmpfr-dev:amd64 4.0.2-1 amd64 multiple precision floating-point computation developers tools
But disabling -DREST_MPFR might be the best option now.
grep MPFR_VERSION_STRING /usr/include/mpfr.h # Ubuntu 22.04 provides "4.1.0"
I think I remember I was able to configure it without “mpfr” but then there were warnings in the form: “MPFR libraries not linked to REST libraries. Try adding -DREST_MPFR=ON to your REST compilation”
humm … next problem was a missing -phread somewhere … alright … moving to a different platform and trying again
Ok, now is certain that I have 4.0.2
grep MPFR_VERSION_STRING /usr/include/mpfr.h
#define MPFR_VERSION_STRING "4.0.2"
You will see then how the
Processobjects do not appear insideShowStreamerInfo().
You will see how the generated outputRawData_01928.rootcall toShowStreamerInfo()contains all theTRestXYZProcessthat were used.
Alright, on Ubuntu 22, the build succeeds and I can reproduce those 2 features. Now diving into it.
1 Like
It looks like the problem is in TFileMerger::Merge when ‘appending-to/updating’ an existing file. It looks like in this case the StreamerInfo is over-written rather than extended.
To work around the problem in TRestProcessRunner::MergeOutputFile() instead of having the 2 final output file being the same make sure they are different:
I.e. in:
fOutputDataFile->Write(nullptr, TObject::kOverwrite);
fOutputDataFile->Close();
fRunInfo->MergeToOutputFile(files_to_merge, fOutputDataFile->GetName());
do
fOutputDataFile->Write(nullptr, TObject::kOverwrite);
fOutputDataFile->Close();
files_to_merge.push_back(fOutputDataFile->GetName());
fRunInfo->MergeToOutputFile(files_to_merge, "someothername.root");
To be clear this is likely to be a bug in TFileMerger::Merge (or less likely in MergeToOutputFile).
I will continue to investigate.
Actually I was wrong. The problem is not there and the work-around wont work.
The real problem is that TRestRun::CloseFile is not run (likely) because the destructor for TRestRun is not run.
The consequence is that the output file that is reopened at the very end of TRestRun::MergeToOutputFile 
// write metadata into the output file
fOutputFile = new TFile(fOutputFileName, "update");
is then closed only by the last ditch effort done by TROOT to close the file during the process tear down. However by that time the library containing the dictionary for the *Process classes has been unloaded and thus their StreamerInfo can not be stored in the file.
So the solution is to insure that that TRestRun::CloseFile is called before the end of the main function.
So, it would also be sufficient to explicitly “delete” the TRestRun object (its destructor calls its CloseFile method).
I have added a CloseFile call where I thought it was more natural to do it, at the end of TRestProcessRunner::RunProcess. I added some additional output to understand what it is contained inside the file, what keys there are in the file just before closing. Still ShowStreamerInfo does not show *Process definitions.
I pushed the changes to the streamer_fix_tests.
The output I get is:
[== - Filename : AxionPhotonProbabilityWindowOffset.root ==]
File output pointer : 0x5608ddb54d50
Closing file!!
Class name : TRestProcessRunner
Class name : TRestAxionGeneratorProcess
Class name : TRestAxionAnalysisProcess
Class name : TRestAxionTransportProcess
Class name : TRestAxionAnalysisProcess
Class name : TRestAxionFieldPropagationProcess
Class name : TRestAxionAnalysisProcess
Class name : TRestAxionTransmissionProcess
Class name : TRestAxionOpticsProcess
Class name : TRestAxionAnalysisProcess
Class name : TRestAxionTransmissionProcess
Class name : TRestAxionAnalysisProcess
Class name : TRestAxionTransportProcess
Class name : TRestAxionAnalysisProcess
Class name : TTree
Class name : TRestAnalysisTree
Class name : TRestRun
Class name : TRestAxionSolarFlux
Class name : TRestAxionMagneticField
Class name : TRestAxionBufferGas
Class name : TRestAxionWolterOptics
Class name : TRestAxionXrayWindow
Class name : TRestAxionXrayWindow
Class name : TRestAxionXrayWindow
Class name : TRestAxionXrayWindow
File closed
Looking at the order of the keys, it might happen that there is some relation between the fact that processes appear before TTree, TRestAnalysisTree and TRestRun? TRestProcessRunner does not appear either when calling ShowStreamerInfo.
This is a “multithreaded” application.
Can it be that the “runner” threads load all libraries (because they explicitly use these “Process” objects), but the “main” thread (which saves the file) does not load some of them (because it doesn’t explicitly use these classes)?
Ok, I finally spotted the bug!
It happened that the file pointer where we were writing the *Process objects was not the same as the file output pointer located at TRestRun.
The problem was also that those objects were written at the beginning of the data processing, now I invoke WriteMetadata just after merging, and force to write to the fRunInfo->GetOutputFile() pointer. Perhaps there was another work around, but the following commit solved the problem.
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.