It’s a 5MB file. What is the standard way of sharing files like this on this forum? I could upload the file (I see the icon as I compose this reply) but I don’t want to fill up whatever space you’ve got.
People usually use free services like Dropbox, CERNBox, Mega, WeTransfer, etc.
Of course; I’m just being stupid this morning. Here’s a link:
Yes, I can confirm that the LArHits TTree gets messed up somehow.
And everything is fine when I do the same procedure with my own files with TTrees, so I guess it is something specific to this particular TTree.
This may require someone who understands the structure of a TFile to comment. What I can offer is this:
- The link to the “good” version of file, as written by Geant4’s analysis manager; this is the same link as above:
- The link to the “broken” version of file, the one created by
auto input=TFile::Open("gramsg4-working.root","UPDATE");input->Close();within ROOT.
- The output of
cmp -l gramsg4-works.root gramsg4-broken.root; cmp is the UNIX utility for comparing binary files. The bytes themselves are displayed in octal.
The binary comparison is shorter than I thought it was going to be. There are only a few differences at the beginning and at the end of the respective files.
The question is: Why should UPDATE have changed the contents of the file at all, much less put the file in an unreadable state? I’ve looked at the description of a ROOT file, but I don’t see a “smoking gun” that explains what’s going on, other than perhaps something is being potentially added at the end of the file.
At this point, my links are probably confusing matters, but I can’t help but try:
This link is the output of the command diff <(xxd gramsg4-works.root) <(xxd gramsg4-broken.root). In other words, it’s comparison of the differences between the two ROOT files in hex. Again, there aren’t many differences; just enough to corrupt the file, I guess.
In a naive attempt to understand the differences, I created a spreadsheet with differences in the headers of the two files. I’ve highlighted the differences in the spreadsheet:
It’s clear that UPDATE is adding a UUID where the original file did not have one. I understand the logic behind that. But I don’t understand the changes in the “free data records” fields; it looks like free space is being reduced somehow.
Any thoughts?
Seeing this:
auto geoManager = gGeoManager->Import("parsed.gdml");
std::shared_ptr<TFile> outputFile ( TFile::Open(g4job.root,"UPDATE") );
geoManager->Write("DetectorGeometry");
outputFile->Close();
which should be:
auto geoManager = gGeoManager->Import("parsed.gdml");
std::shared_ptr<TFile> outputFile ( TFile::Open(g4job.root,"UPDATE") );
geoManager->Write("DetectorGeometry");
outputFile->Write();
outputFile->Close();
I tried that. It made no difference.
Once again, I can cause the problem by entitrely commenting out the TGeoManager lines in Geant4, then simply opening and closing the file post-G4:
root
auto file = TFile("gramsg4.root","UPDATE");
file->Close();
.q
This is sufficient to cause the problem. TGeoManager has nothing to do with it.
Then this must be an issue in the Geant4 re-implementation of ROOT I/O. This sounds like the same as here: Opening a file in update mode causes Error in <TBasket::Streamer>: The value of fNbytes is incorrect - #15 by pcanal
Yep, that was it! Thanks!
I’ll file a bug report with the Geant4 team. We’ll see if we can spare future users the same frustrating experience I had.
Now I have to figure out how to run hadd from within the simulation. (If I create a separate post-processing step, the users will forget to run it.)
For what it’s worth:
#include "TSystem.h"
// ... run Geant4 simulation including G4AnalysisManager
G4String filename = <the output file from G4AnalysisManager after it's been closed>
// Search for hadd in the user's environment.
auto path = gSystem->Getenv("PATH");
auto hadd = gSystem->Which(path,"hadd");
if ( hadd == nullptr ) {
// We could not find hadd.
if (debug || verbose)
G4cout << "gramsg4.cc - Could not find hadd, filename '"
<< filename << "' unchanged; cannot be opened in UPDATE mode"
<< G4endl;
}
else {
// Define the work file name.
G4String workfile = "work_" + filename;
// Rename the output file to the temporary work file name.
gSystem->Rename(filename,workfile);
// Setting the hadd verbosity argument.
G4String vhadd = "-v 0";
if (debug || verbose)
vhadd = "-v 99";
// hadd -v 0 -f gramsg4.root work_gramsg4.root
G4String haddCommand = G4String(hadd) + " " + vhadd
+ " -f " + filename + " " + workfile;
if (debug || verbose)
G4cout << "gramsg4.cc - Executing command '"
<< haddCommand << "'"
<< G4endl;
// Execute the hadd command.
gSystem->Exec(haddCommand);
// Remove the work file; it just wastes disk space at this point.
if (debug || verbose)
G4cout << "gramsg4.cc - Deleting '" << workfile << "'"
<< G4endl;
gSystem->Unlink(workfile);
}
After these “file repair” lines, I can open filename in UPDATE mode and do whatever I like with it.
The code executed by hadd is also available through the class TFileMerger
The lack of TFileMerger documentation makes using the class problematic. I tried it, and it appears to execute:
G4String workfile = "work_" + filename;
// Rename the output file to the temporary work file name.
gSystem->Rename(filename,workfile);
TFileMerger* merger = new TFileMerger(kFALSE);
merger->AddFile(workfile);
merger->OutputFile(filename);
merger->Merge();
// Remove the work file; it just wastes disk space at this point.
gSystem->Unlink(workfile);
The status message appears in the program output:
[TFile::Cp] Total 5.09 MB |====================| 100.00 % [146.4 MB/s]
But all the ntuples in the output file are now corrupted:
root [0]
Attaching file gramsg4.root as _file0...
(TFile *) 0x5632e50c04b0
root [1] TrackInfo->Scan()
Error in <TBufferFile::ReadClassBuffer>: Could not find the StreamerInfo for version 5 of the class TTree, object skipped at offset 77
Error in <TBufferFile::CheckByteCount>: object of class TTree read too few bytes: 2 instead of 9348
There’s probably some enum or boolean I have to set somewhere, but it’s not clear what it should be.
Try
merger->Merge(TFileMerger::kIncremental | TFileMerger::kAll);
Cheers,
Philippe
Same error.
From my perspective, the source code of hadd is a mass of uncommented code. It must somehow do what I want it to do, since just using hadd gives the results I want, but I can’t even tell what the default options to TFileMerger are supposed to be.
I guessed an answer that worked:
G4String workfile = "work_" + filename;
// Rename the output file to the temporary work file name.
gSystem->Rename(filename,workfile);
TFileMerger* merger = new TFileMerger(kFALSE,kFALSE);
auto input = TFile::Open(workfile);
auto compression = input->GetCompressionSettings();
merger->AddFile(input);
merger->OutputFile(filename,"RECREATE",compression);
merger->Merge();
gSystem->Unlink(workfile);
It was setting the compression settings of the output file to be the same as the input file.
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.