TTree compression factor 1.00

Hi-

I’m having an issue where the root files I build as a result of a TTree-based analysis are not being compressed. If I check the compression of the tree with a tree->Print(), each branch and the total tree return a compression factor of 1.00. Previous analysis of similar data have yielded compression factors of > 15. If I manually specify a compression level or compression settings prior to the TFile->Write() it seems to have no effect. I realize this is a somewhat vague question, but what things might prevent a TFile from being compressed?

-Matt


ROOT Version: 6.12.06, 6.14.06
Platform: SL 6.4, MACOSX 10.14.1
Compiler: gcc4.8.2, Xcode


It ‘could’ that this particular data set is uncompressible or too small to be compressed.

If you provide both the odd file and a typical file we can take a closer look at what might have caused it.

Try to enforce the compression algorithm and its level, e.g.:
TFile *f = TFile::Open("f.root", "RECREATE", "", 101); // 101 = 100 * ROOT::kZLIB + 1

Hi Wile, thanks for the tip, but setting at TFile creation or immediately before closing with TFile::SetCompressionSettings() doesn’t seem to have an effect.

Attached are 2 sample root files, the results of processing of the same data set. The only difference between these 2 files is should be the values in the fNPE branch of the tree simTree. samplefile.root is incompressible, while samplefile_small.root has a compression factor of 20.

samplefile.root
samplefile_small.root

The “simTree” in the “samplefile.root” file has one additional branch:

*Br   29 :fAnalysisEventDataVector.fIsVetoOnly :                             *
*         | Bool_t fIsVetoOnly[fAnalysisEventDataVector_]                    *

Also, the “samplefile.root” was created with ROOT 6.12/06 while the “samplefile_small.root” file with ROOT 6.12/04.

Yes, you’re right; I’d missed that. That branch is populated with zeros; I’m not sure how that would affect its compressibility, though.

Removing the fIsVetoOnly branch from the analysis doesn’t affect the compressibility, so the problem is not in that branch.

Hi all-

I seem to have sorted this out. As part of the analysis process I loaded a pair of histograms, a TH2D and a TH3D from an external root file to use as lookup tables. I did this by doing the following:

fMapFile = new TFile(fMapFileName.c_str());
if(fMapFile==NULL) { cerr << "Couldn’t open file: " << fMapFileName << endl; return; }
fProbMapInterior = (TH3D*) fMapFile->Get(“ProbMapInterior”);
fProbMapInterior->SetDirectory(0);
fProbMapExterior = (TH2D*) fMapFile->Get(“ProbMapExterior”);
fProbMapExterior->SetDirectory(0);
fMapFile->Close();

This allowed for the tree building and writing to file to proceed while these 2 histograms remained resident in memory. However either not explicitly returning to the previously created output file before writing the tree, or leaving these 2 histograms in memory in Directory(0) prevented the output tree from being compressible.

Instead of moving these histograms to Directory(0), I leave the file open, and use a cd() call to move to the file when I need to access the histograms, and another cd() to return to the previous directory and it seems to work fine.

Thanks for your input.

This does not sound right. What you describe should not affect the compression of the TTree. So there is either a behavior I forgot about or a deficiency we need to fix. Could you provide a reproducer?

Thanks,
Philippe.

I think I can reproduce the problem (the fix is trivial, as you can see below):

{
  // create a new file
  TFile *f_new = TFile::Open("f_new.root", "recreate", "", 101);
  
#if 1 /* 0 or 1 */
  gROOT->cd(); // switch to RAM
#else /* 0 or 1 */
  // open some old file
  TFile *f_old = TFile::Open("f_old.root");
  // retrieve any objects
  delete f_old;
#endif /* 0 or 1 */
  
  f_new->cd(); // MANDATORY, OTHERWISE THE FILE COMPRESSION WILL NOT WORK !!!
  TTree *t = new TTree("t", "t"); // create a new DISK RESIDENT tree
  
  // fill the tree
  Float_t v; t->Branch("v", &v, "v/F");
  for (Int_t i = 0; i < 999999; i++) { v = gRandom->Gaus(0., 1.); t->Fill(); }
  
  // save the tree to a file
  f_new->cd(); // just a precaution
  t->Write();
  delete f_new; // automatically deletes "t", too
}

@pcanal So, if you create a “RAM resident” tree and then write it to a file, it will NOT be compressed (a bug in ROOT? or a feature?).

@pcanal Unfortunately, I have also found that the ROOT command line tools, which can change the compression settings for the destination file (e.g. “rootcp” and “rootmv”), are NOT able to change the compression level for trees (a bug in ROOT? or a feature?).

So, if you create a “RAM resident” tree and then write it to a file, it will NOT be compressed (a bug in ROOT? or a feature?).

It is a bug …

Unfortunately, I have also found that the ROOT command line tools, which can change the compression settings for the destination file (e.g. “rootcp” and “rootmv”), are NOT able to change the compression level for trees (a bug in ROOT? or a feature?).

likely also a bug.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Could you create a bug report (and possibly fix it, depending on relative priority), @pcanal ?

See https://github.com/root-project/root/issues/6349 (rootcp and rootmv and compression level update)
and https://github.com/root-project/root/issues/6350 (lack of compression)

1 Like