Accessing a root file with two different versions of root

prophecy · March 8, 2010, 6:18pm

I have created a root file with

  *******************************************
  *                                         *
  *        W E L C O M E  to  R O O T       *
  *                                         *
  *   Version  4.04/02b       3 June 2005   *
  *                                         *
  *  You are welcome to visit our Web site  *
  *          http://root.cern.ch            *
  *                                         *
  *******************************************

Compiled on 7 March 2007 for linux with thread support.

CINT/ROOT C/C++ Interpreter version 5.15.169, Mar 14 2005

I then try to read the file with root version

  *******************************************
  *                                         *
  *        W E L C O M E  to  R O O T       *
  *                                         *
  *   Version   5.24/00      29 June 2009   *
  *                                         *
  *  You are welcome to visit our Web site  *
  *          http://root.cern.ch            *
  *                                         *
  *******************************************

ROOT 5.24/00 (trunk@29257, Jun 30 2009, 09:23:51 on macosx)

CINT/ROOT C/C++ Interpreter version 5.17.00, Dec 21, 2008

What I find is that if I use the original root I can read the file extending to all of the entries. When I try to use the new root (the 5.24 version) I can only read to entry 5312017 and after that any other read shows the same values. I checked that the file actually has different values in all of the events by using the original root version. There are actually 5315833 entries in the file.

An interesting thing is that if I try to read entry 5315834 in the original root version I get junk values. I would have thought that a warning would be presenting since trying to access rows that do not exist.

When I use the new root it is acting like there are only 5312017 entries in the file when I try to get the values but it says (though TTree::GetEntries()) that there are 5315833 entries in there. Can somebody help me to see what is going on here. The file is located at

clued0.fnal.gov/~prophecy/FNAL/T … tfile.root

If you are coming from D0 then you can see the file at

/rooms/galactica/prophecy/hwbbjjAnalysis/TMVAOutputs/GoldenTrees21_Dissertation_MCv4_Techni/techni_115_264_240/DTOutput/Datasets/data_trigger_0Tag_rev1_added/analysis_trigger_nocuts_DT.root

Thanks for any help…

brun · March 8, 2010, 8:19pm

Using our production version 5.26 I do not see any problem reading your file.

Rene

prophecy · March 9, 2010, 3:49am

Rene,

 I have taken the time to install the binary version of root on my laptop (Mac OSX Snow Leopard).  I tried to look at that file and the following is what I got:

  *******************************************
  *                                         *
  *        W E L C O M E  to  R O O T       *
  *                                         *
  *   Version   5.26/00  14 December 2009   *
  *                                         *
  *  You are welcome to visit our Web site  *
  *          http://root.cern.ch            *
  *                                         *
  *******************************************

ROOT 5.26/00 (trunk@31882, Dec 14 2009, 20:18:36 on macosx)

CINT/ROOT C/C++ Interpreter version 5.17.00, Dec 21, 2008
Type ? for help. Commands must be C++ statements.
Enclose multiple statements between { }.
root [0] 
Attaching file analysis_trigger_nocuts_DT.root as _file0...
root [1] descision_tree->SetBranchStatus("*", 0);
root [2] descision_tree->SetBranchStatus("DT_Result", 1);
root [3] descision_tree->GetEntries()
(const Long64_t)5315833
root [4] descision_tree->Show(0)
======> EVENT:0
 DT_Result       = -0.216396
root [5] descision_tree->Show(1000)
======> EVENT:1000
 DT_Result       = -0.48963
root [6] descision_tree->Show(5000000)
======> EVENT:5000000
 DT_Result       = -0.332449
root [7] descision_tree->Show(5315833)
Error in <TTree::Show()>: Cannot read entry 5315833 (entry does not exist)
root [8] descision_tree->Show(5000000)
======> EVENT:5000000
 DT_Result       = -0.332449
root [9] descision_tree->Show(5315832)
Error in <TTree::Show()>: Cannot read entry 5315832 (I/O error)
root [10] descision_tree->Show(5000000)
Error in <TTree::Show()>: Cannot read entry 5000000 (I/O error)
root [11] descision_tree->Show(1000)
Error in <TTree::Show()>: Cannot read entry 1000 (I/O error)
root [12] descision_tree->Show(0)
Error in <TTree::Show()>: Cannot read entry 0 (I/O error)

As you can see, there is some sort of a bug here since I can not read any of the entries after I try to read the last entry. I did do a md5sum cross-check to make sure that the file that I am using on my laptop is in fact the same one that I have avail to you. So, I am not sure what is going on. Can you replicate the above error?

Justace

brun · March 9, 2010, 8:52am

The problem is now understood. It happens when processing old files and trying to read beyond the last entry.
The solution is to :
-not read beyond the last entry
-or reload the Tree header after having passed the last entry
-or use the SVN trunk.

Thanks for this report.

Rene

prophecy · March 9, 2010, 7:06pm

Rene,

 It looks as though the problem is a bit deeper.  I have tried to do some more figuring.  I saw a strange bump in a histogram which I wanted to investigate.  I have converted the DT Result variable in my TTree into a text file so that the numbers can be easily compared outside of root.  I have run the same code using the root version at D0 and also on my laptop using the new version of root.  I still see the differences in the output (with the same input file) even when not accessing the last entry till the very end.  The input files were verified to be the same via md5sum like before.  (I have used a smaller file since it would be easier to work with).  The code that I am using is presented below:

#include <TFile.h>
#include <TTree.h>
#include <fstream>

using namespace std;

int main(int argc, char *argv[]) {
  ofstream output(argv[2], ios::trunc);

  TFile input(argv[1], "READ");
  TTree *tree = (TTree*) input.Get("descision_tree");

  double dt;

  tree->SetBranchStatus("*", 0);
  tree->SetBranchStatus("DT_Result", 1);
  tree->SetBranchAddress("DT_Result", &dt);

  for(long i = 0; i < tree->GetEntries(); i++) {
    tree->GetEntry(i);

    output << dt << endl;
  }

  output.close();
  input.Close();
}

I have placed the output at the following web address:

clued0.fnal.gov/~prophecy/FNAL/CROSSCHECK

In there you will find three directories: Remote_1, Local_1, and Local_2. Here the files under Remote_1 are from D0 using the old version of root. Local_1 is using the exact same code here on my laptop using the new version of root and Local_2 is an equivalent file using the original code that brought the problem to my attention. The input files can be found in the “InputFiles” subdirectory.

I have created a small perl script to investigate the differences. The output of the script is in the file “ComparisonDump_3.txt”. The format of this file is four columns. The first column is the index in the file (the first being = 1) while the second and third column are the two values being compared. The final column is the difference between the two files. You see that there are in fact 228262 entries in both files and that the differences begin at row 227374. At this point the value from the new version of root (on the right) shows the same value no matter what.

Thanks for looking into this.

brun · March 10, 2010, 7:37am

Could you clarify what you mean by “the new version of ROOT” ?
Did you install from source from the SVN trunk?
I do not see the kind of symptom that you describe when using the SVN trunk.

Rene

prophecy · March 10, 2010, 2:51pm

Sorry for the confusion. The old version is the one from D0 and has the following header:

  ******************************************* 
  *                                         * 
  *        W E L C O M E  to  R O O T       * 
  *                                         * 
  *   Version  4.04/02b       3 June 2005   * 
  *                                         * 
  *  You are welcome to visit our Web site  * 
  *          http://root.cern.ch            * 
  *                                         * 
  ******************************************* 

Compiled on 7 March 2007 for linux with thread support. 

CINT/ROOT C/C++ Interpreter version 5.15.169, Mar 14 2005

The new version is

  ******************************************* 
  *                                         * 
  *        W E L C O M E  to  R O O T       * 
  *                                         * 
  *   Version   5.26/00  14 December 2009   * 
  *                                         * 
  *  You are welcome to visit our Web site  * 
  *          http://root.cern.ch            * 
  *                                         * 
  ******************************************* 

ROOT 5.26/00 (trunk@31882, Dec 14 2009, 20:18:36 on macosx) 

CINT/ROOT C/C++ Interpreter version 5.17.00, Dec 21, 2008

You say that you do not see the problem. Did you compile the example code and generate a dt output list and compare or did you do the simple test where we access the last event and then things get screwed up?

Justace

pcanal · March 10, 2010, 3:23pm

Hi,

There was a fix added to the trunk since v5.26 that solve the issue. Do you use the trunk (or use the work-around mentioned by Rene).

Cheers,
Philippe.

prophecy · March 10, 2010, 3:45pm

That is the thing. Rene stated that the problem happens when you try to access the last entry. In the last example, I did not do that and there are reading problems even before reading the last entry.

I do not run the trunk since I was having some build issues on my mac.

pcanal · March 11, 2010, 10:43pm

Hi,

The problem was affect all the entries in the last ‘buffer’. In your testfile.root this means that the last 2184 entries where incorrectly read.

The problem is solved only in the trunk and you do no really have any workaround to read the last few entries. You can however improve your script to detect the error case:for(long i = 0; i < tree->GetEntries(); i++) { if ( tree->GetEntry(i) < 0) { cerr << "Humm can't read anymore at :" << i << '\n'; break; }Cheers,
Philippe.

prophecy · March 11, 2010, 10:48pm

Thanks for the info. I will wait for the Mac binary.

Justace

pcanal · March 11, 2010, 11:16pm

Hi Justace,

Building the trunk on macos, should not be too hard , so you do not have to wait until the binary distributions.

Cheers,
Philippe.

prophecy · March 11, 2010, 11:21pm

Alright… I will give it a shot when I have some time. You know how dissertation time can be.

Thanks again for the help