Dear experts,
I am trying to cut a tree, from quite a large tree to begin with, where the cut results in about 28k entries. (I want to cut the tree to feed it into a roodataset)
This results in the following error. How do I get around this?
Thanks in advance for your help!
Regards
Fatima Soomro
Error in TBufferFile::WriteByteCount: bytecount too large (more than 1073741822)
Error in TBufferFile::WriteByteCount: bytecount too large (more than 1073741822)
Error in TBufferFile::CheckByteCount: object of class TObjArray read too many bytes: 1324415422 instead of 250673598
Warning in TBufferFile::CheckByteCount: TObjArray::Streamer() not in sync with data, fix Streamer()
Error in TExMap::Remove: key 356262210 not found at 7083
Warning in TBufferFile::CheckObject: reference to object of unavailable class TObject, offset=356262210 pointer will be 0
Error in TExMap::Remove: key 65536 not found at 601
Warning in TBufferFile::CheckObject: reference to object of unavailable class TObject, offset=65536 pointer will be 0
Error in TExMap::Remove: key 2063347782 not found at 6384
Warning in TBufferFile::CheckObject: reference to an unavailable class, pointers of that type will be 0
Error in TBufferFile::CheckByteCount: Byte count probably corrupted around buffer position 562577846:
-1936385210 for a possible maximum of 797486658
Warning in TBufferFile::CheckObject: reference to object of unavailable class TObject, offset=562577848 pointer will be 0
Error in TExMap::Remove: key 31603983 not found at 4502
Warning in TBufferFile::CheckObject: reference to object of unavailable class TObject, offset=31603983 pointer will be 0
Error in TExMap::Remove: key 81923 not found at 754
Warning in TBufferFile::CheckObject: reference to object of unavailable class TObject, offset=81923 pointer will be 0
Error in TExMap::Remove: key 842019120 not found at 2126
In the way you configured the output file and/or tree, you end up try to save in a single buffer more than 1GB of data (This should not be happening when just filtering an existing TTree).
How did you create the output tree? Is it attached to the output file or is it a memory tree? Which version of ROOT are you using?
I have put the relevant code here:
/afs/cern.ch/user/f/fsoomro/public/roofitProblem/
The root files needed are here:
/castor/cern.ch/user/f/fsoomro/temp_ntuples/
===> In momentum bin 1 entries after cut: (Dst_2010_plus_MM-D0_MM)>100 && (Dst_2010_plus_MM-D0_MM)<180 && KminusL0Global_TIS ==1 && KminusHlt1Global_TIS ==1 && KminusHlt2Global_TIS ==1 && Kminus_P/1000>5.0 && Kminus_P/1000 <10.0 && Kminus_PT/1000 >0.8 && Kminus_PT/1000 < 1.7 are 43029 <===
Error in TBufferFile::WriteByteCount: bytecount too large (more than 1073741822)
Error in TBufferFile::WriteByteCount: bytecount too large (more than 1073741822)
Error in TBufferFile::CheckByteCount: object of class TObjArray read too many bytes: 1570442742 instead of 496700918
Warning in TBufferFile::CheckByteCount: TObjArray::Streamer() not in sync with data, fix Streamer()
Error in TBufferFile::CheckByteCount: object of class TTree read too few bytes: 496701156 instead of 496702707
[#1] INFO:Eval – RooAbsReal::attachToTree(Dst_2010_plus_MM) TTree Float_t branch Dst_2010_plus_MM will be converted to double precision
[#1] INFO:Eval – RooTreeDataStore::loadValues(data) Ignored 43029 out of range events
[#0]
Further, I get:itCB = 0, fitCB && raw =1, fitMethod2 = 3, rawOnly =5,
enter method number:
0
enter rootfile name to open in directory -> /home/fsoomro/roofitProblem/
/afs/cern.ch/user/p/pcanal/user_code/roofitProblem/temp_ntuples/
Will be opening root file /home/fsoomro/roofitProblem//afs/cern.ch/user/p/pcanal/user_code/roofitProblem/temp_ntuples/
stage_get: Unable to find a value for STAGE_HOST.
Please check castor.conf and/or your environment
Error in <TCastorFile::FindServerAndPath>: stage_open failed: Host not known ()
Error in <TCastorFile::Create>: error opening file
This command opens and write a root file in the said directory…
I have changed the relevant lines in the .cpp file and now if should work with the full path as you have entered…
I hope not, but if you see such problem(s) again, please check for the relevant cout in the code, since they are just due to wrong paths, since I used the code on another machine and not lxplus
As it is, I can not reproduce the problem. The main difference is likely to be that you are reading the file from castor and I did not. I would recommend copying those files out of castor before doing the analysis or to upgrade to the latest version of ROOT and the castor client.
This is apparently one of the many leaks in your example. As it is, the process uses more than 5Gb of memory. It is unlikely that you actually need to duplicate (in memory) (almost) the whole chain in order to do an analysis. I strongly recommend you review the RooFit documentation and example to find an alternative (and/or request help on the Math and Stats forum).
The memory use means that valgrind can not be used (it would takes more than 28Gb to run!) to look at your issue (which is very likely to be using a deleted and/or misallocated TTree).
The error I report occurs ‘before’ the fit is made if you follow the output.
So it happens ‘before’ the end of the block of code which creates the tree, i.e where I should delete it.
The way I managed to use this same code i.e. making a new tree in each iteration was to ‘trim’ my initial ntuple. It had many leaves which were not used in this analysis, so I wrote only the few relevant leaves into another root file and supplied that to this code, that seems to work. So I think the problem is that the large ntuple could not be accomodated in the memory, but there must be a way of loading large ntuples into root and fitting.
p.s. I agree that I do not need to make a new tree, I could create the RooDataSet using the initial tree, just applying some cuts. I will try that and see if it changes anything. The above finding seems to suggest that it will not.
[quote] So I think the problem is that the large ntuple could not be accomodated in the memory, but there must be a way of loading large ntuples into root and fitting.[/quote]There is usually no need to load a TTree into memory (except for ‘performance’ reason when reading a TTree many times in random order … which is not your case) as all the tools work as well with a disk based TTree than a in-memory TTree.