TTree::SetAutoSave, does it still work? (5.34.05)

ROOT
1

Dear Rooters,

We are running a DAQ program which writes root files typically
of size 300 Mb. The file has 5 trees, one of them (“readout0”)
containing most of the data. SetAutoSave(10000000) (10 Mbyte) was
called for all of the trees.
Unfortunately the DAQ crashed due to hardware problems in VME for
a few runs leaving files with sizes upto 103 Mbyte.
Despite of the above SetAutoSave no keys could be recovered.

In the attachment (see below) I add diagnostic output from tree->Dump() for
a good run (241raw.root) which indicates that fAutoSave = 9976250
was really set to the required value. Still the complete tree
is contained in one key as f.ls() indicates. I expected O(30)
and fSavedBytes = 0

In addition I add diagnostic output for a bad file(231raw.root).
Not much to see of course apart from sizes and program version (5.34.05)

I looked at similiar data from 2009 (Root V 5.23.02), written with
essentially the same program. There I see as expected:

fTotBytes 1498708707 Total number … before compression
fZipBytes 113642006 Total number … after compression
fSavedBytes 1496267805 Number of autosaved bytes
…
fAutoSave 10000000 Autosave tree when fAutoSave bytes produced

We are concerned about 2 things:

  • How can we make the AutoSave mechanism work in future.

  • Is there any chance to recover some of the bad data.
    In the past Rene indicated something in this direction:
    How to read a Zombie file?
    “This will require some work (not possible in the coming days), but
    it is doable.”

Cheers
Otto

I am unable to add an attachment (any “extension not allowed”)
therefore I include it here:

TFile f(“241raw.root”); // file size 317Mbyte
root [1] f.ls()
TFile** 241raw.root Sidet_Mon-ERD-Configuration
TFile* 241raw.root Sidet_Mon-ERD-Configuration
KEY: TTree Start;1 Start Acquisition
KEY: TTree xstop;1 stop acquisition, trigger 15, user-defined
KEY: TTree DeadTime;1 Dead Time
KEY: TTree readout0;1 readout of branch0 mon
KEY: TTree readout1;1 readout of branch1 sidet
KEY: TTree readout2;1 readout of branch0 current

readout0->Dump()

fEntries 935441 Number of entries
fTotBytes 5123607640 Total number … before compression
fZipBytes 325049919 Total number … after compression
fSavedBytes 0 Number of autosaved bytes
fFlushedBytes 305210266 Number of autoflushed bytes
…
fAutoSave 9976250 Autosave tree when fAutoSave bytes produced
fAutoFlush 79810 Autoflush tree when fAutoFlush entries written
…

TFile f(“231raw.root”); // file size 103Mbyte
Warning in TFile::Init: file 231raw.root probably not closed, trying to recover
Warning in TFile::Init: no keys recovered, file has been made a Zombie

f.ls()
TFile** 231raw.root Sidet_Mon-ERD-Configuration
TFile* 231raw.root Sidet_Mon-ERD-Configuration
root [2] f.Dump()
==> Dumping object at: 0x00000000010d8570, name=231raw.root, class=TFile
…
fBytesRead 15334700 Number of bytes read from this file
…
fBEGIN 100 First used byte in file
fEND 107498518 Last used byte in file
…
fVersion 53405 File format version
…
fRealName 231raw.root Effective real file name (not original url)

2

Hi,

[quote]SetAutoSave(10000000) (10 Mbyte) was
called for all of the trees.[/quote]This actually request the autosaving after 10 millions entries … try with a negative number.

Cheers,
Philippe.

  • Is there any chance to recover some of the bad data.

It would require quite a bit of work/time … that I am unlikely to be able to find.

3

SetAutoSave requires a negative number? This is not indicated in the documentation: http://root.cern.ch/root/html/TTree.html#TTree:SetAutoSave

A positive number forces autosave after a number of entries and a negative number after a number of bytes? How is SetAutoSave different from SetAutoFlush?

4

Hi,

Indeed the documentation of SetAutoSave needs to be updated. It takes the same ‘type’ of argument as SetAutoFlush. Either a positive number, in which case it is a number of entries, or a negative number, in which case the absolute value is in (rough) number of compress bytes between autosave.

[quote]A positive number forces autosave after a number of entries and a negative number after a number of bytes?[/quote]yes, more precisely, when negative, the number of entries between each autosave is determine based on the number of entries it takes to reach the first so many bytes of data.

[quote] How is SetAutoSave different from SetAutoFlush?[/quote]AutoSave control how often the TTree meta data (the content of the TTree object itself) is stored. If the process crash, you usually can only retrieve from the file the entries that were stored before (up to) the last AutoSave.
AutoFlush control when the basket (i.e. the data) are flushed to disk and control the granularity to which you can read the data in a single large read. (i.e. if you read last than that number of entries you may end up with more individual reads).

Cheers,
Philippe.

5

The documentation has been updated to reflect the fact that fAutoSave can indeed have negative (number of bytes) or positive (number of entries) values.

Thanks,
Philippe