Hi! I would like check validity/integrity of a TFile content
I was under the impression that as long not file.IsZombie() and moreover even not file.TestBit(ROOT.TFile.kRecovered) then the file is just fine …
well, i encountered the situation when all these are fine, but reading some entries from some TTrees from file i got
Error in unzip ZSTD. Type = Corrupted block detected . Code = 18446744073709551596
Error in <TBasket::ReadBasketBuffers>: fNbytes = 6787763, fKeylen = 72, fObjlen = 7622608, noutot = 0, nout=0, nin=6787691, nbuf=7622608
Error in <TBranch::GetBasket>: File: broken_AO2D.root at byte:240631153, branch:fZ, entry:0, badread=1, nerrors=1, basketnumber=0
Error in unzip ZSTD. Type = Corrupted block detected . Code = 18446744073709551596
Error in <TBasket::ReadBasketBuffers>: fNbytes = 6787763, fKeylen = 72, fObjlen = 7622608, noutot = 0, nout=0, nin=6787691, nbuf=7622608
Error in <TBranch::GetBasket>: File: broken_AO2D.root at byte:240631153, branch:fZ, entry:1905651, badread=1, nerrors=2, basketnumber=0
Moreover, this output did not throw any error (GetEntry is in try: section and nothing was thrown)
So, is there an way to overall check the health of the file? or the health of it’s content?
Thank you!
In order to catch those error (besides inserting your own custom ‘error message handler’), you can look at the result of GetEntry on the individual branches (we ought to be 0 or negative in those case).
yes, i did this, and while i avoided parsing the whole tree (i just read 0 and (GetEntries - 1) and check for -1) it feels that it takes a little bit of time …
while this works, it seems weird that there is no “unhealthy” status of TFile, while it’s content it is corrupted…
Since it is enough for a bit to be flipped somewhere in the middle of the file, there is no way to know it is corrupted without reading it entirely or at least reading the part that you are interested in.
One option that is sometimes used is to record along side the TFile the result of running md5sum (or similar tool) on the file and then at read time (at the cost of md5sum scanning the whole file) one can check whether the file has been corrupted [Of course, this does not detect the cases where the file was incorrectly written in the first place]
So, the file is check-summed at write and read (to and from grid, md5) so this segment of data transfer should be ok…
But i was wondering if a backward compatible solution can be added for registering a hash for each basket in a TTree or per object in TFile? so with a tfile.fsck() one could know what objects are fine ? or something similar … with the increasing of stored data and the size of chunks of data, maybe it would be useful a more granular and earlier (as in when the object is flushed to tfile) approach?