Continuing to write to a large ROOT binary after it recovers from being closed improperly

[jwruss]

I am writing a series of 255 files, each with a TTree containing at least 377625 entires in them. When creating each file’s TTree, I set TTree::SetAutoSave(377625 / 10). I write these files on a cluster where occasionally they the are closed improperly before they can finish writing. In these instances when I open one of these improperly closed files, recovery mode is enabled and the files are recovered up to the last cycle from which they were written. I would like resume writing to them through a macro, and was led to believe this could be done with something like

	if (!fullSumFile.GetNkeys()) {  //  Check that at least keys are present in the tree.

		fullSumTree = new TTree("treeName", "treeName");
		fullSumTree -> SetAutoSave(partEventsStep / 10);
		fullSumTree -> Branch("summary", & fullSum);

		entryCount = 0;

	} else {

		fullSumFile.Write();  //  Necessary after recovery.

		fullSumTree = (TTree *) fullSumFile.Get("fullSumTree");
		fullSumTree -> SetBranchAddress("summary", & fullSum);

		entryCount = fullSumTree -> GetEntries();
	}

where before the else statement is in case the files close before a cycle can be written. The files written to are always opened in “update” mode and the use of TFile::Write() comes from https://root-forum.cern.ch/t/tfile-recovery-does-not-appear-to-be-working/16579. It may be necessary to then call after TFile::Close(), then TFile::Open() within the else statement, because after opening a file that has improperly closed more than once I get

Error in <TKey::ReadObjWithBuffer>: Unknown class 

and the files close with nothing being apparently recovered. How do I modify the above if/else statement to handle the recovery properly and continue writing to the files?

[jwruss]

If it is a case of that the recovered files have to be closed and then reopened, I am having difficulty reopening the file within the above else statement:

} else {

    fullSumFile.Write();
    fullSumFile.Close();
    fullSumFile.Open("fileName.root", "update");
}

Once fullSumFile.Close() is called, using fullSumFile.Open("fileName.root") doesn’t appear to behave as expected, reopening the file to be updated? From ROOT terminal, I would expect that between the lines

TFile fileName("fileName.root");
fileName.Close();
fileName.Open("fileName.root");
fileName.Close();

I would expect the file to become inaccessible after the second instance of fileName.Close() is called, like after the first instance.

[jwruss]

Again referring to this previous topic, I’m now trying something like this:

TFile recoverFullSumFile(fullSumPath, "update");
recoverFullSumFile.Write();
recoverFullSumFile.Close();

TFile fullSumFile(fullSumPath, "update");

TTree * fullSumTree = 0;
summaryClass * fullSum = 0;
unsigned int entryCount;

if (!fullSumFile.GetNkeys()) {

    fullSumTree = new TTree("treeName", "treeName");
    fullSumTree -> SetAutoSave(377625 / 10);
    fullSumTree -> Branch("summary", & fullSum);

    entryCount = 0;

} else {

    fullSumTree = (TTree *) fullSumFile.Get("treeName");
    fullSumTree -> SetBranchAddress("summary", & fullSum);

    entryCount = fullSumTree -> GetEntries();
}

I think this should work. But if not, why? Calling the same file twice like this, should I replace recoverFullSumFile.Close() with recoverFullSumFile.Close("R")?

[bellenot]

I think this is something for @pcanal, our I/O expert

[pcanal]

fullSumFile.Open("fileName.root", "update");

TFile::Open is a static function that return the address of a newly allocated TFile object. You probably meant to use

  fullSumFile.ReOpen("fileName.root", "update");

[jwruss]

Hello Phillipe,

So just to clarify, rather than using

TFile recoverFullSumFile(fullSumPath, "update");
recoverFullSumFile.Write();
recoverFullSumFile.Close();

TFile fullSumFile(fullSumPath, "update");

before the if-else statement in my first post, are you saying that it is sufficient to do something like

TFile fullSumFile(fullSumPath, "update");
fullSumFile.Write();
fullSumFile.Close();
fullSumFile.ReOpeN(fullSumPath,"update");

or like

TFile fullSumFile(fullSumPath, "update");
fullSumFile.Write();
fullSumFile.ReOpen(fullSumPath, "update");

?

I had tried using TFile::ReOpen() before, but I just tried something like

TFile fullSumFile(fullSumPath);
fullSumFile.ReOpen(fullSumPath, "update");

after the file had improperly closed beforehand.

[pcanal]

TFile fullSumFile(fullSumPath, "update");
fullSumFile.Write();
fullSumFile.Close();
fullSumFile.ReOpen(fullSumPath,"update");

is indeed what I meant

Cheers,
Philippe.

[jwruss]

Thanks, Philippe.

So just to test, with a ROOT binary file that I will call “testFile.root” here, I entered the lines following lines in the ROOT console

TFile testFile("testFile.root","update");
testFile.Write();
testFile.Close();
testFile.ReOpen("testFile.root","update");

and I think I should mention what I found.

1. After entering the line TFile testFile("testFile.root","update");, I am able to list file contents by entering .ls into the console, including KEYs, up until I call testFile.Close();. After this line, I get no resulting output in the terminal when I enter .ls.

2. I guess the last line should be replaced with testFile.ReOpen("update");?

3. After entering in the last line, all that is listed from the the file are its TFile** and TFile*, but none of its KEY entries. Without these KEY entries, I am unable to get pointers like I could before entering testFile.Close().

This suggests to me that perhaps TFile::ReOpen() isn’t behaving as intended (ROOT version 6.14/02), or perhaps what I want is

TFile fullSumFile(fullSumPath, "update");
fullSumFile.Write();
fullSumFile.ReOpen("update");

However, if I’m remembering correctly from when I did something before like these three lines, I think I still got a warning in the ROOT console that the file has to recover, as if TFile::Close() wasn’t implicitly called with TFile::ReOpen(). Is this the expected behavior?