ROOT files not properly closed till after program exit

ROOT
1

ROOT Version: 5.34.36
Platform: CentOS 7
Compiler: gcc4.9.3

This question might be related to my previous post titled “Naive question about TFile::GetSize()” of April 26.

I am running a data acquisition program that works like this:

- open file n.0
- write 10000 events to it
- close file n.0
- open file n.1
- write 10000 events to it
- close file n.1
.etc.etc.etc.

Besides the problem of getting the file size after closing each file, reported in my previous post, I found that if I try to read with an external program one of the closed files while the acquisition program is still running, I get a Warning message:

Warning in TFile::Init: file run_22_001.root probably not closed, trying to recover

If I wait till the acquisition program has exited, then the Warning from the external program disappears.

The actual code used to open and close the file is the following:

fTFileHandle = new TFile();
// First file
fTFileHandle->Open("run_22_000","NEW","PADME Merged Raw Events");
fTTreeMain = new TTree("RawEvents","PADME Raw Events Tree");
fTTreeMain->Branch("RawEvent",&fTRawEvent);
...write events...
fTTreeMain->Write();
fTFileHandle->Close();
delete fTTreeMain;
// Second file
fTFileHandle->Open("run_22_001","NEW","PADME Merged Raw Events");
fTTreeMain = new TTree("RawEvents","PADME Raw Events Tree");
...etc...etc...and finally...
fTTreeMain->Write();
fTFileHandle->Close();
delete fTTreeMain;
// When acquisition program exits
delete fTFileHandle;

Are there obvious errors in the procedure I am using?

In particular, is it correct to create the TFile at the beginning (fTFileHandle) and to reuse it for all the files created during the run?

Thank you

Emanuele Leonardi

2

For each “new_file.root”, use:

fTFileHandle = TFile::Open("new_file.root", "RECREATE", "PADME Merged Raw Events"); // "RECREATE" or "NEW"
if ((!fTFileHandle) || fTFileHandle->IsZombie()) { delete fTFileHandle; return; } // just a precaution
fTTreeMain = new TTree("RawEvents", "PADME Raw Events Tree");
// ...
fTTreeMain->Write();
delete fTFileHandle; // automatically deletes "fTTreeMain", too
3

To support this, you need to follow the doc for TDirectoryFile::ReadKeys:

/// Read the linked list of keys.
///
/// Every directory has a linked list (fKeys). This linked list has been
/// written on the file via WriteKeys as a single data record.
///
/// It is interesting to call this function in the following situation.
/// Assume another process1 is connecting this directory in Update mode
///   - Process1 is adding/updating objects in this directory
///   - You want to see the latest status from process1.
/// Example Process1:
/// ~~~{.cpp}
/// obj1.Write();
/// obj2.Write();
/// gDirectory->SaveSelf();
/// ~~~
///
/// Example Process2:
/// ~~~{.cpp}
/// gDirectory->ReadKeys();
/// obj1->Draw();
/// ~~~
/// This is an efficient way (without opening/closing files) to view
/// the latest updates of a file being modified by another process
/// as it is typically the case in a data acquisition system.

where you can replace gDirectory with the pointer to the TFile.

4

Looking at the original post, I cannot see how the TDirectoryFile::ReadKeys would help (there is just one tree which is written and then the file is closed). Moreover, the original description explicitly claims that the warning appears for an already closed file (i.e. when a new / next file is actually being opened / used for writing).

5

I was responding specific to the part:

Where I guess I skimmed over (and thus ignored) the par “one of the closed files” :slight_smile:

6

The other missing part (in addition to missing the delete of the TFile) in the original question is:

fTFileHandle->Write();
7

I think another problem in the original code is the delete fTTreeMain; (I guess it generates a double delete).

Anyhow, in my first post above, I tried to show a “clean” way of dealing with all these problems (no need for any additional fTFileHandle->Write(); in my example, I believe).

8

(no need for any additional fTFileHandle->Write()

You are right. I mis-remembered the limit of Close (which is called by delete); in this case it is sufficient.

(I guess it generates a double delete).

No double delete. thanks to the implicit memory management mechanism. Upon seeing the deletion of the TTree, the TFile will be informed (via the ListOfCleanups and RecursiveRemove) that the TTree is deleted and will ‘forget’ about it. The original code is indeed missing the deletion of the TFile … without it there is both a memory leak and an incremental slow down (as the list of TFile grows).

So after reading the post again, the problem is indeed about concurrent read and write access to the same file (and/or I can not reproduce the problem).

Besides the problem of getting the file size after closing each file, reported in my previous post, I found that if I try to read with an external program one of the closed files while the acquisition program is still running
…
Warning in TFile::Init: file run_22_001.root probably not closed, trying to recover
…
If I wait till the acquisition program has exited, then the Warning from the external program disappears.

I assume that this is because the file run_22_001.root already exist (TFile::Open was called) but the ‘Close’ operation has not been run or has not completed yet.

Right … I just re-read the code and the see crux of the problem (which Wile correctly understood but did not underscore). The original code did:

fTFileHandle = new TFile();
fTFileHandle->Open("run_22_000","NEW","PADME Merged Raw Events");

However TFile::Open is a static function and calling it on a TFile pointer does not affect the object this pointer points to. Consequently when the execution gets to:

fTFileHandle->Close();

The file that was just opened is not closed and consequently it will not be closed until the very end of the process … hence the behavior you describe.

Not that Wile’s solution will mostly solve the problem. It is still possible that the reader will attempt to read the file after it was created but before the Close function is called (i.e. during the TTree filling). To make the code studier you would need to do:

fTFileHandle = TFile::Open("new_file.root", "RECREATE", "PADME Merged Raw Events"); // "RECREATE" or "NEW"
if ((!fTFileHandle) || fTFileHandle->IsZombie()) { delete fTFileHandle; return; } // just a precaution

fTFileHandle->SaveSelf();  // Insure that the file on disk is in a good state.

fTTreeMain = new TTree("RawEvents", "PADME Raw Events Tree");
// ...
fTTreeMain->Write();`
delete fTFileHandle; // automatically deletes "fTTreeMain", too

and in the reader, you need to do something

fTFileHandle = TFile::Open("new_file.root", "READ");
TTree *tree;  fTFileHandle->GetObject("RawEvents", tree);

if ( tree == null) {
    printf("TTree is not yet there, maybe we need to try later\n");\
    return
}
9

Well, I missed the fact that, in the original source code, instead of fTFileHandle=TFile::Open(..);, there is fTFileHandle->Open(...); (and so the fTFileHandle is not assigned at all).

Just for the reference … the double delete problem, that I mentioned, would be the result of the existing sequence fTTreeMain->Write(); fTFileHandle->Close(); delete fTTreeMain;, while Philippe (correctly) states that everything will be fine when one uses the sequence fTTreeMain->Write(); delete fTTreeMain; fTFileHandle->Close(); (note that the explicit delete fTTreeMain; call is completely redundant in this case).

10

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.