Crash on write due to changed read buffer?

My application reads an object contained in a branch of a tree in a Root file and writes the same object in a tree inside an output file. I am seeing a crash, which I think I have tracked down to a change in the readout buffer of a field of my object.
To be more precise, the object has two TClonesArray* fields, call them A and B. The objects read from the input file have both A and B pointers non-null, but B has 0 elements (the file has been filled with objects made like this, I have no control over it). I read the objects as usual, i.e. I set the branch address for readout and thus A and B are set to point to some memory areas where the data read from file is put. I use the same buffer as the buffer for output.
When I read the first entry, A and B are set to a certain value, I can smoothly save the object on the output file and then go on with the second entry. Here I notice that while A stays the same (i.e. it points to the same memory area) the value of B changes: I interpret this as Root allocating a new memory area for the readout buffer and updating the related pointer:

Read entry 0: B = 0x1026db0, A = 0xf1c5e0
Read entry 1: B = 0xf4d3b0, A = 0xf1c5e0

The event is read witout problems, but now trying to save the object in the output file result in a crash. Using a debugger I have been able to see that the crash happens when the branch corresponding to B is written. the crash happens in TBranchElement.cxx at line 1604:

char **arr = (char **)clones->GetObjectRef(0);

where clones still is 0x1026db0 albeit the buffer has been moved from 0x1026db0 to 0xf4d3b0 (if I understand correctly).

So I think that my crash is due to this relocation of the input buffer. I investigated the Root code and I think I found the reason of this relocation. In TStreamerInfoReadBuffer.cxx at line 1074 I read:

  switch (kase) {

         case TStreamerInfo::kAnyp:    // Class*  not derived from TObject with    comment field //->
         case TStreamerInfo::kAnyp+TStreamerInfo::kOffsetL:
         case TStreamerInfo::kObjectp: // Class*      derived from TObject with    comment field  //->
         case TStreamerInfo::kObjectp+TStreamerInfo::kOffsetL:
            isPreAlloc = 1;
            // Intentional fallthrough now that isPreAlloc is set.
         case TStreamerInfo::kObjectP: // Class* derived from TObject with no comment field NOTE: Re-added by Phil
         case TStreamerInfo::kObjectP+TStreamerInfo::kOffsetL:
         case TStreamerInfo::kAnyP:    // Class*  not derived from TObject with no comment field NOTE:: Re-added by Phil
         case TStreamerInfo::kAnyP+TStreamerInfo::kOffsetL: {
            DOLOOP {
               b.ReadFastArray((void**)(arr[k]+ioffset),cle,compinfo[i]->fLength,isPreAlloc,pstreamer);
            }
         }

When reading B, kase has value 64, i.e. TStreamerInfo::kObjectP. According to the comment, this corresponds to a class without a comment field, and this is how B is actually looking in the input file:


Br 25 :B : TClonesArray *
*Entries : 50 : Total Size= 4452 bytes File Size = 197 *
*Baskets : 1 : Basket Size= 32000 bytes Compression= 20.15 *

I’m not sure about this, but I think that this could be because B has always 0 entries. Anyway, inside the case the variable isPreAlloc is not set to 1 which results in a reallocation of the readout buffer inside TBufferFile::ReadFastArray.

Does this make sense? I am using Root 5.34.20. Thanks

I forgot to mention a very important thing: I read the first entry before setting the branch address for output. The flow of my program is:

  1. open input file, create input buffer object called inBuf, set branch address for input branch
  2. read first entry
  3. open output file, create output tree, create output branch using inBuf as output buffer
  4. save first entry
  5. read second entry
  6. save second entry (crash)

if I create the output branch before reading the first entry i.e. switching 2. and 3. then I see no crash. I think that the problem is that in the sequence above inBuf.B is non-NULL at the moment of creating the output branch, while in the modified sequence it is NULL.
Am I correct?

Hi Nicola,

How do you set the branch address? What is the complete output of tree->Print()?

Cheers,
Philippe.

Hi Philippe, I set the branch address for input in this way:

Actually I’m using a TChain for input, but crash happens also when the chain is made by a single file.
The output branch is created in this way:

Note that, as I said in my second post, the first entry from input file has already been read, so the TClonesArray pointers in event are non-NULL when I create the branch in the output tree.
Below is the result of inputTree->Print. Note that in my original post I simplified the names of the TClonesArrays for the sake of clarity: below, A is fEDepCHD (branch 3) and B is fEDepW (branch 25):

[code]root [2] T->Print()


*Tree :T : sim_data *
*Entries : 50 : Total = 1017414 bytes File Size = 237384 *

  •    :          : Tree compression factor =   4.27                       *
    

*Branch :ev *
*Entries : 50 : BranchElement (see below) *

*Br 0 :fUniqueID : UInt_t *
*Entries : 50 : Total Size= 767 bytes File Size = 94 *
*Baskets : 1 : Basket Size= 32000 bytes Compression= 2.90 *

*Br 1 :fBits : UInt_t *
*Entries : 50 : Total Size= 955 bytes File Size = 208 *
*Baskets : 1 : Basket Size= 32000 bytes Compression= 2.29 *

Br 2 :fInci : TIncidentParticle *
*Entries : 50 : Total Size= 9155 bytes File Size = 2787 *
*Baskets : 1 : Basket Size= 32000 bytes Compression= 3.11 *

Br 3 :fPEDepCHD : TClonesArray *
*Entries : 50 : Total Size= 767 bytes File Size = 94 *
*Baskets : 1 : Basket Size= 32000 bytes Compression= 2.90 *

*Br 4 :fEDepCHD : Int_t fEDepCHD_ *
*Entries : 50 : Total Size= 4791 bytes File Size = 171 *
*Baskets : 1 : Basket Size= 32000 bytes Compression= 1.59 *

*Br 5 :fEDepCHD.fUniqueID : UInt_t fUniqueID[fEDepCHD_] *
*Entries : 50 : Total Size= 3306 bytes File Size = 254 *
*Baskets : 1 : Basket Size= 32000 bytes Compression= 10.61 *

*Br 6 :fEDepCHD.fBits : UInt_t fBits[fEDepCHD_] *
*Entries : 50 : Total Size= 3286 bytes File Size = 255 *
*Baskets : 1 : Basket Size= 32000 bytes Compression= 10.55 *

*Br 7 :fEDepCHD.fID : Int_t fID[fEDepCHD_] *
*Entries : 50 : Total Size= 3276 bytes File Size = 726 *
*Baskets : 1 : Basket Size= 32000 bytes Compression= 3.70 *

*Br 8 :fEDepCHD.fEDep : Float_t fEDep[fEDepCHD_] *
*Entries : 50 : Total Size= 3286 bytes File Size = 2489 *
*Baskets : 1 : Basket Size= 32000 bytes Compression= 1.08 *

Br 9 :fEDepCHD.fSubEDep : TClonesArray fSubEDep[fEDepCHD_] *
*Entries : 50 : Total Size= 3301 bytes File Size = 252 *
*Baskets : 1 : Basket Size= 32000 bytes Compression= 10.69 *

*Br 10 :fEDepCHD.fAuxEDep : Float_t fAuxEDep[fEDepCHD_] *
*Entries : 50 : Total Size= 3301 bytes File Size = 252 *
*Baskets : 1 : Basket Size= 32000 bytes Compression= 10.69 *

*Br 11 :fEDepIMC : Int_t fEDepIMC_ *
*Entries : 50 : Total Size= 4791 bytes File Size = 226 *
*Baskets : 1 : Basket Size= 32000 bytes Compression= 1.20 *

*Br 12 :fEDepIMC.fUniqueID : UInt_t fUniqueID[fEDepIMC_] *
*Entries : 50 : Total Size= 135334 bytes File Size = 1376 *
*Baskets : 5 : Basket Size= 32000 bytes Compression= 97.91 *

*Br 13 :fEDepIMC.fBits : UInt_t fBits[fEDepIMC_] *
*Entries : 50 : Total Size= 135298 bytes File Size = 1443 *
*Baskets : 5 : Basket Size= 32000 bytes Compression= 93.35 *

*Br 14 :fEDepIMC.fID : Int_t fID[fEDepIMC_] *
*Entries : 50 : Total Size= 135280 bytes File Size = 81822 *
*Baskets : 5 : Basket Size= 32000 bytes Compression= 1.65 *

*Br 15 :fEDepIMC.fEDep : Float_t fEDep[fEDepIMC_] *
*Entries : 50 : Total Size= 135298 bytes File Size = 110960 *
*Baskets : 5 : Basket Size= 32000 bytes Compression= 1.21 *

Br 16 :fEDepIMC.fSubEDep : TClonesArray fSubEDep[fEDepIMC_] *
*Entries : 50 : Total Size= 135325 bytes File Size = 1371 *
*Baskets : 5 : Basket Size= 32000 bytes Compression= 98.26 *

*Br 17 :fEDepIMC.fAuxEDep : Float_t fAuxEDep[fEDepIMC_] *
*Entries : 50 : Total Size= 135325 bytes File Size = 1371 *
*Baskets : 5 : Basket Size= 32000 bytes Compression= 98.26 *

*Br 18 :fEDepTASC : Int_t fEDepTASC_ *
*Entries : 50 : Total Size= 4822 bytes File Size = 189 *
*Baskets : 1 : Basket Size= 32000 bytes Compression= 1.44 *

*Br 19 :fEDepTASC.fUniqueID : UInt_t fUniqueID[fEDepTASC_] *
*Entries : 50 : Total Size= 27653 bytes File Size = 402 *
*Baskets : 1 : Basket Size= 32000 bytes Compression= 67.25 *

*Br 20 :fEDepTASC.fBits : UInt_t fBits[fEDepTASC_] *
*Entries : 50 : Total Size= 27633 bytes File Size = 422 *
*Baskets : 1 : Basket Size= 32000 bytes Compression= 64.05 *

*Br 21 :fEDepTASC.fID : Int_t fID[fEDepTASC_] *
*Entries : 50 : Total Size= 27623 bytes File Size = 2445 *
*Baskets : 1 : Basket Size= 32000 bytes Compression= 11.05 *

*Br 22 :fEDepTASC.fEDep : Float_t fEDep[fEDepTASC_] *
*Entries : 50 : Total Size= 27633 bytes File Size = 23620 *
*Baskets : 1 : Basket Size= 32000 bytes Compression= 1.14 *

Br 23 :fEDepTASC.fSubEDep : TClonesArray fSubEDep[fEDepTASC_] *
*Entries : 50 : Total Size= 27648 bytes File Size = 402 *
*Baskets : 1 : Basket Size= 32000 bytes Compression= 67.25 *

*Br 24 :fEDepTASC.fAuxEDep : Float_t fAuxEDep[fEDepTASC_] *
*Entries : 50 : Total Size= 27648 bytes File Size = 402 *
*Baskets : 1 : Basket Size= 32000 bytes Compression= 67.25 *

Br 25 :fEDepW : TClonesArray *
*Entries : 50 : Total Size= 4452 bytes File Size = 197 *
*Baskets : 1 : Basket Size= 32000 bytes Compression= 20.15 *
[/code]

I want to say that I already fixed my code by creating the output branch before reading the first entry of the input tree. Nevertheless, if I actually hit a bug I’m available for some additional testing if needed.

Thanks for your help.

Hi,

On a separate note, there is usually no good reason to hold the TClonesArray by pointer. (If you can change the data model), holding the TClonesArray by value will increase performance in couple of ways. One it will obviously avoid/prevent the deallocation/reallocation of the TClonesArray that you observe ; The main reason to use a TClonesArray is to encourage/benefit from memory re-use, deleting the TClonesArray defeats that purpose. In addition, if the TClonesArray is held by value, the TTree will be able to split it and increase read/write performance and compression factor.

Cheers,
Philippe.

I used TClonesArray* because sometimes they are not needed/available so I set the pointer to NULL, avoiding the storage of empty arrays. But maybe the advantage in terms of saved disk space is much less than the advantages you mentioned when holding TClonesArray by value. Thanks for the info, maybe it’s too late to change my data model but at the next chance of breakage I’ll try to get rid of these pointers.

[quote]the advantage in terms of saved disk space[/quote]When stored in a TTree there is virtually no disk space difference between storing the empty TClonesArray and storing the null pointer (both will store essentially just store the value 0).

Cheers,
Philippe.