How to implement schema evolution properly?

Hi,

I’m having some issues properly implementing schema evolution in my ROOT classes.

Suppose I have a class that I have already persisted to a ROOT file. I then add one or more data members to the header file, and increment the ClassDef version.

Now when I go to read in my old data with the new version of this interface, I get low-level seg faults in the branch/tree code within ROOT, implying that my schema evolution has not worked. This is presumably because ROOT is doing some block memory copying based on the old size/layout of the class, but it has changed in the new version.

So how can I implement this properly such that I can read my old data with a new version of an interface that adds one or more new data members? Is this covered explicitly in the ROOT manual someplace? I spotted some sections having to do with streamers in the I/O manual which looked relevant, but it was hard to tell exactly what I would need to do in order to make my classes backward compatible.

If more concrete examples are needed of my ROOT classes, I can provide them.

Thanks.

Hi,

The magic is with the default constructor:

ROOT initializes the new object using the default constructor (or the io constructor, but I suppose you don’t have that).

Then it reads data from file, setting all those data members that the file contains.

That leaves your new member untouched and thus default initialized.

If you don’t initialize that member (say a pointer member), then the destructor will try to delete invalid memory. Or maybe you initialize it to 0, but your code doesn’t handle the case.

If this doesn’t help then we will need:

  • the error message / back trace,
  • the new classes definition, calling out which member is new,
  • ideally the ROOT file

Cheers, Axel

Thanks for the reply.

Things seem to be mysteriously working better now. Apparently posting to the ROOT forums fixes bugs in my code!

I checked that I can generate data using the new version of the class and read it in using an application compiled using the previous version.

I get a warning message like this.

Info in <TBranchElement::InitializeOffsets>: TTree created with an older schema, 
some data might not be copied in 'slow-cloning' mode; fast-cloning should have 
the correct result. 'processType_' is missing when constructing the branch   
'SimParticles_sim'.

I think it looks okay in rootbrowse though.

Is it anything to worry about or ignore-able?

Hmm this info message seems to be the wrong way around: I would have expected the warning to say “there is a new member that I am not creating a branch for”, instead of “processType_ is missing”.

@pcanal, could you explain, please?

Axel.

Hi,

This happens (at least this is the usual/expected case), when calling CloneTree on the original tree. In this scenario, the request is to make a faithful/exact copy of the original TTree. Usually this indicates that a data member (processType_) that used to be in the schema (in the old file) is no longer there. Slow-copying requires to load the data in memory but there is no space in memory to store the old data member and thus it can not be copied.

Now the odd part is that this is the opposite of what you said " then add one or more data members to the header file ".

So something is unusual. Can you be more specific in the sequence of operation/code that lead to the message?

Cheers,
Philippe.

This happens (at least this is the usual/expected case), when calling CloneTree on the original tree.

Okay, I think this is probably what happened. I may have confused myself between the two different versions of my code, and the two different data file versions. When I now use my current version of the classes with a file which does not have the added data member, things seem to work fine and I do not get any errors or warnings.

Thanks for the clarification.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.