I am finding myself in need of writing a custom streamer in order to keep backward compatibility of a custom class of which i need to do I/O.
But i am struggling to gather the data for such a class, I am implementing the Streamer(TBuffer &R__b) method for the object and I wanted. The first variable that is written in the .hh of the class is an int, so in the streamer i am doing:
R__b.ReadInt(fNumber);
but this appears to read a random value and not the correct one.
Also I will need to read several std::vectors of which i don’t know the size, how can i approach the problem?
I’d like to understand better why you are forced to opt for a custom streamer if you agree. This is something that in general we strongly suggest to avoid and opt for the standard creation of dictionaries to support I/O.
Hi @Danilo,
So I have this class that previously stored a map<int,CustomClass2>, where CustomClass2 is another class that contains an enumerator and other basic c++ types.
I needed to do an upgrade and in order to use polymorphism on CustomClass2 I had now to store a map<int,CustomClass2*>.
The I/O with this upgrade seems to work fine but when I try to read files produced before the upgrade the I/O breaks in particular the TStreamer complains.
I already produce rootmaps and dictionaries for my classes, but I am opened to suggestions (I also updated the version of the class, in ClassDef )
Thanks,
Giorgio
What you did is a “schema evolution” of your class: you can read more about that here.
I understand the streamer related issue is generated by CustomClass2, and not map<int, CustomClass2*>, for which you have a proper dictionary.
If I am wrong, could you proceed with the dictionary generation for the aforementioned container and check if anything is broken?
If I am right, my proposal would be that you go through that information and try to see if the upgrade you implemented is falling in the cases treated by ROOT’s automatic schema evolution. Otherwise, we are left with the implementation of a IO rule.
@Danilo
Thank you! So the CustomClass2 I/O seems to work, what really seems to break is the map made of pointers. Here is the error of when I try to read a preupgrade implementation of the class from a root file:
Warning in <TStreamerInfo::BuildOld>: Cannot convert pair<int,CustomClass2>::second from type: CustomClass2 to type: CustomClass2, skip element
But how do i check which one of your hypotheses is correct? It’s my first time diving in this type of stuff and I am unsure of the steps to take.
Best,
Giorgio
A way to isolate this issue is to have a clean example code, write one single instance of the old CustomClass2 in a clean rootfile and then re-read it with the new code.
From the error type is a schema evolution issue. Therefore, if you have custom streamer, you need to check the class version, and, if it is the old one, use the new streamer, if not the new one.
If you do not have a custom streamer, you can use a IO rule I/O Concepts - ROOT
Hi @Danilo ,
I tried writing an old version of CustomClass2 in a root file and read it out with the updated software and everything seems to be working fine.
For all the classes involved i don’t have any custom streamer (apart from the one i am trying to implement) and all the versions of the classes were updated correctly.
So I really think that the problem is due to the fact that the object written in the file has a map<int,CustomClass2> while the new version of the class has map<int,CustomClass2*>.
I don’t really understand how the I/O Concepts work or how they would solve my issue. Could you elaborate?
Anyhow i think i can fairly easily code a custom streamer for this class but I cannot understand how to read a vector or a map of unknown length for the TBuffer.
Best,
Giorgio
I understand you can write an instance of the old implementation of CustomClass2 and read it back with the new implementation without any issue. Is that correct? If yes, this is a very good news.
However, if yes, I do not understand what is the problem now: how does the map comes into play?
Hi @Danilo,
Yes the I/O of CustomClass2 is working without issues. The situation is:
I have a CustomClass1 that before the upgrade had a map<int,CustomClass2> while after the upgrade it has a map<int,CustomClass2*> in order to use polymorphism on CustomClass2.
If I write an old instance of CustomClass1 and try to read it with the updated software it gives the issues posted above.
From the errors it seems like it can’t read the map<int,CustomClass2> that is written in the old instance of CustomClass1 on the file and set it in the new map<int,CustomClass2*> which is in the new implementation of CustomClass1.
Also worth saying that for CustomClass1 I defined the operator= which use setters and getters from/to the map.
If it can help I can post the code I am working with.
There is something I am missing. Morphing T into T* and vice-versa is supported (as explained here) can you provide a minimal, stand-alone reproducer of the issue (classed code and program being run as well as dictionary generation)?
Hi @Danilo ,
It’s a bit difficult to send a demonstrator because everything is nested into a wider framework. Is there another way to proceed ?
Anyhow my problem at the moment is really how to read a vector from the TBuffer … in general I am puzzled on how to read c++ containers of unknown size from the TBuffer…
Best,
Giorgio
The TBuffer knows how to read the vector as a whole.
I am finding myself in need of writing a custom streamer in order to keep backward compatibility of a custom class of which i need to do I/O.
Writing a custom streamer which is not using the StreamerInfo infrastructure is ‘hard’ and to make it worth it, you need a really go reason (needing extreme performance and minimal sizes) and will make you lose a few features (splitting, reading without the library).
The I/O with this upgrade seems to work fine but when I try to read files produced before the upgrade the I/O breaks in particular the TStreamer complains.
What is the complaint?
If it can help I can post the code I am working with.
Please, it will make things clearer.
Yes the I/O of CustomClass2 is working without issues. The situation is:
A priori, the solution might look like:
#pragma read sourceClass="CustomClass1" targetClass="CustomClass1" version="[ OLD VERSION NUMBER ]" source="map<int,CustomClass2> _nameOfMapDataMember;" targer="_nameOfMapDataMember" code="{ for(elem : onfile._nameOfMapDataMember) _nameOfMapDataMember[elem.first] = new CustomClass2(elem.second); }"
@pcanal thanks for the reply.
I tried the solution of the pragma line and going back to the default streamer function. This seems to work better but it’s still not working.
Basically without the pragma line it gives the warning
Warning in <TStreamerInfo::BuildOld>: Cannot convert pair<int,CustomClass2>::second from type: CustomClass2 to type: CustomClass2, skip element
and then goes in segmentation fault.
While when I include the pragma line in the LinkDef it gives the same warning but does not go in segmentation fault. The map though is not being read correctly, mainly in the four loop it gives the right elem.first but the elem.second has random values instead of the correct ones.
Anyhow I put here the source code: QRunData is CustomClass1, QChannelRunData is CustomClass2. I don’t think you can compile it, but maybe it’s enough to check for major mistakes. rundatadb.zip (18.0 KB)