Problem in merging trees

[SebastienProcureur]

Hello all,
I experience a strange problem in merging 3 trees. These trees contain the same branches, essentially values recorded by sensors at different times. The point is, some of the sensors stopped working during the measurement, so the 3rd file has the same branches as the other 2, but some of the branches are empty. Now, when I merge the 3 files, the branches for which the sensor got broken are completely empty, i.e. the values from the first 2 files were not copied in the merged tree. Of course when I only merge the first 2 files it’s fine.
In other words, I do this:
root -l file1.root
HV_Mon->Draw(“H_METEOMK1_85121:time”) -> fine
root -l file2.root
HV_Mon->Draw(“H_METEOMK1_85121:time”) -> fine
root -l file3.root
HV_Mon->Draw(“H_METEOMK1_85121:time”) -> empty draw
hadd -f5 merge.root file1.root file2.root file3.root
root -l merge.root
HV_Mon->Draw(“H_METEOMK1_85121:time”) -> empty draw
hadd -f5 merge.root file1.root file2.root
root -l merge.root
HV_Mon->Draw(“H_METEOMK1_85121:time”) -> fine

I also tried with TChain and TList, but with same results.
Of course you’ll ask me why I want to merge an empty file, but actually file3 contains some other branches with other working sensors… For them the merging goes fine btw.

I used root 5.34/36 and 6.19/01 on Linux Ubuntu, with again no difference.

Just in case I put the 3 files here:
https://cernbox.cern.ch/index.php/s/5oTpwIgj3pOxYTT

Thanks in advance for your help!

Kind regards,
Sébastien

[SebastienProcureur]

Update: in fact it is a problem with the Draw! If I use the command as in my post, the Draw creates a histogram between -1 and 1 in vertical axis, which turns out to be outside the range of values of this variable (Usually it scales automatically to the proper range, but in this case it fails, no idea why). If I define a histogram and then plot it here, then it works, which means the values are indeed in the merged file…
Seb

[Wile_E_Coyote]

[...]$ root file3.root
root [1] HV_Mon->Scan("H_METEOMK1_85121:time");
root [2] HV_Mon->Scan("H_METEOMK1_85121:time", "H_METEOMK1_85121 != TMath::Infinity()");

[pcanal]

@SebastienProcureur

but some of the branches are empty.

Note that this is a significant problem. The merging result are even vaguely correct if and only if the files with the empty branches are last.

For example if you have:

file1.root
entry   time   b1     b2
0       #11    v11b1   v11b2
1       #12    v12b1   v12b2
2       #13    v13b2   v13b2

file2.root with empty b2
entry   time   b1      b2
0       #21    v1b1   
1       #22    v2b1
2       #23    v3b2 

if you do hadd merged-okay-ish.root file1.root file2.root you get

merged-okay-ish.root
entry   time   b1      b2
0       #11    v11b1   v11b2
1       #12    v12b1   v12b2
2       #13    v13b2   v13b2
3       #21    v21b1   
4       #22    v22b1
5       #23    v23b2 

however if you do hadd merged-bad.root file2.root file1.root you get

merged-okay-ish.root
entry   time   b1      b2
0       #21    v21b1   v11b2
1       #22    v22b1   v12b2
2       #23    v23b2   v13b2
3       #11    v11b1 
4       #12    v12b1
5       #13    v13b2

Where in the merged-bad.root you can see that the 2nd branche values associated with the times of the first files ends up being associated with the times of the 2nd files …

To solve the problem, the best is to always merged files where the branch have always the sames number of entries. To do so you need to have a way to tell that even-though the branch has the value it is ‘invalid’. This can be done either by adding a column b2_valid or by replacing the branches by vector of size 0 or 1.

Cheers,
Philippe.

[SebastienProcureur]

Nice one, thanks a lot