RDataFrame snapshots and RooFit deterministic behaviour

Roofit and RooStats
,
1

Hi experts,
I have a question/remark on the procedure about making RooDataSets given a RDataFrame snapshot TTree.

We observed in our analysis some strange behaviour, which we figured out being related to how Snapshots are done from RDataFrame. As we understood, the Snapshot in MT mode enables shuffling of entries order in the final TTree. If we use the shuffled/unshuffled TTree and we create a DataSet to fit later for it, we do observe different final results.
The fit becomes completely deterministic when exactly the same TTree index order is used.
Therefore the question:
Assuming a fitting routine takes an Input TTree, apply a cut to it and make a DataSet out of it, is it a known behaviour that the order in which a RooDataSet get filled can modify the final results?
I do expect this to happen because of the Strategy used :


Where the strategies chunks the data in n-equal slots, but if one shuffles the dataset entries order, the chunks division becomes not deterministic.

I admittely didn’t know that the fit results can depend on the order in which a dataset is created, but i wonder if there are any more robust and deterministic approach to use here to always get the same result even with a shuffled dataset order filling scheme.

Thanks in advance ,
Renato

2

Hello Renato,

I am involving @StephanH, as he might know the answer to this question.

Cheers,
J.

3

This is one for @moneta .

4

Hi

The results should differ only within some numerical error, and this difference should be mush smaller that the parameter error reported.
If this is not the case, it is am indication the fit is unstable and it is maybe due to some other problematic in the way the fitting model is built.

Lorenzo

5

Hi @moneta the funny thing of the story. Is that even the minNLL value is the same but in one case covqual is 1/2 and in another is 3. We have our fitter trying strategies in repetition until it converge and we give up after 4 trials. Toys fails in 5 percent of the cases and our fits uses gaussian constraints. Do you think it’s reasonable that we do observe this behaviour?

6

Also we use Ipathya functions for our models which internally uses numerical integrators, maybe it can be also linked to this?

7

Hi,
I think this can happen what you observe. One thing helpful to mitigate is using the RooFit option RooFit::Offset(true) when fitting, which improves the nll calculation in RooFit.

Lorenzo