I have a very general question about the updates that RooFit went through from Root version v6.12/06 to v6.18/04. Are you aware or can you point me out to significant changes in RooFit in v6.18/04 with respect to v6.12/06, that could largely affect the yield from the simultaneous data-MC fit, when using the same code, methods, parametrisation, and inputs? I can understand it might be difficult to tell since a lot was ongoing, but I still hope that you can point me to important points in the comparison.
I am asking since I got a large difference in the yield of my fit. However, most of the shape parameters agree in both versions within uncertainties, I wouldn’t complain about the shapes. But its necessary for me to understand the change in the normalization. Additionally, the running time was reduced from ~4 hours to 1 hour in the newer version!
Its unfortunately a bit difficult to share my code. I have a complicated unbinned extended maximum likelihood fit, with components as follows:
signal: Johnson Su+Gauss
background1: Johnson Su+Gauss
background2: Fermi-Dirac+exponential
background3: Crystal Ball
background4: exponential
background5: exponential.
For Johnson Su and Fermi-Dirac, we have our own formulas, and kept the same in the comparison. The other pdfs are used as they were implemented in RooFit. The model after adding all these components is convoluted with a Gaussian PDF. The resulting PDF is used to fit the data.
The first five components are constrained from the simultaneous data-MC fit. The normalization is constrained from the fit to data for all components. Weights are used.
As a trial to understand the differences, I went down to lower versions. I compared v6.12/06 to v6.14/08 and v6.16/00. The yield there agrees with v6.12/06 within uncertainties.
I am more interested in understanding v6.12/06 and v6.18/04, since the first was used in one publication and the second will be used in the updated publication.
there are the release notes [here for 6.16], but there’s not so much that changed according to those.
Maybe it’s not roofit, but minuit that changed. Could it be that one is using Minuit2 as minimiser, whereas the other uses Minuit? You can test by explicitly setting the minimiser in the newer version.
Between 6.16 and 6.18, I found those commits that might be interesting:
Especially the first one could be what you see, since you say that shape parameters are invariant, but yields are different. If you were using a RooAddPdf in an FFT convolution, the interpretation of the coefficients changes based on how large/short the range of the observable is that is used to perform the FFT.
Have a look at those bug reports (requires CERN login):
thanks for the reply. Yes I looked to the release notes, to be honest it was a bit difficult to find the exact change that cause the differences, however, I looked at the convolution bug.
For the minimizer, I never used the default minimizer, I set explicitly Minuit2 for all versions, so I don’t think its a reason.
For the speed, I see in release note 6.18, something about hist2workspace performance optimisations, and Faster, STL-like Collections in RooFit, could both account for the running time, no? https://root.cern/doc/v618/release-notes.html#roofit-libraries
For the convolution, I believe the first can explain something, since we use a RooAddPdf in an FFT convolution: https://github.com/root-project/root/commit/270a9897f050a77083af6ce8843e6573e7e65afc. For the bugs, I have not faced them myself or we haven’t checked in fact, but we probably have them and we didn’t notice, especially this bug https://sft.its.cern.ch/jira/browse/ROOT-9419, since all the pdfs are added via RooAddPdf, then convoluted with a Gaussian before the sum was used to fit the data. These could probably explain the different mass and scale resolution values of the convolution pdf between the two versions:
v6.12/06: sigma: 0.101 +/- 43.407 , mu: -2.349 +/- 0.642
v6.18/04: sigma: 3.366 +/- 1.509 , mu: 0.726 +/- 0.260
However, the shape parameters aren’t really invariant as you mentioned. I said earlier, most of them not changed, and if changed, still acceptable I believe. The only largest change was for the sigma of the Gaussian component in the the signal (has Gauss+Johnson PDF), changed from 51.557 +/- 1.033 to 34.351+/- 5.756. Itsn’t easy to tell if https://github.com/root-project/root/commit/7af50897bc9329248bf3e95a75c7f16ee8076b11 can explain the difference, since a Johnson pdf is added there, and there is some negative correlation between its lambda and the Gauss_sigma, and both have changed.
if you really want to drill down to the bottom of it, the only way is to compile ROOT from source, and jump to before/after the commits I listed. Note that once you have the first commit checked out, and compiled the first run, it’s very fast to jump to the next commit and recompile.
I would bet on the AddPdf used with FFT convolution. The three bug reports I listed are actually all caused by this.
Hi Stephan,
thanks again for the tip.
To confirm my understanding, anything before your commit should belong to v6.16/00 and earlier, so that they still have the bug, but its fixed in v6.18, right? The thing, is that I don’t compile and run my code as a standalone in root, its withing ATLAS software, AnalsysisBase. The easier for me is to test versions rather than commits! and that I did already and mentioned above: v6.12/06, v6.14/08 and 6.16 gave very similar results (within uncertainties). The big difference was only w.r.t v6.18/04 that I tested. But yes, in that way I don’t disentangle the causes, more than one commit could play a role in the observed changes >_<
the only other alternative is to write a mock fit model that you can run standalone.
I am 80% sure that it’s the fix that makes the coefficients of the AddPdf refer to the range before the transformation starts, which will get you a consistent definition of the coefficients. Before this fix, changing the number of bins or the order in which convolutions were performed would change the coefficients.
yes I thought about it too, thanks! I will see what I can do. I may then come back here
Just a second question, looking again at the reported bugs, especially the comment of Peter here RooFFTConvPdf of RooAddPdf gives incorrect coefficient value. He said that summing the pdfs then doing the convolution will give wrong coefficients, but if each pdf is convoluted then summed, this won’t produce wrong coefficients. So my question is, if the second way is used, then I would expect consistent results in all versions, right? or was the problem of wrong coefficients in both ways, because setBins can be used in both, and you said changing the bins would change the coefficients? thanks
The only detail I remember is that the observable that is used to perform the convolution may have an arbitrary range (i.e. you can define an arbitrary number of bins, and you can add or remove buffer zones for the FFT) to the left and right of the actual range.
If a RooAddPdf is used, it’s coefficients (i.e. the normalisation integrals that are used to compute the coefficients) refer to this range. If you do FFT first, copy the results out of the ConvPdf, and use that in a RooAddPdf, the coefficients only depend on the range where the addition happens.
