Dear experts,
I am running into a funny behaviour when doing an extended unbinned likelihood fit.
I am working on a fit over different ranges for my observables of interest, so when I call fitTo I use RooFit::Range() with the appropriate range. I extend my single components with RooExtendPdf and then sum them all in a RooAddPdf.
I have recently added a component to the fit which is modelled with a RooNDKeysPdf. I was expecting a bit of a slowdown due to the fact that its reference dataset is of the order of 5k events, however the minimisation itself slows down by quite a lot, which surprised me, because I was expecting RooFit to cache in someway the value of the PDF in the expression for the likelihood.
Digging a bit and doing some tests, I discovered that the slowing down in the fit is dependent on how the total extended pdf is built. If I avoid using the RooExtendPdf class and just use the constructor of RooAddPdf with the list of components and the list of yields, the fit goes back to being quite quick (apart from the initialisation and initial integral calculcation for the KDE).
Mind you, the different fitting times are only observed when I use the RooFit::Range() option in the fit. If I don’t use it, I see no difference in fit times between the two constructions of the final PDF.
Below is an example script to show this behaviour.
I am not sure if this is meant to be that way, if it’s a bug, or just if I am at an unlikely edge-case.
Thank you in advance!
Lorenzo
void exampleCaching(){
// Generate some data from a polynomial+gauss distribution
RooRealVar x("x", "x", -1., 1.);
RooRealVar y("y", "y", -1., 1.);
x.setRange("reduced", -0.8, 0.8);
y.setRange("reduced", -0.8, 0.8);
RooRealVar a1("a1", "a1", 2.0, -3.0, 3.0);
RooRealVar a2("a2", "a2", 1.0, -3.0, 3.0);
RooRealVar b1("b1", "b1", 1.0, -3.0, 3.0);
RooRealVar b2("b2", "b2", 1.0, -3.0, 3.0);
RooRealVar yieldPoly("yieldPoly", "yield", 2000., 0., 1e5);
RooArgList coeffsX;
coeffsX.add(a1); coeffsX.add(a2);
RooPolynomial polyX("polyX", "polyX", x, coeffsX);
RooArgList coeffsY;
coeffsY.add(b1); coeffsX.add(b2);
RooPolynomial polyY("polyY", "polyY", y, coeffsY);
RooProdPdf poly("poly", "poly", polyX, polyY);
RooExtendPdf polyExt("genPDFExt", "genPDFExt", poly, yieldPoly);
RooRealVar mu("mu", "mu", 0, -3.0, 3.0);
RooRealVar sigma("sigma", "sigma", 0.05, 0., 3.0);
RooRealVar yieldGauss("yieldGauss", "yield", 500., 0., 1e5);
RooGaussian gaussX("gaussX", "gaussX", x, mu, sigma);
RooGaussian gaussY("gaussY", "gaussY", y, mu, sigma);
RooProdPdf gauss("gauss", "gauss", gaussX, gaussY);
RooExtendPdf gaussExt("gaussExt", "gaussExt", gauss, yieldGauss);
RooAddPdf genPDFExt("genPDFExt", "genPDFExt", {polyExt, gaussExt});
RooRandom::randomGenerator()->SetSeed(2023);
RooDataSet* data = genPDFExt.generate({x,y}, RooFit::Extended());
// Model bkg with a KDE, extend components with a RooAddPdf, fit
RooNDKeysPdf kde("kde", "kde", {x,y}, *data);
RooArgList pdfs;
RooArgList yields;
pdfs.add(kde); pdfs.add(gauss);
yields.add(yieldPoly); yields.add(yieldGauss);
RooFitResult* result;
std::cout << "Fitting with RooAddPdf" << std::endl;
RooAddPdf fitPDFAdd("fitPDFAdd", "fitPDFAdd", pdfs, yields);
result = fitPDFAdd.fitTo(*data, RooFit::Range("reduced"));
// Now use RooExtendedPdf and wrap the extended components in a RooAddPdf
std::cout << "Fitting with RooExtendPdf" << std::endl;
RooExtendPdf kdeExt("kdeExt", "kdeExt", kde, yieldPoly);
RooAddPdf fitPDFExt("fitPDFExt", "fitPDFExt", {kdeExt, gaussExt});
result = fitPDFExt.fitTo(*data, RooFit::Range("reduced"));
}