Multi-Range Simultaneous Fit: SplitRange() functionality not working

Dear experts,

I am trying to perform a simultaneous fit over a mass spectrum, where the mass variable has a different lower bound for each category. I have created a simple toy macro to demonstrate, which is attached, along with the plots it generates. I have also broken down the logic below:

I start by initialising the RooRealVar that will be my fitting variables, deltaMass, and defining the 2 different ranges:

  int delta_low = 0;
  int delta_high = 210;

  int delta_nbins = (delta_high - delta_low) / 2;

  RooRealVar deltaMass("Delta_M", "m[D^{*0}] - m[D^{0}]", delta_low, delta_high,
           "MeV/c^{2}");

  deltaMass.setRange("range_pi0", 135, 210);
  deltaMass.setRange("range_gamma", 50, 210);

  deltaMass.setBins(delta_nbins);

I have 2 neutral categories: gamma and pi0 (these represent the 2 neutral particles in the D*0 -> D0 pi0/gamma decay). I need the fit to the D*0 - D0 mass variable to be performed over the range 135-210 MeV for the pi0 category, and 50-210 MeV for the gamma category.

I then define the Categories and PDFs for the 2 neutrals:

 // DEFINE INDEX CATEGORIES
  RooCategory neutral("neutral", "neutral");
  neutral.defineType("pi0");
  neutral.defineType("gamma");

  // DEFINE pi0 PDFs
  // Signal
  RooRealVar meanDeltaSigPi0("meanDeltaSigPi0", "Mean of Sig m[Delta]", 145,
           140, 150);
  RooRealVar sigmaDeltaSigPi0("sigmaDeltaSigPi0", "Sigma of pi Delta Gaussian",
            3, 0, 5);
  RooGaussian pdfDeltaSigPi0("pdfDeltaSigPi0", "Sig pi Delta PDF", deltaMass,
           meanDeltaSigPi0, sigmaDeltaSigPi0);

  // Background
  RooRealVar thDeltaBkgPi0("thDeltaBkgPi0", "", 1.3415e+02);  //, 30, 50);

  RooRealVar aDeltaBkgPi0("aDeltaBkgPi0", "", 5.3768e-01);   //, -1, 1);
  RooRealVar bDeltaBkgPi0("bDeltaBkgPi0", "", -4.4782e-01);  //, -1, 1);
  RooRealVar cDeltaBkgPi0("cDeltaBkgPi0", "", 5.8018e+00);   //, 0, 35);

  RooDstD0BG pdfDeltaBkgPi0("pdfDeltaBkgPi0", "", deltaMass, thDeltaBkgPi0,
          cDeltaBkgPi0, aDeltaBkgPi0, bDeltaBkgPi0);

  // ROOADDPDF
  RooRealVar yieldSigPi0("yieldSigPi0", "", 1000, 0, 5000);
  RooRealVar yieldBkgPi0("yieldBkgPi0", "", 10000, 0, 50000);

  RooArgSet yieldsPi0;
  yieldsPi0.add(yieldSigPi0);
  yieldsPi0.add(yieldBkgPi0);
  RooArgSet functionsPi0;
  functionsPi0.add(pdfDeltaSigPi0);
  functionsPi0.add(pdfDeltaBkgPi0);

  RooAddPdf pdfPi0("pdfPi0", "", functionsPi0, yieldsPi0);

    // DEFINE gamma PDFs
  // Signal Gamma
  RooRealVar meanDeltaSigGamma1("meanDeltaSigGamma1", "Mean of Sig m[Delta]",
        145, 140, 150);
  RooRealVar sigmaDeltaSigGamma1("sigmaDeltaSigGamma1",
         "Sigma of pi Delta Gaussian", 10, 0, 20);
  RooGaussian pdfDeltaSigGamma1("pdfDeltaSigGamma1", "Sig pi Delta PDF",
        deltaMass, meanDeltaSigGamma1,
        sigmaDeltaSigGamma1);

  // Signal Pi0 in Gamma mode
  RooRealVar meanDeltaSigGamma2("meanDeltaSigGamma2", "Mean of Sig m[Delta]",
        85, 80, 90);
  RooRealVar sigmaDeltaSigGamma2("sigmaDeltaSigGamma2",
         "Sigma of pi Delta Gaussian", 10, 0, 20);
  RooGaussian pdfDeltaSigGamma2("pdfDeltaSigGamma2", "Sig pi Delta PDF",
        deltaMass, meanDeltaSigGamma2,
        sigmaDeltaSigGamma2);

  // Background
  RooRealVar thDeltaBkgGamma("thDeltaBkgGamma", "", 5.2160e+01);  //, 30, 50);

  RooRealVar aDeltaBkgGamma("aDeltaBkgGamma", "", 1.0341e+00);   //, -1, 1);
  RooRealVar bDeltaBkgGamma("bDeltaBkgGamma", "", -1.2284e+00);  //, -1, 1);
  RooRealVar cDeltaBkgGamma("cDeltaBkgGamma", "", 4.2450e+01);   //, 0, 35);

  RooDstD0BG pdfDeltaBkgGamma("pdfDeltaBkgGamma", "", deltaMass,
            thDeltaBkgGamma, cDeltaBkgGamma, aDeltaBkgGamma,
            bDeltaBkgGamma);

  // ROOADDPDF
  RooRealVar yieldSigGamma1("yieldSigGamma1", "", 2000, 0, 5000);
  RooRealVar crossFeed("crossFeed", "", 1.5, 0, 5);
  RooFormulaVar yieldSigGamma2("yieldSigGamma2", "", "@0*@1",
             RooArgList(yieldSigPi0, crossFeed));
  RooRealVar yieldBkgGamma("yieldBkgGamma", "", 20000, 0, 50000);

  RooArgSet yieldsGamma;
  yieldsGamma.add(yieldSigGamma1);
  yieldsGamma.add(yieldSigGamma2);
  yieldsGamma.add(yieldBkgGamma);
  RooArgSet functionsGamma;
  functionsGamma.add(pdfDeltaSigGamma1);
  functionsGamma.add(pdfDeltaSigGamma2);
  functionsGamma.add(pdfDeltaBkgGamma);

  RooAddPdf pdfGamma("pdfGamma", "", functionsGamma, yieldsGamma);

I then generate the datasets for each category and cut them to be in the associated range:

// Generate datasets

  auto dataPi0_1 =
      std::unique_ptr<RooDataSet>(pdfPi0.generate(RooArgSet(deltaMass), 11000));
  std::unique_ptr<RooDataSet> dataPi0;
  dataPi0 =
      std::unique_ptr<RooDataSet>(dynamic_cast<RooDataSet *>(dataPi0_1->reduce(
          RooFit::CutRange("range==range::range_pi0"))));
  if (dataPi0.get() == nullptr) {
    throw std::runtime_error("Could not reduce pi0 input dataset.");
  }

  auto dataGamma_1 =
      std::unique_ptr<RooDataSet>(pdfGamma.generate(RooArgSet(deltaMass), 11000));
  std::unique_ptr<RooDataSet> dataGamma;
  dataGamma =
      std::unique_ptr<RooDataSet>(dynamic_cast<RooDataSet *>(dataGamma_1->reduce(
          RooFit::CutRange("range==range::range_gamma"))));
  if (dataGamma.get() == nullptr) {
    throw std::runtime_error("Could not reduce gamma input dataset.");
  }

Next, I make the combined dataset and simultaneous PDF:

  RooDataSet combData("combData", "", RooArgSet(deltaMass),
          RooFit::Index(neutral),
          RooFit::Import("pi0", *dataPi0.get()),
          RooFit::Import("gamma", *dataGamma.get()));

  // CONSTRUCT SIMULTANEOUS PDF
  RooSimultaneous simPdf("simPdf", "", neutral);
  simPdf.addPdf(pdfPi0, "pi0");
  simPdf.addPdf(pdfGamma, "gamma");

I then fit the simultaneous PDF to the combined dataset, with the Range, SumCoefRange and SplitRange functionality specified:

  std::unique_ptr<RooFitResult> result = std::unique_ptr<RooFitResult>(
      simPdf.fitTo(combData, RooFit::Extended(true),
                   RooFit::Range("range"),
                   RooFit::SumCoefRange("range"),
                   RooFit::SplitRange(true),
                   RooFit::Save(),
                   RooFit::Strategy(2), RooFit::Minimizer("Minuit2"),
                   RooFit::Offset(true)));

From the documentation, I understand that SplitRange should match a range to categories using rangeName_{INDEX}, where INDEX is the category label. I have therefore labelled my categories and ranges accordingly. However, this option doesn’t seem to be doing anything. The fit is still being performed over the entire range (0-210 MeV). I have also tried commenting out the Range and SumCoefRange options, but it makes no difference.

I can fit in one specific range by removing the SplitRange option and passing Range and SumCoefRange either “range_gamma” or “range_pi0”, but I cannot get the fit to work simultaneously over the 2 ranges.

I need the fit to be simultaneous because the signal shapes in the pi0 and gamma mode share some physics parameters (the magenta shapes in the plots). The reason I need the different ranges is is because the background PDFs, DstD0BG shapes, cause the fit to become unstable if you extend the fit range below the kinematic threshold.

Your help with this would be appreciated as I’ve searched the tutorial macros and can’t find any example of this functionality.

Best wishes,
Alexandra

SimToy.cpp (8.1 KB)
SimToy.pdf (35.7 KB)

Hi Alexandra,

I had a look at the documentation of SplitRange, but it’s not clear to me what this is meant for. Maybe it doesn’t apply to your case, because to me it looks like it was meant to split a single category. Let’s see if you understand it in the same way:
You have a single category, and define two ranges. Now, using SplitRange, you can transform this into a simultaneous fit with two categories. Does that make sense?

If the above is correct, you don’t need the splitting, because you already have two categories, and you are doing a simultaneous fit.

Hi Alexandra,

a little addendum:
https://root.cern.ch/doc/master/classRooAbsOptTestStatistic.html#a1583b5cb7e52481cd3549a87dd9322a4

According to this documentation, you can use this to set a different range in each category. Let’s say that you have two categories, and both have the variable x. You need the range of x to be different in both. According to how I understand the comment, you would do this:

x.setRange("myRange_1", xxx, xxx);
x.setRange("myRange_2", yyy, yyy);

and then the ranges in the two categories 1 and 2 would be different. It could be that you have to start at 0.

Could you confirm if this works? Maybe even post a code snippet if you get it to work? If that’s the case, I will update the documentation.

H Stephan,

Thank you for your replies.

In reply to your first message:
I understand what you are sating about SplitRange, it is possible I have misunderstood the documentation.

In reply to your second:
I edited the code in my initial post by commenting out SplitRange in FitTo. This should then mirror what you have asked for, as I label my mass ranges in the following way:

  int delta_low = 0;
  int delta_high = 210;

  int delta_nbins = (delta_high - delta_low) / 2;
  RooRealVar deltaMass("Delta_M", "m[D^{*0}] - m[D^{0}]", delta_low, delta_high,
           "MeV/c^{2}");

  deltaMass.setRange("range_pi0", 135, 210);
  deltaMass.setRange("range_gamma", 50, 210);

I also tried commenting out the Range() and SumCoefRange() options when calling FitTo, but the fit is still being performed over the entire distribution (0-210).

Perhaps I misunderstood your message and I have to construct a RooAbsOptTestStatistic object somehow? It’s not clear to me from the documentation how to use this, but I assumed that it worked internally in the simultaneous fit and didn’t have to be called by the user.

Thanks,
Alexandra

Hi Alexandra,

yes, that’s internal to the minimisation process. You don’t need to touch that yourself.

What I understand from your reply is that you indeed want to have different ranges in different categories. Since I’m also not sure, yet, how to interpret the documentation, try the following (and tell me if it works!):

deltaMass.setRange("range_1", 135, 210);
deltaMass.setRange("range_2", 50, 210);

For 1 and 2 please insert the real numbers of the categories. You can read them off from:

neutral.Print("V");

V for verbose.

Use RooFit::Range("range") as the range name and RooFit::SplitRange(true). Now, RooFit may automatically compose names like "range_0", "range_1" etc from this. I don’t know, however, if it starts at 0 or 1.

If you make the message service a bit more verbose, you might actually see which range names are being loaded. You can do that using:

auto& msg = RooMsgService::instance();
msg.setGlobalKillBelow(RooFit::DEBUG);
msg.getStream(0).minLevel= RooFit::DEBUG;
msg.setStreamStatus(0, true);

PS (alternative):
At the bottom of tutorials/roofit/rf404_categories.C in your root folder, there’s also something on setting ranges for each category.
https://root.cern.ch/doc/master/rf404__categories_8C_source.html

Hi Stephan,

I have done as you suggested. I label the ranges according the the number assigned to them:

// DEFINE INDEX CATEGORIES
  RooCategory neutral("neutral", "neutral");
  neutral.defineType("pi0");
  neutral.defineType("gamma");

  neutral.Print("V");

  deltaMass.setRange("range_0", 135, 210);
  deltaMass.setRange("range_1", 50, 210);

The following is my FitTo() command:

  std::unique_ptr<RooFitResult> result = std::unique_ptr<RooFitResult>(
      simPdf.fitTo(combData, RooFit::Extended(true),
                   RooFit::Range("range"),
                   RooFit::SplitRange(true),
                   RooFit::Save(),
                   RooFit::Strategy(2), RooFit::Minimizer("Minuit2"),
                   RooFit::Offset(true)));

I can see that the range names are labelled with the correct index because I get this output:

— RooAbsCategory —
Value is “pi0” (0)
Has the following possible values:
pi0 = 0
gamma = 1
[#1] INFO:Eval – RooRealVar::setRange(Delta_M) new range named ‘range_0’ created with bounds [135,210]
[#1] INFO:Eval – RooRealVar::setRange(Delta_M) new range named ‘range_1’ created with bounds [50,210]

But unfortunately the simultaneous fit is still over 0-210 MeV, for both categories!

Thanks,
Alex

Hi Stephan,

Looking at the documentation, the member function setRange in RooCategory only accepts arguments in the form of a named list:
https://root.cern.ch/doc/v610/classRooCategory.html#a33a5d3909f8aaf9e5549a2aeaf96e3f0

I interpret from this, and the example you sent, that this function creates a range defined by a certain category rather than allowing the user to alter a variable range within a given category. Does that make sense to you?

Thanks,
Alexandra

Yes and no. It’s not a list in a programming sense. It, however, doesn’t seem to apply to your problem. Apparently, this is to define a “range” that automatically switches on/off certain categories.

So we are back to SplitRange. Was there any output as to what range the fit was using?

Also, there is the following output related to RooAbsTestStatistic:

RooAbsTestStatistic::initSimMode: creating slave calculator #0 for state pi0 (11000 dataset entries)
reduceEng varSubset = (Delta_M) _wgtVar =
[#1] INFO:Fitting – RooAbsOptTestStatistic::ctor(pi0) constructing test statistic for sub-range named range
[#1] INFO:Eval – RooRealVar::setRange(Delta_M) new range named ‘NormalizationRangeForrange’ created with bounds [0,210]
[#1] INFO:Eval – RooRealVar::setRange(Delta_M) new range named ‘fit_pi0’ created with bounds [0,210]
[#1] INFO:Fitting – RooAbsOptTestStatistic::ctor(pi0) fixing interpretation of coefficients of any RooAddPdf to full domain of observables
[#1] INFO:NumericIntegration – RooRealIntegral::init(pdfDeltaBkgPi0_Int[Delta_M]) using numeric integrator RooIntegrator1D to calculate Int(Delta_M)
RooAbsTestStatistic::initSimMode: creating slave calculator #1 for state gamma (11000 dataset entries)
reduceEng varSubset = (Delta_M) _wgtVar =
[#1] INFO:Fitting – RooAbsOptTestStatistic::ctor(gamma) constructing test statistic for sub-range named range
[#1] INFO:Eval – RooRealVar::setRange(Delta_M) new range named ‘fit_gamma’ created with bounds [0,210]
[#1] INFO:Fitting – RooAbsOptTestStatistic::ctor(gamma) fixing interpretation of coefficients of any RooAddPdf to full domain of observables

So it is creating different ranges but they are called ‘fit_pi0’ and ‘fit_gamma’ and go from 0-210 MeV… It doesn’t seem to find the ranges I specify?

… I therefore edited my range names to:

  deltaMass.setRange("fit_pi0", 135, 210);
  deltaMass.setRange("fit_gamma", 50, 210);

and called RooFit::Range(“fit”), but the same thing happened, only the output then said:

[#1] INFO:Fitting – RooAbsOptTestStatistic::ctor(pi0) constructing test statistic for sub-range named fit_pi0
[#1] INFO:Eval – RooRealVar::setRange(Delta_M) new range named ‘NormalizationRangeForfit_pi0’ created with bounds [0,210]
[#1] INFO:Fitting – RooAbsOptTestStatistic::ctor(pi0) fixing interpretation of coefficients of any RooAddPdf to full domain of observables
[#1] INFO:NumericIntegration – RooRealIntegral::init(pdfDeltaBkgPi0_Int[Delta_M]) using numeric integrator RooIntegrator1D to calculate Int(Delta_M)
[#1] INFO:NumericIntegration – RooRealIntegral::init(pdfDeltaBkgPi0_Int[Delta_M|NormalizationRangeForfit_pi0]_Norm[Delta_M]) using numeric integrator RooIntegrator1D to calculate
Int(Delta_M) RooAbsTestStatistic::initSimMode: creating slave calculator #1 for state gamma (11000 dataset entries) reduceEng varSubset = (Delta_M) _wgtVar =
[#1] INFO:Fitting – RooAbsOptTestStatistic::ctor(gamma) constructing test statistic for sub-range named fit_gamma
[#1] INFO:Eval – RooRealVar::setRange(Delta_M) new range named ‘NormalizationRangeForfit_gamma’ created with bounds [0,210]
[#1] INFO:Fitting – RooAbsOptTestStatistic::ctor(gamma) fixing interpretation of coefficients of any RooAddPdf to full domain of observables
[#1] INFO:NumericIntegration – RooRealIntegral::init(pdfDeltaBkgGamma_Int[Delta_M]) using numeric integrator RooIntegrator1D to calculate Int(Delta_M)
[#1] INFO:NumericIntegration – RooRealIntegral::init(pdfDeltaBkgGamma_Int[Delta_M|NormalizationRangeForfit_gamma]_Norm[Delta_M]) using numeric integrator RooIntegrator1D to calcu
late Int(Delta_M)

Out of interest I then changed fit to NormalizationRangeForfit and then the output changed the names to NormalizationRangeForfitNormalizationRangeForfit_gamma/pi0.

I really don’t understand what’s happening here…

Hi Stephan,

Yes, see the posts following on from my last reply (sorry, I think I replied to my own message therefore perhaps you were not notified):

Thanks,
Alex

It’s normal that a range fit_<someName> is created if no range is given. Since you now created this by hand, did you verify that it was/wasn’t using this range in the fit?

I tried the debug setting to get a more verbose output, and I searched for any messages to do with Range, but there was no mention of the ‘range_0’ and ‘range_1’ that I specify, except at the beginning when they are created. Instead ranges named NormalizationRangeForrange and fit_<someName>, created by RooFit, are used in the fit.

Yes, that’s why I was asking. Since you created a range fit_pi0, which is the same that RooFit would have created automatically, did it use this range in the fit?

That still doesn’t solve the problem of getting this to work properly, but I’m still trying to investigate in which circumstances certain ranges are picked up.

Hi Stephan,

No, if you see my second reply - it then replaced the range names created internally used in the fit to NormalizationRangeForfit_pi0 and NormalizationRangeForfit_gamma.
I tried using these range names, and then the ones used were called NormalizationRangeForfitNormalizationRangeForfit_pi0 and NormalizationRangeForfitNormalizationRangeForfit_gamma.

I have attached a stand alone macro that you can play around with if that is easier,SimToy.cpp (9.0 KB)

That will be much easier. However, I cannot dedicate any time before mid next week.

Hi Stephan,

The example I attached is the same code as the one I explained originally, but I added an additional gaussian at 25 MeV in the gamma category so that it would be easier to see whether the PDF was being fit to the data below 50 MeV.
If you could get back to me mid-next week, when you become available, I would really appreciate it.

Thanks,
Alexandra

Hi Stefan,

Have you managed to take a look at this yet?

Best wishes,
Alexandra

Hi Alexandra,

could you please try

  deltaMass.setRange("range_{pi0}", 135, 210);
  deltaMass.setRange("range_{gamma}", 50, 210);

Maybe we simply forgot the curly braces.