Wrong fitting results in v5.18 (using RooFit)?

Hello, I am trying to use RooFit to do a mass lifetime combined fitting. Since the lifetime is distorted by the impact parameter significance cuts, an acceptance function is added. I guess here is where the problem from. My codes worked in ROOT 5.12/00e, but after I moved to the latest one, v5.18/00, it failed (also failed in v5.17/**, but not checked again), I mean, the fitting result of tau is wrong (0.761 ± 0.012, the input value is 0.46, and the value returned by 5.12 is 0.479±0.007). The required ASCII file and the visualizations returned by v5.12 and v5.18 (would be produced by the code) are also attached to save the running time.

Did I do something silly? Thank you very much for any help you will give me.

Hereafter is the code.

void SignalFit(){
  gSystem->Load("libRooFit");
  using namespace RooFit;

  RooRealVar *m  = new RooRealVar( "m","invariant mass" , 6100., 6700., "MeV/c^{2}");
  RooRealVar *sm = new RooRealVar("sm","mass resolution",    0.,  100., "MeV/c^{2}");
  RooRealVar *t  = new RooRealVar( "t","proper time",        0.,   5.0, "ps"       );
  RooRealVar *st = new RooRealVar("st","time resolution",    0.,  0.08, "ps"       );

  RooDataSet* data = RooDataSet::read("asciiData.file", RooArgList(*m, *sm, *t, *st) );

  RooPlot* tframe = t->frame();
  data->plotOn(tframe);
  tframe->Draw();

  // the set of observables...
  RooArgSet* observables = new RooArgSet(*m,*sm,*t,*st);

  //Define the PDF  
  //--------------------------------------------------------------------------------
  //Mass
  RooRealVar *SigMass = new RooRealVar("SigMass","Signal mass", 6400, 6100, 6700, "MeV/c^{2}" );
  RooRealVar *MassResScale = new RooRealVar("MassResScale", "mass resolution scale",1.0, 0.1, 3.0 );
  RooFormulaVar *SigMassRes = new RooFormulaVar("SigMassRes", "Sig mass resolution",
						   "(@0*@1)", RooArgList(*MassResScale,*sm) );

  RooAbsPdf  *SigMassPdf = new RooGaussian("SigMassPdf","MassPDF of signal",*m,*SigMass,*SigMassRes);

  //
  //...propertime distributions
  //=================================================================================
  //Scale factor of Resolutions
  RooRealVar *TauResScale = new RooRealVar("TauResScale","time resolution scale",0.9936, 0.1, 3);  

  //
  //Singal
  //------------------------------------------------
  RooRealVar *SigTau = new RooRealVar("SigTau","Signal lifetime",0.10,0.001,10.0,"ps");
  RooRealVar *SigTauResBias = new RooRealVar("SigTauResBias","bias scale",0,-5,5);
  RooResolutionModel *SigTauRes = new RooGaussModel("SigTauRes","gauss resolution model",*t,
                                                   *SigTauResBias,*TauResScale,*st,*st);
  RooAbsPdf  *SigTauPdf = new RooDecay("SigTauPdf","unbiased proper time PDF",*t,
                                      *SigTau,*SigTauRes,RooDecay::SingleSided);

   // Take into account acceptance
  // signal acceptance model
  RooRealVar *Acc_a  = new RooRealVar("Acc_a","acceptance a", 3.189  );
  RooRealVar *Acc_n  = new RooRealVar("Acc_n","acceptance n", 3.14   );
  RooRealVar *Acc_c  = new RooRealVar("Acc_c","acceptance c", 0.1515 );
 
  RooAbsReal *Acc = new RooFormulaVar("Acc","(@0>0)*@3*(@1*@0)**@2/( 1+(@1*@0)**@2 )", RooArgList(*t,*Acc_a,*Acc_n,*Acc_c) );

  RooAbsPdf *SigTauAccPdf = new RooEffProd("SigTauAccPdf","biased proper time PDF", *SigTauPdf, *Acc);

  // product: unconditional
  RooAbsPdf *SigPdf = new RooProdPdf("SigPdf" ,"signal PDF", RooArgList(*SigMassPdf, *SigTauAccPdf) );

  SigTauResBias->setConstant(kTRUE);
  TauResScale->setConstant(kTRUE);

  // fitting
  RooFitResult *s = SigPdf->fitTo(*data,Minos(false),Strategy(1),Save(true),Verbose(true));

  s->Print("v");

  // make plots
  TCanvas *c = new TCanvas();
  c->Divide(1,2);
 
  RooPlot *p1 = m->frame(Bins(100),Title("invariant mass"));
  data->plotOn(p1);
  SigPdf->plotOn(p1,ProjWData(*sm,*data));
  c->cd(1); 
  p1->Draw();
  
  RooPlot *p2 = t->frame(Bins(100),Title("proper time"));
  data->plotOn( p2 );
  SigPdf->plotOn( p2, ProjWData(*st,*data) );
  
  c->cd(2); 
  p2->Draw();
}

ForLocateProblem.tar.gz (95.6 KB)




I can confirm that I am seeing the exact same problem with RooFit 5.18 and my own code- If you remove the acceptance function does the propertime converge correctly?
It does for mine, so I tried substituting RooEffProd for a RooGenericPdf containing the acceptance formula multiplied by the propertime PDF. This also failed.

It leads me to believe that something is wrong either in the way we have written our acceptance functions, or that there is something unusual in the way that multiplying pdfs to be fitted is done.

Can anyone see what we may be doing wrong, or if this is a bug?

Conor

EDIT: Also, I have tried two different acceptance function forms:

e_acc = 1.0 - 1.0 / (1.0+exp((t-a)/b)) (a fermi function)

and

e_acc = (t^a) / (1.0 + (b * t)^a) (Similar to the one above)

In both cases, the lifetime misbehaves unless I remove the acceptance. If I fix the lifetime, I get non-convergent behavior in the extended fit I do, or the acceptance parameters float away to limits. Fixing a,b and the lifetime causes my extended yields to hit limits.

[quote=“Conor”]I can confirm that I am seeing the exact same problem with RooFit 5.18 and my own code- If you remove the acceptance function does the propertime converge correctly?
[/quote]
Yes, it also works for me if I remove the acceptance function (of course, for the data without lifetime biased cuts).

I just follow this one:
twiki.cern.ch/twiki/pub/LHCb/Li … fetime2.cc

but the acceptance is not exactly the same. So, now I just go back to ROOT 5.12. I hope Wouter would give us a hand soon.

Do you know what version of RooFit you are using with 5.12? I’ll try to go back to that too and see if I can get it working there.

Thanks,

EDIT: also, could you try changing your acceptance function to:
"(@0>0)&&@3*(@1*@0)@2/( 1+(@1*@0)@2 )"

instead of

“(@0>0)@3(@1*@0)@2/( 1+(@1*@0)@2 )”

to see if this helps? From what I can see, multiplying by it does nothing, so the function runs away for negative propertimes. using && explicitly sets the function to zero for -ve values

Conor

[quote=“Conor”]Do you know what version of RooFit you are using with 5.12? I’ll try to go back to that too and see if I can get it working there.
[/quote]

Yes, it is RooFit v2.09. and the ROOT version is 5.12.00e, I used the binary for SLC4: root_v5.12.00e.Linux.slc4.gcc3.4.5.tar.gz

[quote=“Conor”]
EDIT: also, could you try changing your acceptance function to:
"(@0>0)&&@3*(@1*@0)@2/( 1+(@1*@0)@2 )"

instead of

“(@0>0)@3(@1*@0)@2/( 1+(@1*@0)@2 )”

to see if this helps? From what I can see, multiplying by it does nothing, so the function runs away for negative propertimes. using && explicitly sets the function to zero for -ve values

Conor[/quote]
Thanks. It still doesn’t work in root 5.18. But it works well in v5.12.

Thanks, I am now running my code on 5.12 and it is working fine. This is a pretty subtle bug, as I have spent the last month trying to get to the bottom of what I thought was a problem in my own code.

I still have no idea what is the source of the problem in 5.18, but I look forward to hearing what is causing it.

Conor

Hi,

I’ll have a look at this and get back to you.

Wouter

Hi,

Sorry for the long delay. This took me a while to debug. There are a couple of issues here. The first one is on my side. It turns out that the class that was handling the data-weighted projections (i.e. anytime you use ProjWData()) had some old code in it that was not consistent with the revised p.d.f optimization scheme. I have now reimplement this completely which fixes the normalization problems you’ve been seeing.

Having done that, I still don’t get the proper time plot to match your data but I am not convinced that that is caused by a RooFit problem. There are a couple of points in your example macro that caught my attention

  • The range of st starts at zero. This is bad practice as your p.d.f
    is singular at st=0 and your fits may not converge. It is better to
    keep a finite distance from zero.

  • You fit your p.d.f as a 4-D p.d.f to m,sm,s,st. This is unlikely to work
    correctly as the shape of your p.d.f in sm and st is nearly flat
    (it is not explicitly modeled that way, but it comes out that way you
    can easily see that by plotting the relevant projections). Since the
    data distributions in sm, st are far from flat your overall fit will
    be a bad fit, even if it looks reasonable projected on m,s.
    If you want to only fit the distributions of (m,s) but still want to
    take the correlations with (sm,st) into account you should fit your
    p.d.f as a conditional p.d.f F(s,m|sm,st) by adding

    ConditionalObservables(RooArgSet(*sm,*st))

    to your fitTo command line. This emake the fit run quite a bit
    slower as p.d.f normalization will have to be calculated for every
    event now rather than for every parameter change, but it will
    do the correct thing ( A NumCPU(8) addition on my dual quad-core
    Xeon is very helpful here :slight_smile:

Even so I do not get your p.d.f to agree with the data you provided. The reason I am not sure this is a RooFit problems is that if I fit to a ToyMC dataset generated from the p.d.f the agreement is good, which to first order proves that the fit and the corresponding plot projections function correctly on that that is known the be consistent with the p.d.f.

You can download the above mentioned new code from ROOT svn here:

svn co root.cern.ch/svn/root/branches/dev/roofit

Wouter

Thanks for this, Wouter, and my apologies for not getting back to this until now;

I’ll be rerunning my analysis in the coming weeks and will let you know if I can get it to work as it did with RooFit 5.09.

Conor

Hi All,
I am using RooFit v2.31, and ROOT 5.18.00a. I saw a similar problem with trying to implement an acceptance function on top of an exponential background. The data generated agreed very well with the PDF, but when I tried to fit it, it just would not produce physically sensible results. I changed the background to a RooGenericPdf which is my own version of the convolution of the back and acceptance, and then it works fine.

Do others cotinue to see this issue?

Cheers,
Steve

This issue is still present in ROOT 5.20/RooFit 2.50. I see identical (incorrect) results to those of 5.18 in a previous analysis.

Could someone possibly look into this again? I’ve found that newer root TFiles are no longer compatible with v5.12, so soon I won’t be able to revert to it anymore.

Thanks,

Conor

Hi,

Will do. I’ll get back to you in 1-2 days.

Wouter

Just wondering if you’ve had time to look at this yet Wouter?

Thanks,

Conor

Dear Wouter,

This problem is still there in RooFit v2.95 (ROOT 5.23.01). There are also errors with the above codes. Would you be so kind to have a look at it? You know, we don’t want to stick to the old ROOT 5.12. :slight_smile:

Thanks a lot.

I’m giving this a 1-year-anniversary bump, as I understand from Laurence that this problem is still not resolved. For his post on the matter which I believe to be pertinent, please see: root.cern.ch/phpBB2/viewtopic.php?t=6371

Wouter- I know you’re busy, but it’d be good to get this resolved before the LHC sees collisions (Maybe I’ll post the same a year from now!) :smiley:

Hi Conor et al,

Sorry about having missed your last postings! My last ‘topic reply notification’ mail on this thread dates from Sep 18. [ Fons: does notification stop some time after the original posting? ]

Can you be a bit specific about what is not working for you?

I just ran my version of your macro again (also attached for your convenience) in which I see that normalization problem is gone
when using the latest ROOT version.

There is a residual issue, which is that I suspect that your p.d.f. cannot describe that dataset that comes along with the macro. It definitely cannot without the ConditionalObservables() modifier that you need, but even with them it might not. I had made some more detailed points of this my last extensive posting that explained why that could be.

I had also shown that if I would fit a dataset sampled from your p.d.f. itself, instead of the provided data, which is a well-defined sanity check,
I obtain a correct fit result. A fit to externally provided data is a less powerful check, unless there is explicit proof that the data can be described by the model. Could you provide some more detail on why you think that is the case?

I’m not excluding them there might be more problems, but I need to know a bit more detail for you. So to go forward: can you run my macro and confirm these findings? Can you address the issues concerning ConditionalObservables() and model suitability that I raised in my original detailed reply.

If you have another (preferrably simpler) macro that illustrates the problem that would be good too.

Wouter
bug4.C (4.14 KB)

Hi again there Wouter,

I’m afraid I haven’t personally confirmed the recurrence of the bug as I have not tried my code on it since 5.20. The reason I bumped was because of an email I received from Laurence which I have copied below.

I’ll ask Laurence to post his code as he seems to still be seeing this problem, but if he is still using 5.18 this may explain it. hejb seems to think it is still in 5.23.01 according to his last post in the thread.

Conor

On Mon 16/03/09 at 15:41:23 +0000, Laurence Carson wrote:

Hi Conor,

I noticed that the (rather ancient) thread on RootTalk about lifetime
fitting including acceptance functions that you were involved in

root.cern.ch/phpBB2/viewtopic.php?t=6248

had some quite recent posts. Specifically the problem is still there in
the newer ROOT versions, and Wouter said he would look at it again, but
then that was it. Do you know if a fix for this exists anywhere?

I’m asking because I was seeing similar problems(see
root.cern.ch/phpBB2/viewtopic.php?t=6371), where the fit PDF
seemed to be basically ignoring the data. I found that switching from
v5.18 to v5.16 got rid of that problem, but a similar one then appeared
in v5.16 (once I’d added some background PDFs) where the normalisation
between my toy data and the PDF that was supposed to fit to the toy data
was all screwed up. I eventually got around this by ditching the
acceptance function, but now I’m revisiting the problem to see if I can
get it to work.

In your thread, Wouter suggested using ConditionalObservables() in the
fitTo() line as a fix, but I don’t see how this can solve everything as
if I plot the PDF before the fit, the normalisation is still wrong at
that stage.

Any advice you could give me would be appreciated.

Cheers,

Laurence

Hi, Conor,

Just now I tried with the head of ROOT, RooFit v2.98, it seems to me that the problem has been resolved. That’s Great! :smiley:

Wouter, Thanks a lot.

Jibo

This is great news! Thanks hejb. I take it you used the same acceptance function as listed earlier in the thread?

Did you use the ConditionalObservables() modifier to achieve this?

Thanks for all the work Wouter, pending a response from Laurence I think it is safe to say this is fixed.

Dear Conor, WOuter and others,

I too, am facing the same problem :frowning:

While generating toys, I realised that the PDFs are generated without the acceptance function. I am using the standard way, using RooEffProd, to convolute the PDF with an acceptance function.

Though, with earlier version of my code, where I used RooProdPdf for multiplying the acceptance function to the rest of the PDF, the generation worked fine.

Regards and thanks,
Nikhil

P.S.: (Un)fortunately, the experiment I am working with, have been taking data already :smiley: