TKDE crash for more than 9999 data points

ROOT
1

Hey everybody,

I encountered a problem with the TKDE class. Apparently the TDKE->GetFunction() returns inf/nan values if the TKDE is filled with more than 9999 data points.
Hence, the plotting of the TF1 returned by TDKE->GetFunction() crashes with the typical warning:
Warning in TCanvas::ResizePad: Inf/NaN propagated to the pad. Check drawn objects.
Warning in TCanvas::ResizePad: TheCanvas height changed from 0 to 10.

Here is what I’m trying to do:

    int entries = inputTree->GetEntries();  
    std::vector<double> data(entries);
    for(int entryIT=0; entryIT<entries; entryIT+=1){
      inputTree->GetEntry(entryIT);
      double keVEnergy = eVenergy/1000;
      data[entryIT] = keVEnergy;
    }
   
    double rho = 1.0;
    TKDE *currentKDE = new TKDE(entries, &data[0], 0, IncidentEnergy, "",rho);

    TCanvas* TheCanvas = new TCanvas("TheCanvas","TheCanvas",1700,1000);

    TF1 * hk = currentKDE->GetFunction(100);
    hk->Draw();

I have a TTree “inputTree” containing a Branch “eVenergy” with signed double data type.
All the enties in the TTree are valid (meaning no nan/inf).

If the TTree contains 9999 or less entries everything works fine, but if the TTree contains 10000 or more entries the hk->Draw() command fails lieke described above.
However, there are certain “rho” values (e.g. 3.0) for which the plot succeeds, but for my application the smooting is way to strong in these cases.

Do you have any clue where this kind of limit might come from?

This is what I found out so far:
The TKDE.cxx in root/hist/hist/src/ has a values of 10000 in its constructor and in TKDE::Instantiate

TKDE::TKDE(UInt_t events, const Double_t* data, Double_t xMin, Double_t xMax, const Option_t* option, Double_t rho) :
 ...
   fNBins(events < 10000 ? 100: events / 10),
   fNEvents(events),
   fUseBinsNEvents(10000),
 ...
{...}

void TKDE::Instantiate(KernelFunction_Ptr kernfunc, UInt_t events, const Double_t* data, Double_t xMin, Double_t xMax, const Option_t* option, Double_t rho) {
 ...
   fNBins = events < 10000 ? 100 : events / 10;
   fNEvents = events;
   fUseBinsNEvents = 10000;
 ...
}

Nevertheless, if I increase these values, recompile root and try my little script it fails under the same conditions. :confused: Well, binning is deactivaded anyway.

Regards
Martin

2

Hi,

There is a limit of 10000 in the way the weights are used for the kernel estimations. If event number is larger than 10000 binned weights are used. Maybe try running with the option “binning:unbinned” passed in the constructor.
However, I would need your input data file to investigate your problem

Best Regards

Lorenzo

3

Hi Lorenzo,

thanks for your reply.
I’ve already tried the binning:unbinned" option and different kernels as well.
However, both didn’t change anything.
The only option to avoid the inf/nan error is to select a rho around 3.0 since significantly lower or higher values lead to the same crash.

Unfortunately my data file is too large to upload here so I put it in a Dropbox. Here is the Link:
dropbox.com/s/ipff2tuwxrtlj … .root?dl=0
It is a root file containing a tree with a single branch. I checked the file for inf/nan values and found none. Hence, the data itself seem to be ok.

Thank you for your help!

Best Regards,
Martin

4

Hi,

Sorry for my late reply. Using the latest version ROOT, 5.34.23. or 6, I don’t have a crash after 10000 events in TKDE. So I cannot reproduce the problem.
Howver I see that your data is having a spike at the maximum value. This is nor very good for a Kernel estimation, o the obtained kernel based distribution is not very good

Best Regards

Lorenzo

5

Hi,

thanks for your help, I made an update of root and its working fine now.
I’m aware of the spike, however, since I’m using a sub-range that excludes the spike the KDE seems to perform quite well.

Regards,

Martin