Different Chi2/ndf results

[Elena_Veli]

Hi,
So I have made a program to do fitting in some values. The chi2/ndf, I obtain from the TGrapherror is very different from the one of the TF1. The TF1 for instance can give around 2 while the TGrapherror gives 0.03… My errors are very small and non existant in some points and they are calculated correctly… but I would like to have them and fit with them, is this ok? or is it better to just do it without them? Why the chi2/ndf is so different? I have seen in other answers to fit with W. I do fit with W in TF1, but in tgrapherror, I want to fit with the tgraph rror in order to account even the small errors. Which of the 2 chi2/ndf should i report as correct?! it seems the higher value to be correct, since there are fits like these in the attached photo. Yes the errors are that small… the code with the tgrapherrors:

`std::unordered_map<PixelPosition, CalibrationFunction> calcCalibrationParams(const BiasedChargeMap& biasedCharges, TTree* data) {
std::unordered_map<PixelPosition, CalibrationFunction> calibrationParams;

for (auto& [p, line] : biasedCharges) {
    if (line.size() < 9) {  // Ensure there's enough data to perform a meaningful fit
        continue;
    }

    // Create TGraphErrors to hold the data points with error bars
    TGraphErrors graph(line.size());

    for (size_t i = 0; i < line.size(); ++i) {
        // Use the mean charge and error directly from the ChargeAtBias structure
        double mean_charge = line[i].charge;
        double error = line[i].error;

        std::cout <<"qtp" << qtp << "charge = " << mean_charge << ", samples = " << line[i].samples
                  << ", error = " << error << '\n';

        // Add the point and error to the graph
        graph.SetPoint(i, qtps[line[i].bias], mean_charge);
        graph.SetPointError(i, 0, error);  // x-error is 0, y-error is the calculated error
    }

    // Define the fitting function, where [2] is dynamically set to the last charge
    double last_charge = line[line.size() - 1].charge;
    std::stringstream function_def;
    function_def << last_charge << "*pow((1-exp(-[0]*x)),[1])";

    TF1 ex("tf_name", function_def.str().c_str(), 0, 60);
    ex.SetParameters(1./12., 1.0);  // Initial guesses for [0] and [1]

    // Add a synthetic point at (0,0) with a very small error to enforce the fit passing through the origin
    graph.SetPoint(graph.GetN(), 0, 0);
    graph.SetPointError(graph.GetN()-1, 0, 1e-10);  // Very small error to enforce fit through (0,0)

    // Perform the fit
    graph.Fit(&ex, "W");

    double chi_square_ndf = ex.GetChisquare() / ex.GetNDF();
    calibrationParams[p] = { ex.GetParameter(0), ex.GetParameter(1), last_charge + 0.001f, chi_square_ndf };

    // Plot the graph with error bars
    TCanvas* c1 = new TCanvas("c1", "Calibration Fit", 800, 800);  // Create a square canvas
    graph.SetMarkerStyle(20); // Marker style
    graph.SetMarkerSize(0.5); // Marker size
    graph.SetMarkerColor(kBlue); // Marker color
    graph.Draw("AP"); // Draw points with error bars
    ex.Draw("same");

    // Set axis ranges
    graph.GetXaxis()->SetLimits(0, 60);  // Set x-axis limits
    graph.SetMinimum(0);                 // Set y-axis minimum
    graph.SetMaximum(15);                // Set y-axis maximum to match the x-axis range

    // Add text annotations
    TLatex latex;
    latex.SetNDC();
    latex.SetTextSize(0.03);

    std::stringstream func_text, chi_square_text;
    func_text << "f(qtp) = " << last_charge << " * (1 - exp(-" << ex.GetParameter(0) << " * qtp))^{" << ex.GetParameter(1) << "}";
    chi_square_text << "#chi^{2}/NDF = " << chi_square_ndf;

    latex.DrawLatex(0.15, 0.85, func_text.str().c_str());
    latex.DrawLatex(0.15, 0.80, chi_square_text.str().c_str());

    // Save the plot
    std::stringstream save_path;
    save_path << "fit_plots/fit_plot_" << p.super_column_id << "_" << p.super_pixel_id << "_" << p.pixel_id << ".png";
    c1->SaveAs(save_path.str().c_str());

    delete c1;
}

return calibrationParams;

}
`

The code without:

`std::unordered_map<PixelPosition, CalibrationFunction> calcCalibrationParams(const >BiasedChargeMap& biasedCharges, TTree* data) {
std::unordered_map<PixelPosition, CalibrationFunction> calibrationParams;

for (auto& [p, line] : biasedCharges) {
    // std::cout << '{' << p.super_column_id << ",sp" << p.super_pixel_id << ",pix" << p.pixel_id << "} -> ";

    // for (auto& point : line) {
    //     std::cout << "{tp: " << point.bias << ", adc: " << point.charge << "}, ";
    // }

    // std::cout << '\n';

    if (line.size() < 9) {
        continue;
    }

    std::stringstream function_def;
    function_def << line[line.size() - 1].charge;
    function_def << "*pow((1-exp(-[0]*x)),[1])";

    auto to_string = function_def.str();

    TF1 ex("tf_name", to_string.c_str());
    ex.SetParameters(1./12., 1);


    // Create a TGraph to plot the data points and the fit
    TGraph graph;
    
    // Add the real data points to the graph
    for (std::size_t i = 0; i < line.size(); ++i) {
        auto& point = line[i];
        graph.SetPoint(i, point.bias, point.charge);
    }

    // Add a synthetic point at (0,0)
    graph.SetPoint(graph.GetN(), 0, 0);
    
    std::stringstream predicate;

    predicate << "pixel_id == " << p.pixel_id
              << "&& super_column_id == " << p.super_column_id
              << "&& super_pixel_id == " << p.super_pixel_id;
    
   
    data->Fit("tf_name", "charge:qtp", predicate.str().c_str(), "W0N");

    //graph.Fit(&ex, "W"); 

    double chi_square_ndf = ex.GetChisquare() / ex.GetNDF();  
    calibrationParams[p] = { ex.GetParameter(0), ex.GetParameter(1), line[line.size() - 1].charge + 0.001f, chi_square_ndf };      

    // Plot the graph and fit
    TCanvas* c1 = new TCanvas("c1", "Calibration Fit", 800, 800);
    graph.SetMarkerStyle(20);
    graph.SetMarkerSize(0.5);
    graph.SetMarkerColor(kBlue);
    graph.Draw("AP");
    ex.Draw("same");

    // Set axis ranges
    graph.GetXaxis()->SetLimits(0, 60);
    graph.SetMinimum(0);
    graph.SetMaximum(15);

    // Add text annotations
    TLatex latex;
    latex.SetNDC();
    latex.SetTextSize(0.03);

    std::stringstream func_text, chi_square_text;
    func_text << "f(qtp) = " << line[line.size() - 1].charge + 0.001f
              << " * (1 - exp(-" << ex.GetParameter(0) << " * qtp))^{" << ex.GetParameter(1) << "}";
    chi_square_text << "#chi^{2}/NDF = " << chi_square_ndf;

    latex.DrawLatex(0.15, 0.85, func_text.str().c_str());
    latex.DrawLatex(0.15, 0.80, chi_square_text.str().c_str());

    // Save the plot
    std::stringstream save_path;
    save_path << "fit_plots/fit_plot_" << p.super_column_id << "_" << p.super_pixel_id << "_" << p.pixel_id << ".png";
    c1->SaveAs(save_path.str().c_str());

}

return calibrationParams;

}`

an example of my errors when they exist:

charge = 6, samples = 183, error = 0 charge = 8, samples = 183, error = 0 charge = 8, samples = 183, error = 0 charge = 10, samples = 183, error = 0 charge = 11, samples = 183, error = 0 charge = 11.456, samples = 182, error = 0.0400562 charge = 12, samples = 183, error = 0 charge = 12, samples = 183, error = 0 charge = 12, samples = 183, error = 0 charge = 12, samples = 183, error = 0 charge = 12, samples = 183, error = 0 charge = 12, samples = 183, error = 0 charge = 12, samples = 183, error = 0 charge = 12, samples = 183, error = 0 charge = 12, samples = 183, error = 0

Note that the png with the appeared red line is with the TGRAPHerrors. The other is with th tf1, it doesnt seem to be able to plot the fitted function… although note that also the data appear differently while are the same! Do you know why is that?! The data are exactly the same!

Note that the code without the errors that i add above its an effort of mine to try to plot and see what i fit from the original code which is this. the following code and the code without errors abo e give the same results:

std::unordered_map<PixelPosition, CalibrationFunction> calcCalibrationParams(const BiasedChargeMap& biasedCharges, TTree* data) {
    std::unordered_map<PixelPosition, CalibrationFunction> calibrationParams;
    
    for (auto& [p, line] : biasedCharges) {
        // std::cout << '{' << p.super_column_id << ",sp" << p.super_pixel_id << ",pix" << p.pixel_id << "} -> ";

        // for (auto& point : line) {
        //     std::cout << "{tp: " << point.bias << ", adc: " << point.charge << "}, ";
        // }

        // std::cout << '\n';

        if (line.size() < 9) {
            continue;
        }

        std::stringstream function_def;
        function_def << line[line.size() - 1].charge;
        function_def << "*pow((1-exp(-[0]*x)),[1])";

        auto to_string = function_def.str();

        TF1 ex("tf_name", to_string.c_str());
        ex.SetParameters(1./12., 1);

        std::stringstream predicate;

        predicate << "pixel_id == " << p.pixel_id
                  << "&& super_column_id == " << p.super_column_id
                  << "&& super_pixel_id == " << p.super_pixel_id;
            
        data->Fit("tf_name", "charge:qtp", predicate.str().c_str(), "WQ0N");

        double chi_square_ndf = ex.GetChisquare() / ex.GetNDF();  
        calibrationParams[p] = { ex.GetParameter(0), ex.GetParameter(1), line[line.size() - 1].charge + 0.001f, chi_square_ndf };      
    }

    return calibrationParams;
}

Thank you very much for your quick answers. I have this problem for months… literally i dont know what else to do…
At last I want the fitting function to pass from (0,0). The different chi2/ndf is not due to the (0,0) point.
I am attaching you the three full code versions. As you will see I do thousands of fittings like these. The above was an example of them.
main_original_plot_effort.cpp (18.7 KB)
main_errors.cpp (17.6 KB)
main_original.cpp (16.9 KB)

---

Please read tips for efficient and successful posting and posting code

ROOT Version: Not Provided
Platform: Not Provided
Compiler: Not Provided

---

[Danilo]

Hi Elena,

This is a lot of code. Could you share with us a simple reproducer, 1 set of points, 1 function, a few tens of lines of code, so that we can discuss the matter?

Best,
D

[Elena_Veli]

Hi,
thank you for replying.
The description of the beginning, actually describes the matter.
a much more simplified code with the tgrapherror:

#include <TFile.h>
#include <TTree.h>
#include <TF1.h>
#include <TGraphErrors.h>
#include <TCanvas.h>
#include <TLatex.h>
#include <iostream>
#include <math.h>
#include <sstream>

// Define the structure for calibration function
struct CalibrationFunction {
    Double_t a;
    Double_t b;
    Double_t c;
    Double_t chi_square_ndf;
};

// Function to fit data with errors and calculate calibration parameters
CalibrationFunction fitDataWithErrors() {
    // Provided x and y data
    Double_t x[] = {14.34, 18.15, 21.78, 25.29, 31.95, 38.12, 43.53, 49.38, 50.55, 51.31, 53.38, 54.25, 54.46, 54.95, 55.08};
    Double_t y[] = {6, 7, 8, 9, 11, 11, 12, 12, 12, 12, 12, 12, 12, 12, 12};
    Double_t errors[] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.1, 0, 0};
    int n = sizeof(x) / sizeof(x[0]);

    // Create a TGraphErrors
    TGraphErrors *graph = new TGraphErrors(n, x, y, 0, errors);

    // Define the fitting function
    TF1 *fitFunc = new TF1("fitFunc", "[2]*pow((1-exp(-[0]*x)),[1])", 0, 60);
    fitFunc->SetParameters(1.0/12.0, 1.0, 12.0);  // Initial parameter guesses

    // Perform the fit
    graph->Fit("fitFunc", "W");

    // Extract the fit parameters
    CalibrationFunction calibFunc;
    calibFunc.a = fitFunc->GetParameter(0);
    calibFunc.b = fitFunc->GetParameter(1);
    calibFunc.c = fitFunc->GetParameter(2);
    calibFunc.chi_square_ndf = fitFunc->GetChisquare() / fitFunc->GetNDF();

    // Print the fit results
    printf("Fit results:\n");
    printf("a = %f\n", calibFunc.a);
    printf("b = %f\n", calibFunc.b);
    printf("c = %f\n", calibFunc.c);
    printf("Chi-square/NDF = %f\n", calibFunc.chi_square_ndf);

    // Draw the graph with the fit
    TCanvas *c1 = new TCanvas("c1", "Fit with Errors", 800, 600);
    graph->SetMarkerStyle(20);
    graph->Draw("AP");
    fitFunc->Draw("Same");

    // Annotate the canvas
    TLatex latex;
    latex.SetNDC();
    latex.SetTextSize(0.03);
    latex.DrawLatex(0.15, 0.85, "Fit Function: c*(1-exp(-a*x))^b");

    c1->SaveAs("fit_with_errors.png");

    // Clean up
    delete c1;
    delete graph;
    delete fitFunc;

    return calibFunc;
}

int main(int argc, char **argv) {
    // Directly call the function to perform the fit
    CalibrationFunction calibFunc = fitDataWithErrors();

    return 0;
}

without the error:

#include <TF1.h>
#include <TGraph.h>
#include <iostream>
#include <sstream>

struct CalibrationFunction {
    Double_t a;
    Double_t b;
    Double_t c;
    Double_t chi_square_ndf;
};

CalibrationFunction fitData() {
    Double_t x[] = {14.34, 18.15, 21.78, 25.29, 31.95, 38.12, 43.53, 49.38, 50.55, 51.31, 53.38, 54.25, 54.46, 54.95, 55.08};
    Double_t y[] = {6, 7, 8, 9, 11, 11, 12, 12, 12, 12, 12, 12, 12, 12, 12};
    int n = sizeof(x) / sizeof(x[0]);

    // Create a TGraph to hold the data
    TGraph graph(n, x, y);

    // Define the fitting function
    std::stringstream function_def;
    function_def << 12 << "*pow((1-exp(-[0]*x)),[1])";
    TF1 ex("fitFunc", function_def.str().c_str(), 0, 60);
    ex.SetParameters(1.0 / 12.0, 1.0);

    // Perform the fit
    graph.Fit(&ex, "W");

    // Extract the fit parameters
    CalibrationFunction calibFunc;
    calibFunc.a = ex.GetParameter(0);
    calibFunc.b = ex.GetParameter(1);
    calibFunc.c = 12.0;  // Fixed value
    calibFunc.chi_square_ndf = ex.GetChisquare() / ex.GetNDF();

    // Print the fit results
    std::cout << "Fit results:" << std::endl;
    std::cout << "a = " << calibFunc.a << std::endl;
    std::cout << "b = " << calibFunc.b << std::endl;
    std::cout << "c = " << calibFunc.c << std::endl;
    std::cout << "Chi-square/NDF = " << calibFunc.chi_square_ndf << std::endl;

    return calibFunc;
}

int main() {
    fitData();
    return 0;
}

Moreover the figures above describe also the matter.
Thank you

[dastudillo]

The reduced examples give exactly the same graphs (but different fit results) (I only set the same minimum and maximum for both, to 5 and 13, to better compare the points).

So if your graphs are different I guess there’s some difference in the data you are putting in them, so better use exactly the same code to read and input the data in both cases (be 100% sure! print the values you are putting into the graphs and check one by one if you have to).

Update:
As for the fit results, in the second case you are already fixing one parameter to 12 (function_def << 12 << "*pow((1-exp(-[0]*x)),[1])";) whereas it is a fit variable in the first case. So again, use the same code in both and you will get the same results.

[Elena_Veli]

Hi
Thank you for you reply.
My data is exactly the same. My bad is that in the second message when I was asked to create indipended programs, I still used TGraph in the one case to fit, while in the original code I am not using it and that is why I have different fit results. If you remove the initial parameter, 12, and set it fixed at 12, the 2 above codes will give the same result. And therefore they are not representative for my problem, so sorry for the confusion. Again this code in my original program:

    for (auto& [p, line] : biasedCharges) {
        // std::cout << '{' << p.super_column_id << ",sp" << p.super_pixel_id << ",pix" << p.pixel_id << "} -> ";

        // for (auto& point : line) {
        //     std::cout << "{tp: " << point.bias << ", adc: " << point.charge << "}, ";
        // }

        // std::cout << '\n';

        if (line.size() < 9) {
            continue;
        }

        std::stringstream function_def;
        function_def << line[line.size() - 1].charge;
        function_def << "*pow((1-exp(-[0]*x)),[1])";

        auto to_string = function_def.str();

        TF1 ex("tf_name", to_string.c_str());
        ex.SetParameters(1./12., 1);

        std::stringstream predicate;

        predicate << "pixel_id == " << p.pixel_id
                  << "&& super_column_id == " << p.super_column_id
                  << "&& super_pixel_id == " << p.super_pixel_id;
            
        data->Fit("tf_name", "charge:qtp", predicate.str().c_str(), "W");

        double chi_square_ndf = ex.GetChisquare() / ex.GetNDF();  
        calibrationParams[p] = { ex.GetParameter(0), ex.GetParameter(1), line[line.size() - 1].charge + 0.001f, chi_square_ndf };      
    }

gives ddifferent result from this code in my other original program:

    for (auto& [p, line] : biasedCharges) {
        if (line.size() < 9) {  // Ensure there's enough data to perform a meaningful fit
            continue;
        }

        // Create TGraphErrors to hold the data points with error bars
        TGraphErrors graph(line.size());

        for (size_t i = 0; i < line.size(); ++i) {
            // Use the mean charge and error directly from the ChargeAtBias structure
            double mean_charge = line[i].charge;
            double error = line[i].error;

            std::cout <<"qtp" << qtp << "charge = " << mean_charge << ", samples = " << line[i].samples
                      << ", error = " << error << '\n';

        }

        // Define the fitting function, where [2] is dynamically set to the last charge
        double last_charge = line[line.size() - 1].charge;
        std::stringstream function_def;
        function_def << last_charge << "*pow((1-exp(-[0]*x)),[1])";

        TF1 ex("tf_name", function_def.str().c_str(), 0, 60);
        ex.SetParameters(1./12., 1.0);  // Initial guesses for [0] and [1]

        // Perform the fit
        graph.Fit(&ex, "W");

        double chi_square_ndf = ex.GetChisquare() / ex.GetNDF();
        calibrationParams[p] = { ex.GetParameter(0), ex.GetParameter(1), last_charge + 0.001f, chi_square_ndf };

        // Plot the graph with error bars
        TCanvas* c1 = new TCanvas("c1", "Calibration Fit", 800, 800);  // Create a square canvas
        graph.SetMarkerStyle(20); // Marker style
        graph.SetMarkerSize(0.5); // Marker size
        graph.SetMarkerColor(kBlue); // Marker color
        graph.Draw("AP"); // Draw points with error bars
        ex.Draw("same");

        // Set axis ranges
        graph.GetXaxis()->SetLimits(0, 60);  // Set x-axis limits
        graph.SetMinimum(0);                 // Set y-axis minimum
        graph.SetMaximum(15);                // Set y-axis maximum to match the x-axis range

        // Add text annotations
        TLatex latex;
        latex.SetNDC();
        latex.SetTextSize(0.03);

        std::stringstream func_text, chi_square_text;
        func_text << "f(qtp) = " << last_charge << " * (1 - exp(-" << ex.GetParameter(0) << " * qtp))^{" << ex.GetParameter(1) << "}";
        chi_square_text << "#chi^{2}/NDF = " << chi_square_ndf;

        latex.DrawLatex(0.15, 0.85, func_text.str().c_str());
        latex.DrawLatex(0.15, 0.80, chi_square_text.str().c_str());

        // Save the plot
        std::stringstream save_path;
        save_path << "fit_plots/fit_plot_" << p.super_column_id << "_" << p.super_pixel_id << "_" << p.pixel_id << ".png";
        c1->SaveAs(save_path.str().c_str());

        delete c1;
    }

Could you maybe explain why, with the TGrapherror is so different? My errors are minimal to zero and they are correctly calculated. My data are the same and accessed the same way and it is crosschecked. I have also seen other posts in the forum with different results with the tgrapherror and the other fit and while it does not seem to fit that good the ch2/ndf is extremely low. And the answer is usually fit with W, which honestly I do perform the fit with W. What is in your opinion the best way to fit when I have these small errors? which value should I trust? If you can also replicate an independent program for the no error/tgraph case, it would be great since what I do currently to replicate my no error case, in the independent program, in my second message, doesn’t seem to compile.

Thank you very much again

[Elena_Veli]

Hi,
I would appreciate if someone could provide any kind of help or insight!
Thanks,
Elena

[couet]

may be @moneta can help