Free(): invalid next size(fast)//malloc(): memory corruption

Hello dear Rooters,

I wrote a program that finds all the hits and identifies those which overlap from data taken with an oscilloscope. In event::reduce() (event class corresponds to one file-profile and has members as vector, noise, threshold, etc, where hit is another class corresponding to a single pulse - I can attach the scripts under request) I identify the superpositions, each hit has a label member which sorts them out (same group number means pulses overlapping). Then, in analysis.cxx, the code below, I reduce all the hits. I fit the pulses overlapping and create two event branches, one with the original info (hits without fit parameters) and another one with hits with the fit parameters, a branch TMultiGraph with the graphs of the superpositions and another branch TClonesArray with TFitResult instances of the fits. Apparently everything works well when I process a group of files with only one file with a superposition (file 12 is first file with a superposition). So, when I run ./analysis.exe /eth/ethraid5/home/dnarrias/data_Daniel-Thomas/15Nov/Waveforms MI-1_687_ampl 20 RT_687_test.root, the program works, but when I run on 30 instead of 20 files, when processing file 29 with superpositions, the programs crashes. May the problem be an allocation error/accessing non allocated memory one?. Unfortunately, I don’t see where the problem is in the code… The error message I get is the following one:

The analysis.cxx script is below.


//creating fit Functor
class  exp_functor {
public:

        exp_functor(int nn){ nHits = nn;}

        double operator() (const double *x, const double *p) {
                double fit = 0.;
                for(int i = 0; i < nHits; i++)  fit += hit(x[0], p[3*i], p[3*i+1], p[3*i+2]);
                return fit;
        }

        //this is the fit function for one hit
        double hit(double x, double p0, double p1, double p2){

                double h = p0*(x-p1)*exp(-(x-p1)/p2);
                if(x < p1) return 0.;
                        else return h;
        }

private:
        int nHits;
};


/*********************************
See argument pass by reference: TFitResultPtr*& fitResult.
I pass it by reference as here it gets the content I want to write into the TFitResultPtr branch (with tree->Fill() command in another function).
Same for TMultiGraph*& MultiGOverlap
***********************************/

double* OverlapSplit(double* inputPars, int nHitsOverlap, double lastWidth, int*& wave, double vGain, double vOffset, int nevent, TMultiGraph*& MultiGOverlap, TClonesArray& fitResultArray, int nOverlap, double pedestal){
//nOverlap is the number of an overlap in an event.     
// inputPars[n]:StartTime, inputPars[n+1]:riseTime, inputPars[n+2]:maximum. n: number of hit in the superposition. Only vertical parameters are in V, horizontal parameters (times) are in ns (sampling time 4ns).

        double* outputPars = new double[40];
// outputPars[n]: riseTime, outputPars[n+1]:scalar (= maximum* e / riseTime), output[n+2]:offset (startTime).
        double *hitOverlap = new double[200];//let's assume a hit has 50 samples, 50*4=200ns width. If we consider superpositions with up to 4 hits, 50*4=200 samples:number of elements in this array.  

        const double e = exp(1.0);

        double StartOverlap = *inputPars;

        *hitOverlap = 0.;

        //At this point start's and width's are double but integer numbers, ex. 4., 12., etc.

        for(int ii = (int)(StartOverlap/4); ii <= (int)(*(inputPars + 3*(nHitsOverlap-1))) + (int)(lastWidth/4); ii++) *(hitOverlap + ii + 1 - (int)(StartOverlap/4)) = (vGain*(double)wave[ii] - vOffset)-pedestal;
        for(int ii = (int)(*(inputPars + 3*(nHitsOverlap-1))) + (int)(lastWidth/4) + 1; ii < (int)(StartOverlap/4) + 199; ii++) *(hitOverlap + ii + 1 - (int)(StartOverlap/4)) = 0.;

        double *times = new double[200];
        for(int ii = 0; ii < 200; ii++) *(times + ii) = StartOverlap + 4*(ii - 1);//time in ns, [StartOverlap-4, StartOverlap + 199*4).

        TGraph *GOverlap = new TGraph(200, times, hitOverlap);
        char *overlapName = new char[200];
        sprintf(overlapName, "startOverlap%dns_nOverlap%d_event%d", StartOverlap, nOverlap, nevent);
        GOverlap->SetName(overlapName);
        char *overlapTitle = new char[200];
        char *startTimesTitle = new char[180];
        sprintf(startTimesTitle, "Start Hit1: %dns, Start Hit%d: %dns ", *inputPars, nHitsOverlap, *(inputPars + 3*(nHitsOverlap-1)));
        sprintf(overlapTitle, "%s; nOverlap: %d; event: %d", startTimesTitle, nOverlap, nevent);
        GOverlap->SetTitle(overlapTitle);
        GOverlap->SetMarkerStyle(20);
        GOverlap->SetMarkerSize(1);
        GOverlap->SetMarkerColor(1);

        char *fitFunctName = new char[80];
        sprintf(fitFunctName, "fitfunction_startOverlap%dns_event%d", StartOverlap, nevent);

        exp_functor *fit = new exp_functor(nHitsOverlap);

        TF1 *fitFunction = new TF1("fitFunction", fit, StartOverlap -4, StartOverlap -4 + 4*100, 3*nHitsOverlap, "exp_functor");//+4*200 but we assume last half is empty data

        for(int i = 0; i < nHitsOverlap; i++){
                 fitFunction->SetParameter( 3*i, e*(*(inputPars + 3*i + 2))/(*(inputPars + 3*i + 1)));
                 fitFunction->SetParameter( 3*i + 1, *(inputPars + 3*i));
                 fitFunction->SetParameter( 3*i + 2, *(inputPars + 3*i + 1));
        }

        char *ParName = new char[15];
        char *scalar = new char[10];
        char *offset = new char[10];
        char *riseTime = new char[10];

        sprintf(scalar, "scalar");
        sprintf(offset, "offset");
        sprintf(riseTime, "riseTime");

        for(int i = 0; i < nHitsOverlap; i++){
                sprintf(ParName, "%s%d", scalar, i);
                fitFunction->SetParName(3*i, ParName);
                sprintf(ParName, "%s%d", offset, i);
                fitFunction->SetParName(3*i + 1, ParName);
                sprintf(ParName, "%s%d", riseTime, i);
                fitFunction->SetParName(3*i + 2, ParName);
        }

        fitFunction->SetLineColor(2);

        TFitResultPtr fitResultPtr;
        fitResultPtr = GOverlap->Fit(fitFunction, "RS");

        new(fitResultArray[nOverlap]) TFitResult((*fitResultPtr.Get()));

        MultiGOverlap->Add(GOverlap);

        //we could get all the parameters in one line: const double* TFitResult::GetParams(), but we want to keep our dynamically allocated pointer outputPars.
        for(int i = 0; i < nHitsOverlap; i++){
                *(outputPars + 3*i) = fitFunction->GetParameter(3*i + 2);
                *(outputPars + 3*i + 1) = fitFunction->GetParameter(3*i);
                *(outputPars + 3*i + 2) = fitFunction->GetParameter(3*i + 1);

        }

        delete[] overlapTitle;
        delete[] startTimesTitle;
        delete[] fitFunctName;
        delete[] ParName;
        delete[] scalar;
        delete[] offset;
        delete[] riseTime;
        fitFunction->TF1::~TF1();

        delete[] overlapName;
        delete[] times;
        delete[] hitOverlap;
        return outputPars;
}



//outputs a root file with reduced events
int main(int argc, char *argv[]){

        if(argc != 5){
                fprintf(stderr, "%s <inputFolderName> <inputFileBaseName> <numberOfEvents> <outputFileName>\n\n", argv[0]);
                return 1;
        }
        int ii, cnt = 0, nn = atoi(argv[3]);
        FILE * file;
        int *wave;
        double vGain, vOffset;
        TFile * out = new TFile(argv[4], "RECREATE");
        TTree * tree = new TTree("tree", argv[4]);

        waveform * wa = new waveform();
        event * ev = new event();
        event * rEv = new event();
        //OBS: TClonesArray and TObjArray can only collect classes which inherit from TObject. TFitResultPtr doesn't inherit from TObject. On the other hand, TFitResult does inherit from TObject. 
        TClonesArray *PfitResultArray = new TClonesArray("TFitResult", 6);
        TClonesArray &fitResultArray = *PfitResultArray;
        TMultiGraph *MultiGOverlap = new TMultiGraph("MultiGOverlaps", "MultiGOverlaps");//default constructor
        tree->Branch("Event", "event", &ev, 32000, 0);
        tree->Branch("reducedEvent", "event", &rEv, 32000, 0);
        tree->Branch("FitResultArray", "TClonesArray", &PfitResultArray, 32000, 0);
        tree->Branch("MultiGraphOverlap", "TMultiGraph", &MultiGOverlap, 32000, 0) ;

        double* inputPars = new double[40];
        double* outputPars = new double[40];
        int nHitsOverlap = 1;
        int nOverlap = 0;
        double lastWidth;
        vector<hit> hits;
        hit Hit, prevHit;
        double pedestal;

       const double e = exp(1.0);
        const double a = exp(-5.0);
        double scalar, riseTime, offset, integral, maximum;

        char fileName[128];
        for(ii = 0; ii < nn; ii++){

                sprintf(fileName, "%s/C1%s%05d.trc", argv[1], argv[2], ii);

                if(ii < 60) continue; //.trc file is corrupt            

                if((file = fopen(fileName, "r")) == NULL){
                        fclose(file);
                        continue;
                }
                wa->importLeCroyBinary(file);
                fclose(file);
                if((wa->getWave()) == NULL){
                        continue;
                }
                ev->reduce(wa, 32, 2., 6);// 32*4ns = 128ns < pedestalWidth; 2.*noise = threshold; 6*4ns=24ns < minimumHitWidth accepted. 

                *rEv = *ev;

                if(ev->getnHits() > 1){

                        wave = wa->getWave();
                        vGain = wa->getyDelta();
                        vOffset = wa->getyOffset();
                        hits = ev->getHits();
                        pedestal = ev->getPedestal();
                        for(int j = 1; j < ev->getnHits(); j++){
                                prevHit = hits.at(j-1);
                                Hit = hits.at(j);
                                if(prevHit.getGroupOverlap() == Hit.getGroupOverlap()){

                                        *(inputPars + 3*(nHitsOverlap-1)) = prevHit.getStart();//in ns units
                                        *(inputPars + 3*(nHitsOverlap-1) + 1) = prevHit.getRiseTime();
                                        *(inputPars + 3*(nHitsOverlap-1) + 2) = prevHit.getMaximum();
                                        *(inputPars + 3*nHitsOverlap) = Hit.getStart();
                                        *(inputPars + 3*nHitsOverlap + 1) = Hit.getRiseTime();
                                        *(inputPars + 3*nHitsOverlap + 2) = Hit.getMaximum();

                                        lastWidth = Hit.getWidth();
                                        nHitsOverlap++;

                                        if(j == (ev->getnHits()-1)){
                                                printf("nHits: %d, event %d\n", ev->getnHits(), ii);

                                                outputPars = OverlapSplit(inputPars, nHitsOverlap, lastWidth, wave, vGain, vOffset, ii, MultiGOverlap, fitResultArray, nOverlap, pedestal);

                                                printf("Hola");

                                                for(int k = 0; k < nHitsOverlap; k++){

                                                        riseTime = *(outputPars + 3*k);
                                                        scalar = *(outputPars + 3*k + 1);
                                                        offset = *(outputPars + 3*k + 2);
                                                        maximum = scalar*riseTime/e + pedestal;
                                                        integral = scalar*riseTime*riseTime*(1-6*a)/5 + 5*riseTime*pedestal/5;//to obtain the charge, the integral must be divided by 50Ohm, we divide by 5 and the charge is in [e-10 C] units ---- With the model we use for hit fitting, time(9%ofMaximum)=5.022*riseTime, which we consider as hitWidth. By adding the last term to the integral, we're allowed to plot charge with pedestal charge and then correct the charge.

                                                        printf("Pedestal: %f V\n", pedestal);
                                                        printf("Start%d\tnOverlap%d\tevent%d\n", (int)(*inputPars), nOverlap, ii);
                                                        printf("-------------------- Hit%d\tStart: (a)%f (b)%f\tPeakTime: (a)%f (b)%f\tRisetime: (a)%f (b)%f\tWidth: (a)%f (b)%f\n-------------------------\tMaximum: (a)%f (b)%f\tIntegral: (a)%f (b)%f\n", k, (hits.at(j-nHitsOverlap+k+1)).getStart(), offset, (hits.at(j-nHitsOverlap+k+1)).getPeakTime(), offset + riseTime, (hits.at(j-nHitsOverlap+k+1)).getRiseTime(), riseTime, (hits.at(j-nHitsOverlap+k+1)).getWidth(), 5*riseTime, (hits.at(j-nHitsOverlap+k+1)).getMaximum(), maximum, (hits.at(j-nHitsOverlap+k+1)).getIntegral(), integral);

                                                        hit tmp;
                                                        tmp.init(offset, riseTime, 5*riseTime, offset + riseTime, maximum, integral, prevHit.getGroupOverlap());
                                                        rEv->replaceHit(j-nHitsOverlap+k+1, tmp);
                                                }
                                        }
                                } else if(nHitsOverlap > 1){
                                                printf("nHits: %d, event %d\n", ev->getnHits(), ii);

                                                outputPars = OverlapSplit(inputPars, nHitsOverlap, lastWidth, wave, vGain, vOffset, ii, MultiGOverlap, fitResultArray, nOverlap, pedestal);

                                                for(int k = 0; k < nHitsOverlap; k++){

                                                        riseTime = *(outputPars + 3*k);
                                                        scalar = *(outputPars + 3*k + 1);
                                                        offset = *(outputPars + 3*k + 2);
                                                        maximum = scalar*riseTime/e + pedestal;
                                                        integral = scalar*riseTime*riseTime*(1-6*a)/5 + 5*riseTime*pedestal/5;

                                                        printf("Pedestal: %f V\n", pedestal);
                                                        printf("Start%d\tnOverlap%d\tevent%d\n",(int)(*inputPars), nOverlap, ii);
                                                        printf("-------------------- Hit%d\tStart: (a)%f (b)%f\tPeakTime: (a)%f (b)%f\tRisetime: (a)%f (b)%f\tWidth: (a)%f (b)%f\n-------------------------\tMaximum: (a)%f (b)%f\tIntegral: (a)%f (b)%f\n", k, (hits.at(j-nHitsOverlap+k)).getStart(), offset, (hits.at(j-nHitsOverlap+k)).getPeakTime(), offset + riseTime, (hits.at(j-nHitsOverlap+k)).getRiseTime(), riseTime, (hits.at(j-nHitsOverlap+k)).getWidth(), 5*riseTime, (hits.at(j-nHitsOverlap+k)).getMaximum(), maximum, (hits.at(j-nHitsOverlap+k)).getIntegral(), integral);

                                                        hit tmp;
                                                        tmp.init(offset, riseTime, 5*riseTime, offset + riseTime, maximum, integral, prevHit.getGroupOverlap());//initializes a hit
                                                        rEv->replaceHit(j-nHitsOverlap+k, tmp);

                                                }

                                                nOverlap++;
                                                nHitsOverlap = 1;
                                        }
                        }

                }

                nOverlap = 0;
                nHitsOverlap = 1;

                if(ii%1000 == 0) fprintf(stderr, "%6d\t%6d\r", cnt, nn);
                cnt++;


                tree->Fill();
                //Reinitialize fitResult and MultiGOverlap.
                fitResultArray.Clear();
                MultiGOverlap->TMultiGraph::~TMultiGraph();
                MultiGOverlap = new TMultiGraph("MultiGOverlaps", "MultiGOverlaps");
                hits.clear();

        }

        fprintf(stderr, "%6d\t%6d\n", cnt, nn);
        delete wa;
        delete ev;
        delete rEv;

        tree->Print();
        tree->Write();
        out->Close();

        return 0;
}
                                 

Below find a bt from gdb.

[quote]#0 0x00f2c402 in __kernel_vsyscall ()
#1 0x053eedf0 in raise () from /lib/libc.so.6
#2 0x053f0701 in abort () from /lib/libc.so.6
#3 0x05425bcb in __libc_message () from /lib/libc.so.6
#4 0x0542dee5 in _int_free () from /lib/libc.so.6
#5 0x0542e329 in free () from /lib/libc.so.6
#6 0x026ba5c1 in operator delete(void*) () from /usr/lib/libstdc++.so.6
#7 0x026ba61d in operator delete () from /usr/lib/libstdc++.so.6
#8 0x058229a5 in TString::~TString() () from /eth/sw/ROOT/5.30.00/i386-linux/root/lib/libCore.so
#9 0x057fde77 in TPluginManager::LoadHandlersFromPluginDirs(char const*) () from /eth/sw/ROOT/5.30.00/i386-linux/root/lib/libCore.so
#10 0x057ff638 in TPluginManager::FindHandler(char const*, char const*) () from /eth/sw/ROOT/5.30.00/i386-linux/root/lib/libCore.so
#11 0x035a6cb7 in ROOT::Math::factory::CreateMinimizer(std::basic_string<char, std::char_traits, std::allocator > const&, std::basic_string<char, std::char_traits, std::allocator > const&) () from /eth/sw/ROOT/5.30.00/i386-linux/root/lib/libMathCore.so
#12 0x035a83c1 in ROOT::Fit::FitConfig::CreateMinimizer() () from /eth/sw/ROOT/5.30.00/i386-linux/root/lib/libMathCore.so
#13 0x035b3d2b in ROOT::Fit::Fitter::DoInitMinimizer() () from /eth/sw/ROOT/5.30.00/i386-linux/root/lib/libMathCore.so
#14 0x035b4a48 in ROOT::Fit::Fitter::DoMinimization(ROOT::Math::IBaseFunctionMultiDim const&, ROOT::Math::IBaseFunctionMultiDim const*) ()
from /eth/sw/ROOT/5.30.00/i386-linux/root/lib/libMathCore.so
#15 0x035b6348 in ROOT::Fit::Fitter::DoLeastSquareFit(ROOT::Fit::BinData const&) () from /eth/sw/ROOT/5.30.00/i386-linux/root/lib/libMathCore.so
#16 0x003eb3e4 in TFitResultPtr HFit::Fit(TGraph*, TF1*, Foption_t&, ROOT::Math::MinimizerOptions const&, char const*, ROOT::Fit::DataRange&) ()
from /eth/sw/ROOT/5.30.00/i386-linux/root/lib/libHist.so
#17 0x003e4047 in ROOT::Fit::FitObject(TGraph*, TF1*, Foption_t&, ROOT::Math::MinimizerOptions const&, char const*, ROOT::Fit::DataRange&) ()
from /eth/sw/ROOT/5.30.00/i386-linux/root/lib/libHist.so
#18 0x00458cc0 in TGraph::Fit(TF1*, char const*, char const*, double, double) () from /eth/sw/ROOT/5.30.00/i386-linux/root/lib/libHist.so
#19 0x0804c8b9 in OverlapSplit(double*, int, double, int*&, double, double, int, TMultiGraph*&, TClonesArray&, int, int, double) ()
#20 0x0804d8bd in main ()[/quote]

but it doesn’t help me much…

When I exclude the first 60 files and run the executable until the file number 95, the execution finishes well and it processes fine all the files with superpositions, but when run on files until the number 100, it crashes. The error message I get is the following:

and the bt is

I’m puzzled…

I know that only with this information it’s difficult for you to figure out how to fix the problem. Please let me know if interested and I send you the whole macro so you can run the program. It’s a hugeee program with all the class implementations ( event, hit) and other classes to get data from binary files and handle the data ( waveform, readLeCroyBinary classes, etc), let alone the necessary data binary files.

I hope that someone looking for challenging code/ROOT problems gets interested in this one.
Thanks in advance.
Daniel

I found the error in my code, I was trying to access unallocated memory. Just a remark: I don’t think gdb gives you so many hints on the errors, because it doesn’t really lead you to the line where the error is or what the error is, it just gives you complicated messages related to internal processing but not really related to the code itself… just a newbie opinion here perhaps…

Try to run your code using valgrind (and carefully study messages that appear in the beginning of the output):
valgrind --tool=memcheck --leak-check=full [–show-reachable=yes] [–num-callers=50] [–track-origins=yes] [–db-attach=yes] --suppressions=root-config --etcdir/valgrind-root.supp root-config --bindir/root.exe -l -q 'YourMacro.cxx[++][(Any, Parameters, You, Need)]'
or:
valgrind --tool=memcheck --leak-check=full [–show-reachable=yes] [–num-callers=50] [–track-origins=yes] [–db-attach=yes] --suppressions=root-config --etcdir/valgrind-root.supp YourExecutable [Any Options You Need]
(Note: the “–show-reachable=yes” option will give you too many warnings, I believe.)

Thanks! I’ll try it out next time.