Memory issues with Arrays

The following code seems to stomp on some memory, causing some “heisenbug” like problems. Any ideas where I am going wrong?

//header

     NCCoherentFitter(string refit_file_name, int fit_num=0, int nc_coh_norm_label=2, int had_sys_num=0, string had_sys_name="no_sys_set");
     .
     .
     .
    TH1F**  covar_1D_plots;
    TH2F*** covar_2D_plots;
    TLine** one_sigma;
    TMarker*** best_fit;
    TH2F*** covar_2D_plots_clone;
    double** covariance;
      .
      .
      .
//.cxx file
NCCoherentFitter::NCCoherentFitter(string refit_file_name, int fit_num, int nc_coh_norm_label, int had_sys_num, string had_sys_name){
     .
     .
     .
  const int num_of_params = 3+had_sys_num;
  covariance = new double*[num_of_params];
  one_sigma = new TLine*[num_of_params];
  best_fit = new TMarker**[num_of_params];
  covar_2D_plots_clone = new TH2F**[num_of_params];
  covar_1D_plots = new TH1F*[num_of_params];
  covar_2D_plots = new TH2F**[num_of_params];
  for(int k = 0; k<had_sys_num+3; k++){
    covariance[k] = new double[num_of_params];
    best_fit[k] = new TMarker*[num_of_params];
    covar_2D_plots_clone[k] = new TH2F*[3+had_sys_num];
    covar_2D_plots[k] = new TH2F*[3+had_sys_num];
    for(int j = 0; j<had_sys_num+3; j++){
      covariance[k][j] = 0.0;
      if(j==k) covar_1D_plots[k] = new TH1F(Form("covar_%d_%d",k,k),Form("covar_%d_%d",k,k),bins[k],min[k],max[k]);
      covar_2D_plots[k][j] = new TH2F(Form("covar_%d_%d",k,j),Form("covar_%d_%d",k,j),bins[k],min[k],max[k],bins[j],min[j],max[j]);
    }
  }
     .
     .
     .
}

Thanks

Dan

Hi,

please don’t use Form() but TString::Format(). Does that fix it?

Cheers, Axel.

Hi Axel,

Thanks for the quick reply.

I Changed all the Form()'s to TString::Format()'s. Unfortunately, the problem persists. I believe the problem is in this chunk of code, because I have a vector of ints that is filled before it is executed, and as soon as it is executed the values for the vector go nuts. When I move where the vector is defined in the header file the problem goes away (and the code runs). However, I am afraid this solution allows other memory to be stepped on that I am not aware because it does not crash the program.

Here is an excerpt of the output. “cell” is the name of the vector (2D, 3 wide) of ints. (// denotes my comments added later)

//sampling the vector as I go to see where things go wrong
cell.size() = 144 cell_cont.size() = 144 cell_cont_np.size() = 144
cell.size() = 144 cell_cont.size() = 144 cell_cont_np.size() = 144
cell.size() = 144 cell_cont.size() = 144 cell_cont_np.size() = 144
cell.size() = 144 cell_cont.size() = 144 cell_cont_np.size() = 144
//executing the bad "chunk of code"
Warning in TFile::Append: Replacing existing TH1: covar_0_0 (Potential memory leak).
Warning in TFile::Append: Replacing existing TH1: covar_1_1 (Potential memory leak).
Warning in TFile::Append: Replacing existing TH1: covar_2_2 (Potential memory leak).
Warning in TFile::Append: Replacing existing TH1: covar_3_3 (Potential memory leak).
Warning in TFile::Append: Replacing existing TH1: covar_4_4 (Potential memory leak).
Warning in TFile::Append: Replacing existing TH1: covar_5_5 (Potential memory leak).
//now things have gone all wrong!
cell.size() = 1466 cell_cont.size() = 2518 cell_cont_np.size() = 4025306562
cell.size() = 1466 cell_cont.size() = 2518 cell_cont_np.size() = 4025306562

.
.
.

//eventually the code tries to access the cell vector, and crashes
cell.size() = 1466 cell_cont.size() = 2518 cell_cont_np.size() = 4025306562
/cluster/home/d/c/dcherd01/.lsbatch/1257869453.632822: line 8: 16822 Segmentation fault (core dumped) loon -bq NueAna/Make_nccoh_DataMC_Fits2.C("/cluster/tufts/minos/Hydra_Production/NC_Coherent_Fits/MockDataStudy_23/Fit_Set_2/Refit_File_Data_MC_sidband_histograms_100209_Mock_Data_Test_23_30000.txt",30000,“Mock_Data_Test_23”,"",2,3,4)

As I said, if I change where I define the vector in the header file by a couple of lines, this problem goes away, and everything runs, but I am afraid something else is going wrong that I can not see.

On another note what is wrong with Form, and why is Format preferable?

Thanks

Dan

Hi,

Form() is reusing a static buffer, Format() doesn’t.

Can you compile your code with ACLiC with debug info, i.e. instead of .L or .x MyCode.C so .x or “.L MyCode.C+g” (the ‘+’ is for “compile”, the ‘g’ for “with debug info”)? Does the compiler complain about anything?

If you then still get the problem, please run “valgrind root.exe” instead of “root.exe”. valgrind is an extremely useful memory checker that will tell you where the problem comes from. If you have difficulties interpreting its output then please post the log.

Cheers, Axel.

Hi,

It took a little while for the sys admin to install valgrind, but I finally have some results. The error I get prior to the crash is:

==15190== Invalid free() / delete / delete[]
==15190== at 0x43BFFDA: free (vg_replace_malloc.c:233)
==15190== by 0x7FE95D: free_mem (in /lib/libc-2.5.so)
==15190== by 0x7FE4D6: __libc_freeres (in /lib/libc-2.5.so)
==15190== by 0x43BC1E6: _vgnU_freeres (vg_preloaded.c:60)
==15190== by 0x77FBE3: _Exit (in /lib/libc-2.5.so)
==15190== by 0x704DF3: (below main) (in /lib/libc-2.5.so)
==15190== Address 0xAE58810 is not stack’d, malloc’d or (recently) free’d

This is of little help to me. Does it shed any light for you?

Thanks

Dan

Hi,

and this is the only message by valgrind?

If it is: no idea, I have never seen something like that before. It could well be a stack overwrite. To check that you can rebuild your sources (and possibly ROOT’s) with -fstack-check. For ROOT, get recent ROOT sources (e.g. v5.24/04), create the file MyConfig.mk in $ROOTSYS, put the following inside:

CXXFLAGS+=-fstack-check -g
CFLAGS+=-fstack-check -g
CINTCXXFLAGS+=-fstack-check -g
CINTCFLAGS+=-fstack-check -g
LDFLAGS+=fstack-check -g

run ./configure && make -j3. Takes about 2-10 minutes, depending on the speed of your machine.

Cheers, Axel.

Hi Axel,

No that was not the only valgrind message, but it was the only one the tripped around the piece of code in question. Since valgrind was getting tedious I went another route, and rewrote the offending code piece by piece. I started by moving all the TObject Arrays (TH2F*, TLine*, etc) to TObjArray’s. This did not fix the problem. The only thing left was the 2D array of doubles. I rewrote that function so it did not need the array, and the problem went away. I have held off posting until now to do extensive testing, and everything is working fine. Any ideas on why this would cause a problem? The only thing I can think of is that I am running on a 64 bit cluster, and compiling on a 32 bit node (as minossoft will not compile in 64 bits), and something gets lost in translation.

Thanks for all your help,

Dan

//.cxx file 
 NCCoherentFitter::NCCoherentFitter(string refit_file_name, int fit_num, int nc_coh_norm_label, int had_sys_num, string had_sys_name){ 
      . 
      . 
      . 
   const int num_of_params = 3+had_sys_num; 
   covariance = new double*[num_of_params];  <---Problematic
   one_sigma = new TLine*[num_of_params]; 
   best_fit = new TMarker**[num_of_params]; 
   covar_2D_plots_clone = new TH2F**[num_of_params]; 
   covar_1D_plots = new TH1F*[num_of_params]; 
   covar_2D_plots = new TH2F**[num_of_params]; 
   for(int k = 0; k<had_sys_num+3; k++){ 
     covariance[k] = new double[num_of_params];   <---Problematic
     best_fit[k] = new TMarker*[num_of_params]; 
     covar_2D_plots_clone[k] = new TH2F*[3+had_sys_num]; 
     covar_2D_plots[k] = new TH2F*[3+had_sys_num]; 
     for(int j = 0; j<had_sys_num+3; j++){ 
       covariance[k][j] = 0.0;                              <---Problematic
       if(j==k) covar_1D_plots[k] = new TH1F(Form("covar_%d_%d",k,k),Form("covar_%d_%d",k,k),bins[k],min[k],max[k]); 
       covar_2D_plots[k][j] = new TH2F(Form("covar_%d_%d",k,j),Form("covar_%d_%d",k,j),bins[k],min[k],max[k],bins[j],min[j],max[j]); 
     } 
   } 
      . 
      . 
      . 
 }

covariance[k] = new double[num_of_params]; <---Problematic best_fit[k] = new TMarker*[num_of_params]; covar_2D_plots_clone[k] = new TH2F*[3+had_sys_num]; covar_2D_plots[k] = new TH2F*[3+had_sys_num]; for(int j = 0; j<had_sys_num+3; j++){ covariance[k][j] = 0.0; <---Problematic I noted that you loop for 0 to had_sys_num+3 but allocated num_of_params element in covariance[k]; this could be a problem if you are not guaranteed that (had_sys_num+3) is always less than num_of_params.

Cheers,
Philippe.