Boost library and root6 - error in begin(array)

ROOT
1

Dear co-rooters,

I am trying to playing with matrices in root.
Since I experienced some issues using TMatrix and SMatrix (Resizing and transposing a TMatrixFSparse - row/col indices are not set, Cannot transpose a TMatrixFSparse matrix - Error in : matrix has wrong shape, TMatrix for large matrices, Using SMatrix for large matrices) so I used the boost library after the suggestion of @Wile_E_Coyote

The first try to build a symmetric matrix using an array can be seen below

//#if !defined(__CINT__)

#include <iostream>
#include <fstream>
#include <vector>
#include <algorithm>// std::max()
#include <iterator>

#include </usr/include/boost/numeric/ublas/matrix.hpp>
#include </usr/include/boost/numeric/ublas/matrix_sparse.hpp>
#include </usr/include/boost/numeric/ublas/symmetric.hpp>
#include </usr/include/boost/numeric/ublas/io.hpp>
//#endif /* !defined(__CINT__) */

#include "covariance_lib.cc"//The function fill_symmetric has an issue - begin(array) throws an error

using namespace std;

int test_boost () {
    using namespace boost::numeric::ublas;
    symmetric_matrix<double, upper> m_sym1 (3, 3);

    float filler[6] = {0, 1, 2, 3, 4, 5};
    float const* in1 = std::begin(filler); 
    
    for (size_t i = 0; i < m_sym1.size1(); ++ i)
    	for (size_t j = 0; j <= i && in1 != std::end(filler); ++ j)
    	   	m_sym1 (i, j) = *in1++;

    std::cout << m_sym1 << std::endl;
    
    symmetric_matrix<float, upper> m_sym (3, 3);
    m_sym1 = fill_symmetric( m_sym1, filler);
    
    std::cout << m_sym << std::endl;
    
    return 0;
}

where covariance_lib.cc can be seen below

// Inputs :  	-name   = name of the ascii file to be read - two column file with x and y values, rescpectively assumed
//		-X_USER = array that contains the user defined energy binning
//		-output = if output = TRUE  exports the interpolation array
//			  if output = FALSE exports the index of the input values used in the interpolation
void linear_interpolation(int size_USER, int size_EVAL, float* X_USER, float* Y_USER, float* X_EVAL, float* Y_EVAL, int* index){

	float x1, x2,y1, y2, a, b;
	bool  boo;
	x1 = 0.; x2 = 0.; y1 = 0; y2 = 0.; a  = 0.; b  = 0.;
    	for(int i=0; i<=size_USER; ++i){
		boo = true;
		for (int j=0; j<=size_EVAL; j++){
			if (boo==true){
				if(X_USER[i] == X_EVAL[j]){
					Y_USER[i] = Y_EVAL[j];
					index[i]  = j;
					boo = false;
				}
				else if(X_USER[i]>X_EVAL[j]){
					x1 = X_EVAL[j];
					y1 = Y_EVAL[j];
					index[i] = j;
				}
				else if( X_USER[i]<X_EVAL[j] && boo == true ){
					x2 = X_EVAL[j];
					y2 = Y_EVAL[j];
					a = (y2-y1)/(x2-x1);
					b = y2-a*x2;
					Y_USER[i] = a*X_USER[i]+b;
					//index[i]  = j;
					boo = false;
				}
			}
		}//end of loop over EVAL
	}//end of loop over USER	
}//____linear_interpolation()


void input_ascii(TString filename, float* X_EVAL, float* Y_EVAL){

	std::ifstream myfile;
	myfile.open(filename);
	float x = 0, y = 0;
	int   i = 0;
  	while(1){
		myfile >> x >> y;
		if (!myfile.good()) break;
		X_EVAL[i]    = x;
		Y_EVAL[i]    = y;
		i++;
	}
    	myfile.close();
}//___input_ascii()


int count_lines(TString filename){

	float	x, y;
	int nlines;	
	// (1) Open the file to count its lines
	std::ifstream myfile;
	myfile.open(filename);
  	if(!myfile){
		cout << "File not found" << endl;
  	}
  	nlines = 0;
  	while(1){
  		myfile >> x >> y;
  		if (!myfile.good()) break;
		nlines++;
	}
	myfile.close();
	return nlines;

}//___count_lines()


std::vector<float> isolethargic_binning(int min, int max, int bpd){

	int 	ndec  = max - min;
	int 	nbins = (int) ndec*bpd;
	float	step = (float) ndec / nbins;
	std::vector<float> xbins(nbins+1);

	for(int i=0; i <= nbins; i++) {
		xbins[i] = (float) pow(10., step * (float) i + min);
	}
	
	return xbins;

}//___isolethargic_binning()

boost::numeric::ublas::symmetric_matrix<float> fill_symmetric (boost::numeric::ublas::symmetric_matrix<float> m_sym, float* filler){

	//assert(("ERROR! Filler's dimension have to be n*(n+1)/2!", sizeof(filler)/sizeof(filler[0]) == m_sym.size2() ));
	float const* in = std::begin(filler); 
    
	for (size_t i = 0; i < m_sym.size1(); ++ i)
    		for (size_t j = 0; j <= i && in != std::end(filler); ++ j)
        		m_sym (i, j) = *in++;
	
	return m_sym;

}//___fill_symmetric

The problem is when compiling the code (ACLiC using ROOT6 on lxplus) I get the following output

root [1] .L boost.C++
Info in <TUnixSystem::ACLiC>: creating shared library /eos/user/a/astamato/covariance/code/./boost_C.so
Warning in cling::IncrementalParser::CheckABICompatibility():
  C++ ABI mismatch, compiled with __GLIBCXX__ v20130531 running with v20141219
In file included from input_line_12:9:
In file included from ././boost.C:27:
./covariance_lib.cc:96:20: error: no matching function for call to 'begin'
        float const* in = std::begin(filler); 
                          ^~~~~~~~~~
/afs/cern.ch/sw/lcg/contrib/gcc/4.8.4/x86_64-slc6/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.8.4/../../../../include/c++/4.8.4/bits/range_access.h:48:5: note: candidate template ignored: substitution failure
      [with _Container = float *]: member reference base type 'float *' is not a structure or union
    begin(_Container& __cont) -> decltype(__cont.begin())
    ^                                           ~
/afs/cern.ch/sw/lcg/contrib/gcc/4.8.4/x86_64-slc6/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.8.4/../../../../include/c++/4.8.4/bits/range_access.h:58:5: note: candidate template ignored: substitution failure
      [with _Container = float *]: member reference base type 'float *const' is not a structure or union
    begin(const _Container& __cont) -> decltype(__cont.begin())
    ^                                                 ~
/afs/cern.ch/sw/lcg/contrib/gcc/4.8.4/x86_64-slc6/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.8.4/../../../../include/c++/4.8.4/bits/range_access.h:87:5: note: candidate template ignored: could not match
      '_Tp [_Nm]' against 'float *'
    begin(_Tp (&__arr)[_Nm])
    ^
/afs/cern.ch/sw/lcg/contrib/gcc/4.8.4/x86_64-slc6/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.8.4/../../../../include/c++/4.8.4/initializer_list:89:5: note: candidate template ignored: could not match
      'initializer_list<type-parameter-0-0>' against 'float *'
    begin(initializer_list<_Tp> __ils) noexcept
    ^
In file included from input_line_12:9:
In file included from ././boost.C:27:
./covariance_lib.cc:99:42: error: no matching function for call to 'end'
                for (size_t j = 0; j <= i && in != std::end(filler); ++ j)
                                                   ^~~~~~~~
/afs/cern.ch/sw/lcg/contrib/gcc/4.8.4/x86_64-slc6/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.8.4/../../../../include/c++/4.8.4/bits/range_access.h:68:5: note: candidate template ignored: substitution failure
      [with _Container = float *]: member reference base type 'float *' is not a structure or union
    end(_Container& __cont) -> decltype(__cont.end())
    ^                                         ~
/afs/cern.ch/sw/lcg/contrib/gcc/4.8.4/x86_64-slc6/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.8.4/../../../../include/c++/4.8.4/bits/range_access.h:78:5: note: candidate template ignored: substitution failure
      [with _Container = float *]: member reference base type 'float *const' is not a structure or union
    end(const _Container& __cont) -> decltype(__cont.end())
    ^                                               ~
/afs/cern.ch/sw/lcg/contrib/gcc/4.8.4/x86_64-slc6/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.8.4/../../../../include/c++/4.8.4/bits/range_access.h:97:5: note: candidate template ignored: could not match
      '_Tp [_Nm]' against 'float *'
    end(_Tp (&__arr)[_Nm])
    ^
/afs/cern.ch/sw/lcg/contrib/gcc/4.8.4/x86_64-slc6/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.8.4/../../../../include/c++/4.8.4/initializer_list:99:5: note: candidate template ignored: could not match
      'initializer_list<type-parameter-0-0>' against 'float *'
    end(initializer_list<_Tp> __ils) noexcept
    ^
Error in <ACLiC>: Dictionary generation failed!

It seems that the error is in the function fill_symmetric() and I don’t get why begin(array) gives an error!

The even strangest thing is that if I execute the filling in the main code, it works fine!

Any idea why this is happening and how to fix it?

Thanks in advance!

2

This gets down to calling an std function for a pointer, so it makes no sense… So now I am able to solve the compilation errors, but still I have an issue…

When filling the matrix, the last element doesn’t get filled in correctly. So although I expect to get

[3,3]((0,1,3),(1,2,4),(3,4,5))

the output of the code is

[3,3]((0,1,3),(1,2,4),(3,4,2.6681e-315))

I am compiling this code in ROOT6. I don’t think it’s related to that, I am just mentioning it for completion.

A small sample of the code follows

#include <iterator>
#include <iostream>
#include <fstream>

#include </usr/include/boost/numeric/ublas/matrix.hpp>
#include </usr/include/boost/numeric/ublas/matrix_sparse.hpp>
#include </usr/include/boost/numeric/ublas/symmetric.hpp>
#include </usr/include/boost/numeric/ublas/io.hpp>

using namespace std;

int test_boost () {

    using namespace boost::numeric::ublas;
    symmetric_matrix<double, upper> m_sym1 (3, 3);

    float* filler = new float[6];
    for (int i = 0; i<6; ++i) filler[i] = i;
    float const* in1 = filler;

    for (size_t i = 0; i < m_sym1.size1(); ++ i)
        for (size_t j = 0; j <= i && in1 != &filler[5]; ++ j)
            m_sym1 (i, j) = *in1++;

    std::cout << m_sym1 << std::endl;
    return 0;
}

Any idea on how to solve that?

3

Look at the end condition more closely. End is &filler[6], not [5].

Use a std::vector for filler, fill it with std::iota (well, I guess the real data is different anyway…) and use std::begin+std::end.

4

As for your original question: that can also be solved by replacing the filler with a std::vector. The problem is: When using a float* as function argument, you cannot use std::begin on that variable any more. The array from the caller decays to a pointer (yes, there IS a difference between an array and a pointer!). Instead, either pass a const vector<float> &filler or do it STL-like and pass begin+end iterators to the filler.

Just for completeness (DON’T do it this way): you could also pass the array as array. Then you need to write: fill_symmetric(..., float (&filler)[6])

5

Thanks a lot for your help!

The problem is that this will be used to handle large arrays and matrices (20000 x 20000) so I am forced to store then on the heap.

So far I only know

float* array = new float[size].

Is there another way to store them on the heap when size is known in pre-compilation time or is calculated within the code?

6

use a std::vector!

(that’s all I wanted to write, but that seems to short to submit, so: vectors store the data on the heap, sizeof(vector<any type>) = 3 * sizeof(pointer), i.e. 24. So a vector stores 24 bytes metadata (size, capacity, pointer to data [1]) on the stack, but data itself is on the heap)

By the way: 20000 x 20000 x 4 Bytes = 1.6 GB RAM, do you really need this vector if you want to copy it to your matrix anyway?

[1] actually, vectors can also consist of 3 pointers: a pointer to data, a pointer to the end of capacity, a pointer to the end of the data. But that’s equivalent and an implementation detail. libstdc++ (last time I had a look) was using 3 pointers instead of pointer + 2x size.

7

Thank you very much for the info!!!

Unfortunately I do need such big arrays.
Off course later on I will delete them

delete[] array

That is why I wanted to use new'. So, as far as I understood, if I define anstd:vector` with a predefined size, it is stored on the heap. Can I delete it afterwards when it has served its purpose?

8

You are working with Boost so why not use its built-in matrices (and simply pass their “references” between your routines)?

9

I don’t get what you mean…

My problem is that the constructor of a symmetric matrix in boost, doesn’t support to fill it with an array.
So I will have to find a way to fill the matrix. Since my only experience is with TMatrixFSym, which uses an array to fill the matrix, I am trying to use the same “technique”

10

Question: What is your favorite command in C++?
Answer: }

The closing curly closes a block and calls all destructors of local objects. Therefore the memory reserved by the vector is freed once the vector goes out of scope. Usually that is all you need to know. Sometimes you might want to “free” your vector before the closing }. In that case you can swap your vector with a temporary vector: vector<float>{}.swap(your_vector);

I think what @Wile_E_Coyote wants to say: when using ublas, you could as well start using ublas::matrix / vector types from the beginning (I also didn’t unserstand what the posted links have to do with it) or somehow directly pass a row or col to the ublas matrix without copying. As I have no experience with ublas, I’m afraid I cannot help you any futher.

11

Wow! Thank you very much for the vector<float>{}.swap(your_vector); command!!! It seems to be very useful!

The thing eventually is that at some point I will have to make TGraph's or TH1/2's, that’s why I think I can’t move to a <vector>.

12

a) vector swap: actually, in C++11 you can also use clear() followed by shrink_to_fit(), though the C++ standard does not guarantee it will really shrink & free memory.

b) Don’t understand why you can’t use a vector. What does it have to do with TH1, TH2, TGraph? (are you looking for vector::data?)

13

a) So the preferable method is still vector<float>{}.swap(your_vector);?

b) I am not an expert, as you can definitely tell, so when I see the constructor for a TGraph ( TGraph (Int_t n, const Float_t *x, const Float_t *y) ), the first thing that comes to my mind is to declare the input as an array using float* array = new array[size]. So later I can pass that to a function that requests a float *x as an argument.

14

a) the preferrable method is to not use this at all and let the vector go out of scope, the swap idiom is just a trick to free the memory before reusing the vector. Do you really need to keep that vector? If not, prefer to find a } not too far away…
b) you can use vector::data for these arguments

15

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.