Memory leak and poor performance i PyROOT when adding arrays to vector

[LeWhoo]

@vpadulan

until recently, PyROOT was not able to add the contents of multidimensional numpy arrays to a multidimensional std vector

To what feature are you referring to here exactly? Could you provide a snippet of code to show me what you mean?

It is described in this topic:

STL vector and numpy types in PyROOT

Hello, It seems ROOT does not recognise at least some numpy types when they are being pushed to STL vector: import ROOT import numpy as np a = ROOT.vector("unsigned int")() b = np.array([1], dtype=np.uint32) a.push_back(b) The problem is, that in general python, as far as I know, there is no type for unsigned int. So if we want to be precise with types, for example, when writing to TTrees, we need to use some more specific variables, for example those of numpy type. These work fine with TT…

and there was a ticket for it. If you want I can try to find it. Basically, initialising or += of a numpy >=2D array to a vector crashed. Seems to be fixed now, at least for 2D numpy arrays. Haven’t tried for 3D yet.

@vpadulan

It seems that something is not dereferenced after adding a list of array.array to a vector

I do not understand this statement, can you elaborate? What should be “dereferenced” in your opinion?

I understand, that the garbage collector in Python cleans up object to which there are (nearly) no references. Here, clearly, there is a memory leak, so something is not cleaned by garbage collector, and I have a feeling that the contents of the array.array are copied inside the vector, but somehow the array.array object is being held after this copying, not derefrenced.

@vpadulan

What is the use case you are trying to solve here? Why do you need to append a type array.array into a type ROOT.std.vector[ROOT.std.vector["double"]]? What would happen in your application after this step?

This is also described in the post I quote at the beginning. Basically, std vector was not accepting numpy arrays with types such as uint32. It was crashing. It was, however, accepting array.array types. Thus in this line I use array.array to convert numpy array to something acceptable by ROOT std vector, as a workaround. In this example I’ve given a double which I think worked OK, but in my code I deal with unsigned ints or unsigned shorts, which were crashing. Perhaps this conversion is working well now (I need to check), but it was not at the moment of writing the post that I refer to. Anyway, thanks to this workaround, I discovered the reported memory leak and slowdown.