@jprochaz: yes, to your point 1, I presume that is a bug (is why I first got interested in this thread). There is quite a bit of string manipulation going on in creating the list. Among others, dealing with std:: that ROOT/meta likes to remove, and which is my biggest headache at the moment. Not just here; that issue comes up again and again. In fact, you worry about ROOT classes in the global namespace, but at least ROOT classes do live in the global namespace. All of std:: does not and yet it’s shoe-horned into it.
No, the cppyy.py in ROOT is not an older version of cppyy. That is, in fact, the old PyCintex.py, which itself was based on libPyROOT. What happened was that as part of the move to ROOT6, I wanted users of PyCintex.py to move their codes to cppyy.gbl rather than ROOT.py. This for two reasons: 1) be ready for PyPy (which supports cppyy), and 2) allow ROOT to be as ROOTy as it wanted, w/o affecting Gaudi, Athena, COOL, etc. that were previous users of PyCintex. Think about e.g. the handling of graphics that ROOT.py does: building old PyCintex codes on top of ROOT.py would be too complex a tangle at run-time.
As for mixing PyROOT code and other cppyy-bound code: I don’t think that will be an issue per se. A priori, I don’t think that PyROOT should expose anything cppyy, and as you say: the rest of the world uses proper namespaces, thus allowing one bad player. What is already mixed (within HEP) has been mixed through PyROOT for years. And probably all of that will go away with ROOT7.
As for its use outside of HEP v.s. its use being ‘too “ROOT-centric”’, remember the history here. I left HEP and decided to fork PyROOT since the code was quickly acquiring lots of bit rot. I thought that, after cleanup, here was a product ready to be used, since it could already handle large code bases (ROOT, Gaudi, and lots of experiment software). Pfah! I was so wrong. I spent so far almost two years dealing with support for modern C++, portability, and plain old performance. It is that new cppyy that is being back-ported into ROOT and although all effort seems to be going towards backwards compatibility of ROOT.py, there is simply no comparison in performance and handling of modern C++ between cppyy and libPyROOT.
Then for packaging/exposure of bindings: I only recently started to look into it. First I find, again, that this ROOT stuff simply isn’t ready for large-scale deployment. I tried modules (promised since 2012), only to find that to crash and burn as well. So, there’s a lot of work still to be done.
That said, what the “outside” world wants are something like the cmake fragments that ship with cppyy: a simple way of dropping in some headers, specifying some libraries, some conventional names, and then get a Python module. There is, at that point, no direct use of cppyy.gbl. The way that that works, is by generating a Python package of the proper name and a list of C++ entities using the clang python module (rootcling being beyond fixing/saving), which are then prepped at run-time.
There are some examples linked from the main cppyy documentation (e.g. https://github.com/camillescott/cppyy-bbhash), so you can see how that works. But again, first time this is used on something large (e.g. the Point Cloud Library), Cling performance (ie. lack of) kills it, but also still more missing modern C++ features in ROOT/meta (e.g. anonymous unions). So, as said, working on that right now. After that, there may be a better option, or at least clearer path, to make things more modular even when the C++ codes themselves are not cleanly structured.