ROOT feature overload versus R package management

[cstrato]

Dear Rooters

Since ROOT is currently undergoing an exponential growth phase with the continuous
addition of new packages, i.e. libraries, please allow me to make some suggestions
regarding a proposed re-organization of ROOT.

Since many years I am using R ( cran.r-project.org/ ) at work and ROOT at home,
thus I believe that I am able to compare both software systems, both having advantages
and disadvantages (see the attached file “ROOTvsR.jpg”).

Today I would like to concentrate on the package management system of R and suggest a
similar system for ROOT.

In principle the R system consists of a collection of packages which can be divided into:
1, Core packages: e.g. base, graphics, stats
2, Recommended packages: e.g. MASS, boot
3, Contributed packages: e.g. gam, tree ( cran.r-project.org/src/contrib/PACKAGES.html )
4, Special software systems: e.g. for bioinformatics ( www.bioconductor.org )

When you download and install R you will find that only the core and recommended
packages are installed which together constitute the “packages in the standard library”.
Additional packages can be installed on demand from within R with a simple command
“install.packages()”.

Looking e.g. at package tree: cran.r-project.org/src/contrib/D … /tree.html
you can see the following advantages of the package system:

• the basic R system has a reasonable size and can be maintained separately
• additional packages can be installed when necessary
• contributed packages each:
• • have their own author(s) and maintainer(s)
• • can be updated independent of the main R package
• • can have their own license, e.g. GPL, LGPL, free for academic use only, etc.
• • can have their own dedicated website with documentation etc.

Analogously to the R package system I would like to suggest that the ROOT system
could be devided into the following packages, i.e. libraries:
1, Core libraries
2, Recommended libraries
3, ROOT compliant libraries:
These are libraries especially created for ROOT with classes derived from TObject, and
adhering to the ROOT coding conventions
4, External libraries:
These are e.g. wrappers to external libraries for ROOT, or libraries with classes not
derived from TObject or not adhering to the ROOT coding conventions

Here is my suggestion for the different libraries:
1, Core libraries:
These include the ROOT core classes as shown in Figure 1-2 of the ROOT UserGuide,
and e.g. libGraph, libHist, libMinuit, libTree, etc

2, Recommended libraries:
These include e.g. libGui, libGX11, libProof, libTreeViewer, etc

3, ROOT compliant libraries:
These include libraries recently added to ROOT such as e.g.:

• the Geometry package libGeom, libGeomPainter
• the FOAM package libFoam
• the MultiLayerPerceptron package libMLP
• the Fumili package libFumily
• Toolkit for Multivariate Analysis (TMVA)
• libMinuit2, libSPlot, libMonaLisa etc

4, External libraries:
These include wrappers to external libraries such as

• TFFT, wrapper to FFTW3
• maybe RooCARDS, a wrapper to the Stuttgart Neural Network Simulator
• maybe a wrapper to the GNU Scientific Library
• maybe a wrapper to bonsai.ims.u-tokyo.ac.jp/~mdehoo … e/cluster/
  and libraries with classes not derived from TObject or not adhering to the ROOT
  coding conventions such as:
• libMathCore and libMathMore
• libRooFit

The core and recommended libraries constitute the basic ROOT system which can be
downloaded from the ROOT website and is available from cvs. It is updated on a
regular basis.
Both ROOT compliant libraries and external libraries can be downloaded and added
on demand, having the following advantages:

• the basic ROOT system has a reasonable size and can be maintained separately
• additional libraries can be installed when necessary
• contributed and external libraries each:
• • can have their own author(s) and maintainer(s)
• • can be updated independent of the main ROOT system
• • can have their own license, e.g. GPL, LGPL, free for academic use only, etc.
• • can have their own dedicated website with documentation etc.

As I said this is a suggestion only based on my experiences with both R and ROOT.
Maybe others might be willing to comment.

Best regards
Christian

[Axel]

Hi Christian,

though this answer is all but authoritative, I still cannot resist and have to give some comments.

The only argument in your posting I could find for splitting ROOT into these layers are a) flexibility for licenses and b) smaller core distribution size.

The license is a matter of taste - I personally appreciate the fact that ROOT is (at least LGPLed and forces everything under its hood to be (at least) LGPLed, too.

I don’t see how the other arguments (rephrasing here: doc spread over multiple sites, no coding convention, no central repository, no central place to bug people, no consistency in versioning and thus in compatibility) support your suggestion. Btw, already now we have multiple (as in non-CERN) authors. And even if a potential author feels ignored, I believe there is a non-CERN repository (which I cannot find right now - was it at imperial college? Manchester? Can’t remember.) for user contributed pieces of ROOT-related code.

That leaves us with the size of the distribution. For that, Rene has his own ideas…

Cheers, Axel.

[brun]

Hi Christian,

Thanks for sending this feedback.

ROOT is being developed with High Energy Physics requirements in mind first.
It could be that some of the packages are of little interest
for the outside community, and even we are not always sure
for our community if some packages should be declared optional or not.
Our experience and feedback shows that most users prefer to see
one system that looks the same wherever they work.
To install it, they would like to just do
./configure
make
make install
Many complain that our option list is becoming too long.
Our main distribution includes most options. See:
root.cern.ch/root/myconfig/Versi … onfig.html
As soon as a package is declared optional, its weight diminishes considerably.
People will have tendancy to not use it and come with their own solutions.

Deciding if a package is part of the default installation, optional,
to be removed, or added is a tricky thing. Sometime we are under pressure
from some groups to include a new system, even if the expected use is limited.
Sometime we include a new package (eg tmva) because we believe that it will
become strategic in the near future.
I do not think that ROOT is going to grow for ever. We are now in an intense
development phase in view of the LHC. Personally, I would like to keep
the distribution file as small as possible. We also make some effort
to improve the installation time and procedure.

We made several attempts to collect external packages or references
to external packages. I must admit that this has not been a big success.
We have a web page (root.cern.ch/root/ExApplications.html).
Several packages in this list are obsolete (or often with a broken link).
On the other hand, there are hundreds of applications that are not shown
in this list.

Your comparison between R and ROOT is strongly biaised by your personal
profile. For example what about
-general solution for persistency (not just array oriented)
-2-d graphics
-3-d graphics and GL viewers
-image processing
-geometry description and navigation
-histograming
-Minimisation
-linear algebra
-networking, client/server

By the way, we are not developing ROOT as an alternative to R. We have
been discussing many times interfaces between the two systems. Not much
has been happening so far, simply because users do not request it
with high priority. I would be delighted to see a strong contribution
in this direction.

Rene

[Valeriy_Onuchin]

Hi,
the main question to R developers -
"why to invent just another language? ".

V.O.

[Valeriy_Onuchin]

concerning ROOT and external libraries …

I think (and I said it many times) - it’s possible to create
"ROOT plugin" which will allow to load external libs into
intepreter without “wrapping”.
Such functionality is already presented in CINT (in very limitted way).

Implementation of this mechanism would be the great breakthrough
in the ROOT development.

[cstrato]

Dear All

Thank you for your extensive reply. As I said these are my personal thoughts only.
The recent additions such as TFFT and especially TMVA (which I consider to be of high
interest) reminded me of the package system of R, so I thought it might be an
alternative to the current way of simply adding more classes to ROOT.

Furthermore, at the different workshops (ROOT, CHEP, PHYStat) there were requests
to add e.g. the GNU Scientific Library (which is GPL) and a package for hierarchical
cluster analysis (which I would appreciate). (BTW, FFTW is also GPL and not LGPL.)

I have just checked some of these workshop presentations:

• Rene’s BOOT system could be a step in this direction. Maybe, it will even be more
  advanced than the R package system
• At PHYstat05 there was a talk about R and ROOT ( www.physics.ox.ac.uk/phystat05/Talks/LinnemanOxford.pdf )
  where it was mentioned that “Adam Lyon wrote a Root Tree reader for R”. I do not know
  what the current status of this package is.

The “Example Applications” on the ROOT web site were simply considered to be examples,
and I have never considered these examples to be real add-ons. If there would be an
official package system, users would surely use this system. If the ROOT users feel
that the current system is fine, this is fine with me, too. I am glad to have the
options to disable/enable different packages in the configure file.
BTW, looking at the mentioned config site, does this mean that e.g. mathcore, mathmore
and reflex do not work on MacOS X PPC, HP-UX and IRIX?

Best regards
Christian

P.S.: Dear Rene, my comparison between R and ROOT is shown in “ROOTvsR.jpg”, it lists
only the main features and I hope that it is not biased. Furthermore, it is three years
old. I would appreciate if you suggest features which I should update/add without
the table getting too large. ( BTW, R has e.g. linear algebra, histograms, minimisation
and extensive 2-d graphics, too.)

P.P.S.: Dear Valeriy, please ask the Bell Labs:
Why did Kernighan & Richie from Bell Labs invent “C”?
Why did John Chambers from Bell Labs invent “S”?
(“R” is simply the public domain version of “S”)
(S-Plus, SAS and SPSS are the three most widely used statistical software systems.)