Boost-1.70 released, new library boost::histogram

Evolution Proposals, Apps & Ideas
1

It has been a while since I announced here that the boost::histogram library is nearing completion. And now it happened, Boost-1.70 was released on April 12th, 2019, with boost::histogram.

github https://github.com/boostorg/histogram
documentation https://www.boost.org/doc/libs/1_70_0/libs/histogram/doc/html/index.html
chat with me https://gitter.im/Scikit-HEP/community or https://cpplang.slack.com

boost::histogram is a powerful library to make histograms, profiles, and more. boost::histogram was designed to be useful for the beginner and expert alike. It is very customizable, but the defaults are set so that it is safe, fast, memory-efficient for people who just want to get work done. The design is similar to RHist, but offers even more flexibility. It can replace TH1, TH2, TH3, THn, THnSparse, TProfile, TProfile2D, TProfile3D. In the future, it could also replace TProfile2Poly.

A Python frontend is currently developed by Henry Schreiner and myself, an alpha version can be accessed here.

2 Likes
2

Thanks, Hans!

I am aware of a couple of other (also C++) histogram libraries, too. Yours is nice, and we want RHist to be at least as nice :wink: I’m curious to hear how we can improve ROOT’s histograms based on what you have learned!

Cheers, Axel.

3

Why not reuse the existing code?
Why create yet another library which will not be compatible with anything else?

Last but not least, can ROOT be convinced to deal with / import “boost::histogram”?

4

Dear Axel,

the point of making boost::histogram was to offer a standard solution so that people can stop reinventing histograms. All code has bugs, but if many people use the same code, it is going to have fewer bugs. That’s why boost::histogram was designed to be super flexible, so that if you don’t like a given part, you can switch that out and still use the rest.

Conceptually, histograms and profiles can be split into three orthogonal components (as explained here)

  • Axes: converts values to bin indices
  • Storage: manages memory for the histogram cells
  • Accumulators: the object that sits in a cell and reacts to data that is sorted into the cell

The components are orthogonal, because they can be combined arbitrarily, they only talk through their respective interfaces to each other. To get a sparse histogram, you replace a array-based storage with a map-based storage. To go from histogram to profile, you replace an integer or floating point counter per cell with an object that keeps track of the sum of values and the number of times a value was entered in each cell. This design allows maximum code re-use.

You could replicate these ideas in ROOT, but that would go against the idea of sharing as much code as possible to have fewer bugs and ease maintenance.

It is said here that ROOT 7 will use “standard C++ types”, so why not use boost::histogram as the foundation for a custom histogram class. ROOT 7 could inherit from boost::histogram and implement a ROOT-like interface and additional capabilities on top of that foundation. boost is the best standard in the C++ world next to having something in the official C++ standard library.

I also want to be be clear that boost::histogram only addresses the focused problem of histogramming and profiling data, it does not provide functionality to fit data or to plot data. These features should come from other libraries such as ROOT.

5

Thank you for this statement, I also think that it best if we share as much code as possible. boost::histogram is an honest attempt to establish an open standard for histograms in the C++ world, which should benefit everyone. As explained in my answer to Axel, I think it would be great if libraries such as ROOT would build their tools on top of standard solutions. C++ offers interfaces and encapsulation, so nobody has to see boost::histogram on the surface. Still, by using it internally, one gains from external improvements and maintenance of the implementation.

6

Note that you can depend only on header only parts of Boost in order to use Boost::Histogram, no built components are required. Might make building RHist on top of Boost::Histogram an interesting possibility…

7

Libraries like boost::histogram are usually much more accessible than any ROOT solution since the overhead of getting it is much lower.

It would be nice to move as many general features out of ROOT to make them available to the general public :).

8

Hello,

I am currently working with boost::histograms and they are really useful and a powerful tool.
In the context of my work, I would need to write them into a ROOT Tree.

Does anyone has already done it before ?

I already thank you for your help,

Eliott Ramoisiaux

9

Dear Eliott,

great to hear that you like boost::histogram, we tried very hard to make them powerful and easy to use at the same time, and continue to add more features.

Unfortunately, boost::histogram is currently not directly supported by ROOT, but if you are working in Python, there are some options.

If you just need to save the histogram to the disk, you can use the Python pickle module. The output is going to be large, but you can compress it easily with the gzip module.

import gzip
import pickle

# make and fill some boost histogram
hist = ...

# write histogram to disk
with gzip.open("myfile.pkl.gz", "wb") as f:
    pickle.dump(hist, f)

# load histogram from disk
with gzip.open("myfile.pkl.gz", "rb") as f:
    hist = pickle.load(f)

# continue working with hist

If you must safe it in a ROOT tree, e.g. to give it to a colleage, you can try this tutorial.

Best regards,
Hans

PS: This Gitter chat is also a good place to ask questions about boost::histogram. https://gitter.im/HSF/PyHEP-histogramming

10

@pcanal In general, nowadays, ROOT 6 should be able to save / retrieve objects of any class into / from a ROOT file.

11

Hello everyone,

Thank you both for your answers.

I’ll keep in mind the possibility to use Python if I do not succeed to do it as @Wile_E_Coyote proposed.

In order to save boost::histogram into a ROOT file, I created a simple example whose goal was to write a Boost and a ROOT histograms into a ROOT file.

The main file is

#include <iostream>
#include <fstream>
#include <sstream>
#include "TH1F.h"
#include "TFile.h"
#include "TTree.h"
#include "TRandom.h"

#include <boost/histogram.hpp>
#include <boost/format.hpp>
#include "BDSBH4D.h"

int main() {

    BDSBH4D* h1 =  new BDSBH4D();

    h1->h.operator()(0.7,0.7,0.7,120);

    // Print of the 4D - histogram.
    std::ostringstream os4;
    for (auto&& x : indexed(h1->h)) {
        os4 << boost::format("(%i, %i, %i, %i) [%.3f, %.3f) [%.3f, %.3f) [%.3f, %.3f) [%.3f, %.3f) %i\n")
               % x.index(0) % x.index(1) % x.index(2) % x.index(3)
               % x.bin(0).lower() % x.bin(0).upper()
               % x.bin(1).lower() % x.bin(1).upper()
               % x.bin(2).lower() % x.bin(2).upper()
               % x.bin(3).lower() % x.bin(3).upper()
               % *x;
    }

    std::cout << os4.str() << std::flush;

    TH1F* histogram = new TH1F("stored_histogram","histogram;X;# of entries",100,-5,5);
    histogram->FillRandom("gaus");

    TFile *hfile = new TFile("my_rootfile.root","RECREATE","Example");
    TTree *tree = new TTree("Event","A Root Tree");

    tree->Branch("Histos",&histogram,32000,0);
    tree->Branch("BOOST_histogram","BDSBH4D",&h1,32000,0);
    tree->Fill();

    hfile->Write();
    hfile->ls();
    hfile->Close();


    return 0;
}

with BDSBH4D.cpp

#include "BDSBH4D.h"

ClassImp(BDSBH4D)

BDSBH4D::BDSBH4D()
{
h = boost::histogram::make_histogram(boost::histogram::axis::regular<double> {3, 0.0, 1.0, "x"},
                                     boost::histogram::axis::regular<double> {3, 0.0, 1.0, "y"},
                                     boost::histogram::axis::regular<double> {3, 0.0, 1.0, "z"},
                                     boost::histogram::axis::regular<double, boost::histogram::axis::transform::log> {3, 1.0, 230.0, "Energy_log"});
}

BDSBH4D.h

#ifndef ROOT_TEST_BDSBH4D_HH
#define ROOT_TEST_BDSBH4D_HH

#include <boost/histogram.hpp>

#include "Rtypes.h"
#include "TObject.h"

typedef boost::histogram::histogram<std::__1::tuple<boost::histogram::axis::regular<double, boost::use_default, boost::use_default, boost::use_default>, boost::histogram::axis::regular<double, boost::use_default, boost::use_default, boost::use_default>, boost::histogram::axis::regular<double, boost::use_default, boost::use_default, boost::use_default>, boost::histogram::axis::regular<double, boost::histogram::axis::transform::log, boost::use_default, boost::use_default> >, boost::histogram::unlimited_storage<std::__1::allocator<char> > > boost_histogram;

class BDSBH4D : public TObject {

public:
    BDSBH4D();
    virtual ~BDSBH4D() {;}

    boost_histogram h;

ClassDef(BDSBH4D,1);

};


#endif //ROOT_TEST_BDSBH4D_HH

and BDSBH4DLinkDef.h

#ifndef ROOT_TEST_BDSBH4DLINKDEF_H
#define ROOT_TEST_BDSBH4DLINKDEF_H

#pragma link C++ class boost::histogram::histogram<std::__1::tuple<boost::histogram::axis::regular<double, boost::use_default, boost::use_default, boost::use_default>, boost::histogram::axis::regular<double, boost::use_default, boost::use_default, boost::use_default>, boost::histogram::axis::regular<double, boost::use_default, boost::use_default, boost::use_default>, boost::histogram::axis::regular<double, boost::histogram::axis::transform::log, boost::use_default, boost::use_default> >, boost::histogram::unlimited_storage<std::__1::allocator<char> > >+;
#pragma link C++ class BDSBH4D+;
#pragma link C++ class boost::histogram::detail::mutex_base<tuple<boost::histogram::axis::regular<double,boost::use_default,boost::use_default,boost::use_default>,boost::histogram::axis::regular<double,boost::use_default,boost::use_default,boost::use_default>,boost::histogram::axis::regular<double,boost::use_default,boost::use_default,boost::use_default>,boost::histogram::axis::regular<double,boost::histogram::axis::transform::log,boost::use_default,boost::use_default> >,boost::histogram::unlimited_storage<allocator<char> >,boost::histogram::detail::null_mutex>+;
#pragma link C++ class boost::empty_::empty_value<boost::histogram::detail::null_mutex,0,true>+;
#pragma link C++ class boost::histogram::detail::null_mutex+;
#pragma link C++ class tuple<boost::histogram::axis::regular<double,boost::use_default,boost::use_default,boost::use_default>,boost::histogram::axis::regular<double,boost::use_default,boost::use_default,boost::use_default>,boost::histogram::axis::regular<double,boost::use_default,boost::use_default,boost::use_default>,boost::histogram::axis::regular<double,boost::histogram::axis::transform::log,boost::use_default,boost::use_default> >+;
#pragma link C++ class boost::histogram::unlimited_storage<allocator<char> >+;
#pragma link C++ class boost::histogram::axis::regular<double,boost::use_default,boost::use_default,boost::use_default>+;
#pragma link C++ class boost::histogram::unlimited_storage<allocator<char> >::buffer_type+;
#pragma link C++ class boost::histogram::axis::regular<double,boost::histogram::axis::transform::log,boost::use_default,boost::use_default>+;
#pragma link C++ class boost::histogram::axis::iterator_mixin<boost::histogram::axis::regular<double,boost::use_default,boost::use_default,boost::use_default> >+;
#pragma link C++ class boost::histogram::axis::transform::id+;
#pragma link C++ class boost::histogram::axis::metadata_base<boost::use_default,string>+;
#pragma link C++ class allocator<char>+;
#pragma link C++ class boost::histogram::axis::iterator_mixin<boost::histogram::axis::regular<double,boost::histogram::axis::transform::log,boost::use_default,boost::use_default> >+;
#pragma link C++ class boost::histogram::axis::transform::log+;
#pragma link C++ class boost::empty_::empty_value<string,0,false>+;

#endif //ROOT_TEST_BDSBH4DLINKDEF_H

Unfortunately, when I run this simple example, I get this error:

Error in <TStreamerInfo::Build>: boost::histogram::unlimited_storage<allocator<char> >::buffer_type, unknown type: void* ptr

coming from the boost::histogram::unlimited_storage.hpp file at the line 256

mutable void* ptr = nullptr;

I have quite some difficulty to correct this error as I do not understand why ROOT detect void* ptr as a type.

Have you already encountered such a situation? I am quite new with c++ and dictionaries so maybe I forgot a step.

I already thank you for your help,

Here is a dropbox link to the small example if it can help: https://www.dropbox.com/sh/c9jjnuonkinumy5/AACs9uz4-s9Cg8l5bfjnjPdHa?dl=0

Have a nice day,

Eliott Ramoisiaux

12

Hi Eliott,

the default storage of a boost::histogram in C++ is the unlimited_storage, which is a complex piece of technology and cannot be easily serialized automatically. boost::histogram fully supports serialization with boost::serialization and possibly other other compatible serialization libraries, like cereal, but that is not tested.

As a workaround, you could try to change the storage of the histogram from boost::histogram::unlimited_storage<std::__1::allocator<char> > to std::vector<double>. Perhaps it works then.

If using the unlimited storage is important to you, I don’t know an easy way forward. You can look into the serialization code of that class for inspiration on how to do this with ROOT streamers (I am not an expert on ROOT streamers, only on boost::serialization).

  template <class Archive>
  void serialize(Archive& ar, unsigned /* version */) {
    if (Archive::is_loading::value) {
      buffer_type tmp(buffer_.alloc);
      std::size_t size;
      ar& make_nvp("type", tmp.type);
      ar& make_nvp("size", size);
      tmp.visit([this, size](auto* tp) {
        assert(tp == nullptr);
        using T = std::decay_t<decltype(*tp)>;
        buffer_.template make<T>(size);
      });
    } else {
      ar& make_nvp("type", buffer_.type);
      ar& make_nvp("size", buffer_.size);
    }
    buffer_.visit([this, &ar](auto* tp) {
      auto w = detail::make_array_wrapper(tp, this->buffer_.size);
      ar& make_nvp("buffer", w);
    });
  }