Go-HEP - v0.4: reading ROOT files from Go (and more)

Evolution Proposals, Apps & Ideas
1

hi there,

just to let you know I have just released Go-HEP v0.4:

the instructions to install Go and Go-HEP are here:

Version v0.4 sports:

  • reading/decoding of ROOT files
  • extracting TH1x and TH2x from ROOT files
  • reading “simple” TTrees (flat trees containing builtins and arrays/slices of builtins, event-trees of builtins, arrays/slices/std::vectors of builtins/simple-structs, std::strings and TStrings)
  • root-ls: a command to inspect ROOT files
  • root-srv: a command that launches a local web-server from which one can inspect ROOT files and plot TH1x, TH2x and TTrees
  • root2npy: a command that converts “simple” TTrees into numpy data files
  • root2yoda: a command that converts ROOT files into YODA files

and that’s just the purely ROOT-related stuff.

With Go-HEP, Gonum (numpy/scipy for Go) and Go you can:

  • create nice plots,
  • do BLAS/LAPACK and linear algebra
  • do stats (univariate, multivariate, histograms, …)
  • etc…

Come to Go for the concurrency, and stay for everything else :slight_smile:

Cheers,
Sebastien.

1 Like
2

Hi,
this sounds great, especially the root2npy tool I think is something many people might benefit from.

I am no go expert, and I took a look at your website to see how a parallel loop over TTree events looks with this framework, but could not find anything. Could you point me to an example please :slight_smile: ?

3

FYI, I have updated the doc for root2npy:

wrt reading TTrees: the central type to do such a thing in Go-HEP is rootio.Scanner.
here are some examples:

as can be seen in the first example, the looping is performed like so:

type Data struct {
    I64    int64       `rootio:"Int64"`
    F64    float64     `rootio:"Float64"`
    Str    string      `rootio:"Str"`
    ArrF64 [10]float64 `rootio:"ArrayFloat64"`
    N      int32       `rootio:"N"`
    SliF64 []float64   `rootio:"SliceFloat64"`
}

sc, err := rootio.NewScanner(tree, &Data{})
if err != nil {
    log.Fatal(err)
}
defer sc.Close()

for sc.Next() {
    var data Data
    err := sc.Scan(&data)
    if err != nil {
        log.Fatal(err)
    }

    fmt.Printf("entry[%d]: %+v\n", sc.Entry(), data)
    if sc.Entry() == 9 {
        break
    }
}

you create a new data type Data, decorated to bind TTree branch names ("Int64") with struct fields (I64).
the rootio.Scanner will take care of populating the branch’s value into the corresponding Data's field value.

Modifying that loop to make it concurrent (and thus, with multi-cores, parallel) is as easy as:

pipe := make(chan Data)
// create a goroutine to extract data from the tree
// this will execute asynchronously
go func() {
    for sc.Next() {
        var data Data
        err := sc.Scan(&data)
        if err != nil {
            log.Fatal(err)
        }

        fmt.Printf("entry[%d]: %+v\n", sc.Entry(), data)
    }
    // signal we are done with feeding data
    close(pipe)
}()

then, one can create multiple consumers, feeding off that pipe channel.

results := make(chan MyResultType)
for i := 0; i < runtime.NumCPU()*2; i++ {
    go process(pipe, results)
}

// iterate over the results
for result := range results {
    fmt.Printf("result=%v\n", result)
}

where process is a regular Go function:

type MyResultType float64

func process(in chan Data, out chan MyResultType) {
    // extract data from the input channel
    for v := range in {
        // send some data down the line
        out <- MyResultType(math.Sqrt(v.F64))
    }
}

and voila.

here are a few pointers about the Go concurrency model:

(I can’t recommand the videos enough: Rob Pike is a great speaker.)

hth,
-s

1 Like
4

BTW, the root-srv command that is shipped with go-hep/rootio and serves a web server to inspect a ROOT file, is available on Google’s AppEngine:

(a slightly older (~4months) version is also served using CERN’s OpenShift: http://cern.ch/go-rootio)

5

For ROOT to numpy conversion, see: https://pypi.python.org/pypi/root_numpy

6

Different use-case I think?
One is a standalone command-line tool to quickly convert your (simple) data-set, the other is a python library with many more features but that requires you write your own script to do the conversion exactly as you want it.

In other words, you could write sbinet’s root2npy with root_numpy.

1 Like
7

@wlav,

yes.
there’s also: https://github.com/diana-hep/c2numpy

the nice things about root2npy – if I may say so myself – are:

  • it doesn’t require ROOT5 nor ROOT6 to be installed
  • it doesn’t require Python2 nor Python3 to be installed
  • it’s completely statically compiled (and can be easily cross-compiled to Linux, Windows, Darwin, …), so deployment is just a simple scp|Putty|curl away.

You just need git and go, and you’re good to go.

That said, root2npy isn’t able to read all the TTrees in the HEP creation. yet.

Here are a few cross-compiled root2npy binaries:


/afs/cern.ch/user/b/binet/public/go-hep/root2npy-darwin-amd64.exe
/afs/cern.ch/user/b/binet/public/go-hep/root2npy-linux-386.exe
/afs/cern.ch/user/b/binet/public/go-hep/root2npy-linux-amd64.exe
/afs/cern.ch/user/b/binet/public/go-hep/root2npy-linux-arm64.exe
/afs/cern.ch/user/b/binet/public/go-hep/root2npy-windows-386.exe
/afs/cern.ch/user/b/binet/public/go-hep/root2npy-windows-amd64.exe

they were cross-compiled with the official Go toolchain like so:

GOOS=linux   GOARCH=amd64 go build -o root2npy-linux-amd64.exe
GOOS=linux   GOARCH=386   go build -o root2npy-linux-386.exe
GOOS=linux   GOARCH=arm64 go build -o root2npy-linux-arm64.exe
GOOS=darwin  GOARCH=amd64 go build -o root2npy-darwin-amd64.exe
GOOS=windows GOARCH=amd64 go build -o root2npy-windows-amd64.exe
GOOS=windows GOARCH=386   go build -o root2npy-windows-386.exe
8

Ah, by transliterating ROOT core class headers and including them, you don’t need ROOT installed. So e.g. $ROOTSYS/include/TAttFill.h becomes (attfill.go):

type attfill struct {
	color int16
	style int16
}

and is then textually committed into go-hep, with an associated streamer to read two int16’s.

9

yep.

this was the first step to get me going.
the end game, now that I can more or less correctly decode TStreamers, is to automatically generate type and the associated streamers, sans having to go the pain of writing and maintaining all that tedious TFoo.cxx -> foo.go code.

and the ultimate HEP-world Gomination is to be able to actually write ROOT files/trees.