Retrieving data from Web Server

[amvargash]

Hi Everyone,

TFile can be used to fetch a ROOT file with something like TFile::Open("http://root.cern.ch/files/hsimple.root"). Is there any way (workaround/hack) to retrieve a json/csv data file from a web server?

I tried with TWebFile("https://someserver/somefile.csv") but It looks like it is only intended to fetch ROOT files exclusively

EDIT: The goal is to fill a TTree with the csv data by means of TTree::ReadFile()

Any suggestions appreciated!

All the best,
Andres

[Danilo]

Hi,

I don’t think that presently there is a way to natively read from ROOT remote files which are not in the root format.

Cheers,
D

[amadio]

My suggestion is for you to use Python, download the CSV file using the requests module, then load it locally via the interfaces offered by ROOT (TDataFrame’s CSV data source).

import requests

url = "http://someserver/somefile.csv"
response = requests.get(url)
data = [ line.split(',') for line in response.contents.splitlines() ]

# use data here

[linev]

I can just confirm @Danilo comment - there is no dedicated classes in ROOT to read data from http servers.
Potentially one could reuse a lot of code, hidden behind TDavixFile, but this was never done before.
For the moment I can suggest to use external libraries like https://curl.haxx.se/libcurl/.

[amvargash]

Thank you guys for the suggestions. I opened a JIRA Request ROOT-9229 as @linev suggested perhaps code can be reused to implement this function directly into TTree::ReadFile(). Thanks again!

[sbinet]

only tangentially relevant but, in Go:

import (
    "io"
    "os"
    "net/http"
)

resp, err := http.Get("https://example.org/somefile.csv")
if err != nil { panic(err) }
defer resp.Body.Close()

f, err := os.Create("out.csv")
if err != nil { panic(err) }
defer f.Close()

_, err = io.Copy(f, resp.Body)
if err != nil { panic(err) }

alternatively, to directly process the CSV file, without going through the creation of a temporary file:

import (
    "encoding/csv"
    "fmt"
)

r := csv.NewReader(resp.Body)
r.Comma = ','
records, err := r.ReadAll()
if err != nil { panic(err) }
for _, record := range records {
    fmt.Printf("%v\n", record)
}

and, if you were to use go-hep:

import (
    "fmt"
    "go-hep.org/x/hep/hbook/ntup/ntcsv"
)

nt, err := ntcsv.Open("https://someserver.org/file.csv", ntcsv.Comma(','))
if err != nil { panic(err) }
defer nt.DB().Close()

for _, col := range nt.Cols() {
    fmt.Printf("%s: %v", col.Name(), col.Type())
}

and, eventually, when go-hep/rootio gets support for writing ROOT files:

import (
    "fmt"

    "go-hep.org/x/hep/hbook/ntup"
    "go-hep.org/x/hep/hbook/ntup/ntcsv"
    "go-hep.org/x/hep/rootio/ntroot"
)

nt, err := ntcsv.Open("http://someserver.org/file.csv", ntcsv.Comma(','))
if err != nil { panic(err) }
defer nt.DB().Close()

for _, col := range nt.Cols() {
    fmt.Printf("%s: %v", col.Name(), col.Type())
}

out, err := ntroot.Create("out.root")
if err != nil { panic(err) }

err = ntup.Copy(out, nt)
if err != nil { panic(err) }

the last snippet is pure science-fiction for the moment