HDF5 to root

[quote=“delaossa”]Hello,
Thanks for the reply, but I think I didn’t explain properly my problem.

  • We would like to convert data files from hdf5 to root format.
  • The program proposed in this post uses the C++ interface to HDF5.
  • The HDF5 version that we have doesn’t support C++ (there is no C++ support for the MPI I/O version of HDF5).
  • I wonder if there is a program around to convert from hdf5 to root which uses C HDF5 interface instead of the C++ one.

Any help or advice?
Thanks!

PS: I have also found this linux.softpedia.com/get/Utilitie … 1518.shtml .
But again, it uses the C++ the interface to HDF5.[/quote]

I think we did get your explanation. We are (trying to) tell you to write a C++ function (declared in an extern “C” block that you can call from openmpi. This C++ wrapper function will be able to call Root.

Cheers,
Charles

Hi,

See www2.research.att.com/~bs/bs_faq2.html#callCpp

Philippe.

Hi,
Excuse me, please, if I am being annoying, but I know that C++ functions can be called from C and viceversa.
However, I will never be capable to use the Giuseppe Vacanti’s program to convert hdf5 files to root ones if I don’t have neither the C++ headers nor the C++ libraries for HDF5:
H5Cpp.h and libhdf5_cpp.so don’t exist in my system, but hdf5.h and libhdf5.so.

Cheers, Alberto

[quote=“delaossa”]Hi,
Excuse me, please, if I am being annoying, but I know that C++ functions can be called from C and viceversa.
However, I will never be capable to use the Giuseppe Vacanti’s program to convert hdf5 files to root ones if I don’t have neither the C++ headers nor the C++ libraries for HDF5:
H5Cpp.h and libhdf5_cpp.so don’t exist in my system, but hdf5.h and libhdf5.so.

Cheers, Alberto[/quote]

Yes, you won’t be able to use his program. But you ought to be able to use it to figure out how to write your own. If you don’t want to do that, then maybe your effort would best be spent finding the libraries so that his will work for you as is.

Cheers,
Charles

Ok! That’s what I thought… thanks.
Anyway, I managed to compile C++ libraries for HDF5 and then, I could finally run this program as it is (pure C++).
However, it is not working properly with my data files and I don’t know why.
It fails in line 89:

  const int nm = type.getNmembers();

throwing the following message:

[quote]HDF5-DIAG: Error detected in HDF5 (1.8.7) thread 0:
#000: H5Tfields.c line 90 in H5Tget_nmembers(): cannot return member number
major: Invalid arguments to routine
minor: Inappropriate type
#001: H5Tfields.c line 131 in H5T_get_nmembers(): operation not supported for type class
major: Invalid arguments to routine
minor: Inappropriate type
terminate called after throwing an instance of ‘H5::DataTypeIException’
Abort[/quote]
Here I attach the file which I am trying to convert:
gamma.h5.tar.gz (842 Bytes)
It is a very simple one with just 1 dataset called ‘gamma_|charge|’ under the main ‘/’ group.
Any clues?

Hello all,
After several days struggling with the HDF5 to ROOT conversion, I have created a program based on the original one posted by Giuseppe Vacanti:hdf2root.tar.gz (3.74 KB)
It converts the whole HDF5 structure into TTrees:
For every Group in the HDF5 file, hdf2root creates a TTree (named as the orginal Group) which stores the info contained in the original DataSets. Moreover, the Attributes of the Group are stored in another TTree (named as the original group plus “_att”), together with the Attributes of every DataSet in the Group.
The program is thought to work in any kind of HDF5 file and make a “general” conversion.
It works in all the data files I have tested, which amounts to 4 or 5 different types (only!).
A clear limitation would be how to handle more complex DataTypes. Up to know, only simple types are supported, but this could be enough for many users (it is actually enough for the files I have to handle!).
The usage is quite simple:

Usage: hdf2root [--seq] <inFile.h5> <outFile.root(default=inFile.root)>
 
            --seq  Enables sequential mode.

hdf2root just needs the original HDF5 file as input to create a root file with the same name. If you want a different output name, put it in as a second argument. If the –seq is set, hdf2root create the TTrees for the DataSets in sequential mode. This means that the first dimension of the data set is used like an “event index” (if you know what I mean…).

I aim you (interested people) to take a look and send some feedback.
One could start with the example file I posted in a previous post in this thread (gamma.h5).

Warning: I am only experiencing problems with the conversion of hdf5_strings mostly present in Attributes.
The size of these objects is not 1 to 1 with their length and, as a result, larger strings of characters are stored in root unnecessarily. This could be a problem of the particular hdf5 files I am using, instead of the converter itself.

I hope it helps to the community.

Cheers,
Alberto

Hi all,

This also looks promising: amas.web.psi.ch/tools/H5root/index.html

I have not tried it yet, but there is an interactive tool as well as a library that you can use to open/read/plot hdf5 files from the root prompt…

Hi delaossa,

I tried to compile the program you attached but I get the msg below. I am guessing that I am missing mpif77. I tried to install that through yum searching for openmpi but none of the packages I found help. Do you know if mpif77 is compatible with SLC6?

======================================
[wht34@icarus hdf2root]$ make
make: mpif77: Command not found
Linking hdf2root
hdf2root.o: In function main': /home/wht34/MyCodes/hdf2root/hdf2root.cpp:394: undefined reference toH5check_version’
/usr/lib/gcc/x86_64-redhat-linux/4.4.7/…/…/…/…/lib64/libhdf5_cpp.so: undefined reference to H5Tunregister' /usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../lib64/libhdf5_cpp.so: undefined reference toH5set_free_list_limits’

Hello,

Try this new version:hdf2root.tar.gz (4.86 KB)
You will need a root installation and the HDF5 libraries for C++.
There are a configuration script where you can define the paths to the needed libraries.
Let me know wether you succeed to compile and or not.

Cheers,
Alberto

Hi Alberto,

I modified the env.sh file to point to my root installation directory. I am not sure if I pointed the hdf5 interface correctly but they’re the only hdf5 libraries I found on my computer. Make is still complaining about
undefined reference to `H5check_version’ and so on with all these H5 commands being undefined.

Thanks,

Wing

======================
#!/bin/bash

Setting up Root environment…

#export ROOTSYS=/usr/local/root
export ROOTSYS=/usr/share/root
export DYLD_LIBRARY_PATH=$ROOTSYS/lib:$DYLD_LIBRARY_PATH
export PATH=$ROOTSYS/bin:$PATH

HDF5: C++ interface

#export HDF5CPP=/usr/share/hdf5-c++
export HDF5CPP=/usr/bin/h5c++
#export DYLD_LIBRARY_PATH=$HDF5CPP/lib:$DYLD_LIBRARY_PATH
export DYLD_LIBRARY_PATH=/usr/lib64/:$DYLD_LIBRARY_PATH

============================
[wht34@icarus hdf2root]$ make
Linking hdf2root.e
g++ -fPIC -pthread -m64 -I/usr/share/root/include -I/include -I. hdf2root.cc -o hdf2root.e -L. -L/usr/share/root/lib -lCore -lCint -lRIO -lNet -lHist -lGraf -lGraf3d -lGpad -lTree -lRint -lPostscript -lMatrix -lPhysics -lMathCore -lThread -pthread -lm -ldl -rdynamic -L/lib -lhdf5_cpp
/tmp/wht34/ccAALUJ7.o: In function main': hdf2root.cc:(.text+0x2834): undefined reference toH5check_version’
/usr/lib/…/lib64/libhdf5_cpp.so: undefined reference to `H5Tunregister’

Hello,

The compiler also has to know where the headers for HDF5 are.
If in the configuration script you define the environment variable HDF5CPP, the Makefile would define the path to the HDF5 headers in this way: $HDF5CPP/include.

Make sure that the file H5Cpp.h is located there.

Cheers,
Alberto

IMPORTANT:

I somehow messed up the code in the version that I uploaded yesterday.
It was working, but it was full of unnecessary repeated function definitions.

Download please the following (fix) version:
hdf2root.tar.gz (4.39 KB)

Cheers,
Alberto

Hi Alberto,

The code compiles now but I run into this runtime error now. I have libhdf5_cpp.so.6 but not 8 in my /usr/lib64.

[wht34@icarus hdf2root]$ ./hdf2root --ver lux10_20130611T0605_f000001_eb00014.hdf5 lux10_20130611T0605_f000001_eb00014.hdf5.root
./hdf2root: error while loading shared libraries: libhdf5_cpp.so.8: cannot open shared object file: No such file or directory
[wht34@icarus hdf2root]$ ldd hdf2root.e | grep libhdf5_cpp.so.8
libhdf5_cpp.so.8 => not found

Okay, I found these libraries in

/usr/local/hdf5/lib/ … but the program is looking for it in /usr/lib64…

If i make symblic links for the two missing libraries. The program runs but doesn’t give any output.

sudo ln -s /usr/local/hdf5/lib/libhdf5.so.8 /usr/lib64/libhdf5.so.8
sudo ln -s /usr/local/hdf5/lib/libhdf5_cpp.so.8 /usr/lib64/libhdf5_cpp.so.8
[wht34@icarus hdf2root]$ ./hdf2root lux10_20130611T0605_f000001_eb00014.hdf5 lux10_20130611T0605_f000001_eb00014.hdf5.root
Converting …
[wht34@icarus hdf2root]$ ll
total 12M
-rw-r–r--. 1 wht34 caselux 275 Mar 25 12:06 env.csh
-rwxr-xr-x. 1 wht34 caselux 373 Mar 26 15:04 env.sh
lrwxrwxrwx. 1 wht34 caselux 10 Mar 26 15:53 hdf2root -> hdf2root.e
-rw-r–r--. 1 wht34 caselux 14K Mar 26 14:44 hdf2root.cc
-rwxr-xr-x. 1 wht34 caselux 157K Mar 26 15:53 hdf2root.e
-rw-r–r--. 1 wht34 caselux 462 Mar 26 14:44 hdf2root.hh
-rw-r–r--. 1 wht34 caselux 12M Mar 26 14:55 lux10_20130611T0605_f000001_eb00014.hdf5
-rw-r–r--. 1 wht34 caselux 615 Mar 26 15:06 Makefile

Hello,

If the hdf5 libraries are located at /usr/local/hdf5/lib, the include files should be at /usr/local/hdf5/include.
In the configuration script you should define:
export HDF5CPP= /usr/local/hdf5
export LD_LIBRARY_PATH=$HDF5CPP/lib:$LD_LIBRARY_PATH

By the way, in which system are you trying to build the program?
DYLD_LIBRARY_PATH is used for macos, for linux systems it is LD_LIBRARY_PATH the variable which holds the path to the needed libraries.

Cheers,
Alberti

Hi Alberto,

I am on SLC6, so I changed the DYLD_LIBRARY_PATH to LD_LIBRARY_PATH as you showed. The code compiles but nothing happen when I ran the executable.

[wht34@icarus hdf2root]$ echo $HDF5CPP
/usr/local/hdf5
[wht34@icarus hdf2root]$ echo $LD_LIBRARY_PATH
/usr/local/hdf5/lib:/usr/share/root//lib:/usr/share/clhep/install/lib:/usr/share/geant4/install/lib/geant4/Linux-g++:/usr/share/clhep/install/lib:/usr/share/root//lib

[wht34@icarus hdf2root]$ make
Linking hdf2root.e
g++ -fPIC -pthread -m64 -I/usr/share/root/include -I/usr/local/hdf5/include -I. hdf2root.cc -o hdf2root.e -L. -L/usr/share/root/lib -lCore -lCint -lRIO -lNet -lHist -lGraf -lGraf3d -lGpad -lTree -lRint -lPostscript -lMatrix -lPhysics -lMathCore -lThread -pthread -lm -ldl -rdynamic -L/usr/local/hdf5/lib -lhdf5_cpp

[wht34@icarus hdf2root]$ ./hdf2root.e test.hdf5
Converting …
[wht34@icarus hdf2root]$ ll
total 24M
-rw-r–r--. 1 wht34 caselux 275 Mar 25 12:06 env.csh
-rwxr-xr-x. 1 wht34 caselux 365 Apr 2 14:48 env.sh
lrwxrwxrwx. 1 wht34 caselux 10 Apr 2 14:52 hdf2root -> hdf2root.e
-rw-r–r--. 1 wht34 caselux 14K Mar 26 14:44 hdf2root.cc
-rwxr-xr-x. 1 wht34 caselux 157K Apr 2 14:52 hdf2root.e
-rw-r–r--. 1 wht34 caselux 462 Mar 26 14:44 hdf2root.hh
-rw-r–r--. 1 wht34 caselux 615 Mar 26 15:06 Makefile
-rw-r–r--. 1 wht34 caselux 12M Mar 26 16:11 test.hdf5

Hello,
Thanks to your interest I have updated the code into a more professional fashion.
I have tried it and it works in both linux and mac os platforms.
Give it a try: github.com/delaossa/hdf2root

In the config folder there are various examples of configuration files.
Pick up the most similar to your system and modify it accordingly.
Good luck!

Hi Alberto,

The git version of your code compiles fine but the executable still doesn’t do anything to the hdf5 test file.
It’ll just say converting … , but no root output comes out with hdf2root test.hdf5 test.root.
Here’s the log for the compilation and my env.sh file.

#!/bin/bash

MAC flag

#export PLATFORM=linux
export ROOTSYS=/usr/share/root/
export LD_LIBRARY_PATH=$ROOTSYS/lib:$LD_LIBRARY_PATH
export PATH=$ROOTSYS/bin:$PATH
export HDF5CPP=/usr/local/hdf5
export LD_LIBRARY_PATH=$HDF5CPP/lib:$LD_LIBRARY_PATH
export HDF2ROOT=/home/wht34/SVN/Scratch/wht34/hdf2root-master
export LD_LIBRARY_PATH=$HDF2ROOT/lib:$LD_LIBRARY_PATH
export PATH=$HDF2ROOT/bin:$PATH

wht34@icarus hdf2root-master]$ source config/env.sh
[wht34@icarus hdf2root-master]$ make
make[1]: Entering directory /home/wht34/SVN/Scratch/wht34/hdf2root-master/src' Compiling hdf2root_methods.cc g++ -c -O2 -Wall -fPIC -pthread -m64 -I/usr/share/root/include -I/usr/local/hdf5/include -I/home/wht34/SVN/Scratch/wht34/hdf2root-master/inc -I. hdf2root_methods.cc Building shared library /home/wht34/SVN/Scratch/wht34/hdf2root-master/lib/libhdf2root.so ... g++ -shared hdf2root_methods.o -o /home/wht34/SVN/Scratch/wht34/hdf2root-master/lib/libhdf2root.so -L/usr/share/root/lib -lCore -lRIO -lTree -L/usr/local/hdf5/lib -lhdf5 -lhdf5_cpp Done Compiling hdf2root.cc g++ -c -O2 -Wall -fPIC -pthread -m64 -I/usr/share/root/include -I/usr/local/hdf5/include -I/home/wht34/SVN/Scratch/wht34/hdf2root-master/inc -I. hdf2root.cc Linking hdf2root.o g++ -O2 -m64 -o hdf2root.e hdf2root.o -L/usr/share/root/lib -lCore -lRIO -lTree -L/usr/local/hdf5/lib -lhdf5 -lhdf5_cpp -L/home/wht34/SVN/Scratch/wht34/hdf2root-master/lib -lhdf2root ln -sf /home/wht34/SVN/Scratch/wht34/hdf2root-master/src/hdf2root.e /home/wht34/SVN/Scratch/wht34/hdf2root-master/bin/hdf2root rm hdf2root.o make[1]: Leaving directory/home/wht34/SVN/Scratch/wht34/hdf2root-master/src’

Strange… What happens if you just execute the hdf2root program without any input parameters?
This is what i get:

[code][Behemoth] ~/plasma/software/hdf2root $> hdf2root

Usage: hdf2root [–seq] <inFile.h5> <outFile.root(=inFile.root)>

        --seq  Enables sequential mode. 
        --ver  Enables print out. 

[/code]

I get the same thing. Do you have a hdf5 file that I can test this program on? Maybe the file I have may not work with your program.

[wht34@icarus hdf2root-master]$ hdf2root

Usage: hdf2root [–seq] <inFile.h5> <outFile.root(=inFile.root)>

        --seq  Enables sequential mode. 
        --ver  Enables print out.