Speed of storing columnar data

Hello!

I am quite new to ROOT and trying to determine if this would be a good framework for a project I have. I have a device that is producing packets of binary data at a rate of GB/s, and I need to store this data into some kind of file as quickly as possible. ROOT files seem to have a lot of nice features, specifically GB/s reading and being a compressed binary format, but I was wondering how quickly ROOT can store tabular data?

Thanks!

Hi @clu ,

welcome to the ROOT forum.
@pcanal or @Axel can comment with more authority, but as far as I know the bottleneck is typically data compression or the disk writing bandwidth, not ROOT itself.

(If your application is multi-threaded, the simplest way to not create synchronization bottlenecks is to have each thread write a different file, but there are also solutions for multi-thread writing to the same file).

I hope this helps!
Enrico

1 Like

Not much to add :slight_smile:

The fastest writing you get is to write uncompressed blobs of your raw bytes to raw FILEs (or mmapped etc). Use ROOT if you want to write structured, columnar data.

1 Like

Got it, thank you and @eguiraud !