Improve I/O when reading a TTree

ROOT
1

Hi,

I’m trying to read a 10 GB root flat tree (compressed 60 GB uncompressed). Is there a way to improve the I/O efficiency of reading the TTrees into memory. I’ve been playing around with SetMaxVirtualSize and LoadBaskets as previous topics mentioned an improvement in reading in data. As the file is too big to place all the Tree into memory at once what happens when the you reach the allocated virtual size? Is there any best practices for improving the I/O?

Cheers
Dom

2

Hi Dom,
Let’s first figure out what the issue is if any, before trying to tweak things.

What’s the read speed you observe? Is that I/O or CPU limited (i.e. what’s the CPU usage during reads)?
Cheers, Axel.

3

Hi,

I believe it is I/O limited. The CPU efficiency is roughly ~60% for one job. However the job submission procedure means that multiple jobs are being sent to one server node. When there is multiple jobs running on the node the efficiency drops to around 10%. I was thinking I maybe could read the data into memory more efficiently maybe with LoadBaskets and that would fix the issue.

Thanks for the help
Dom

4

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.