I have a problem that I’m working on where I need to analyse a large amount of data. The analysis will require that the data are organized by time and by geographic location (among other factors). I am considering two strategies and was hoping that I might get some guidance from the ROOT community in terms of I might best proceed. The first option I am considering is breaking the dataset into multiple TFiles/TTree where each file contains data for a particular interval of time and a particular location. This will likely result in many small files. The second option I’m considering is to put all the data into a single file/tree. The question I have is what are the trade-offs between having the data pre-filtered and organized into chunks (multiple files) vs. using TSelector and/or cuts in the tree searching mechansims within a single file. I expect there will be a lot of performance penalty with accessing multiple files and so forth. On the other hand, I assume there is a considerable cost in filtering values from a large tree.
Thank you for any advice.