Is this a good TTreeCache Stats print out?

ROOT
1

Hi,
After running on about 18 GB of files (and a small sub-set of branches), I get the following from PrintCacheStats. Is this a symptom of a well configured cache?

[03:13:48][Step 4/4] ******TreeCache statistics for tree: CollectionTree in file: \\...\user.omh.007168.AANT._00222.root ****** [03:13:48][Step 4/4] Number of branches in the cache ...: 18 [03:13:48][Step 4/4] Cache Efficiency ..................: 0.999986 [03:13:48][Step 4/4] Cache Efficiency Rel...............: 0.999959 [03:13:48][Step 4/4] Learn entries......................: 100 [03:13:48][Step 4/4] Cached Reading.....................: 10691668648 bytes in 59796 transactions [03:13:48][Step 4/4] Reading............................: 449659 bytes in 23 uncached transactions [03:13:48][Step 4/4] Readahead..........................: 256000 bytes with overhead = 1115894386 bytes [03:13:48][Step 4/4] Average transaction................: 178.802406 Kbytes [03:13:48][Step 4/4] Number of blocks in current cache..: 2226, total size: 42304900

-Gordon.

2

Hi Gordon,

Yes but could be better. There is a lot of individual read done during the learning phase, if you were to teach the cache which branch you will be reading rather then letting it figure out, you will get better performance for the first 100 entries (and same good performance for the rest).

Cheers,
Philippe.

3

Thanks!

That is very weird. Because I auto-generate the C++ code, I know before hand what leaves are needed for training (unless I have a bug), so I turn off training. Here is the ramp-up to actually running the loop.

[code] tree.CacheSize = 1024 * 1024 * 100; // 100 MB cache
foreach (var leaf in LeafNames)
{
tree.AddBranchToCache(leaf, true);
}
tree.StopCacheLearningPhase();

        // Always Do the async prefetching (this is off by default for some reason, but...).

        ROOTNET.Globals.gEnv.Value.SetValue("TFile.AsynchPrefetching", 1);

        ///
        /// Finally, run the whole thing
        /// 

        TraceHelpers.TraceInfo(21, "RunNtupleQuery: Running TSelector");
        if (Environment.BreakToDebugger)
            System.Diagnostics.Debugger.Break();
        tree.Process(selector);
        TraceHelpers.TraceInfo(22, "RunNtupleQuery: Done");

[/code]

Does that code look like it should be turning off the learning phase? Or if it does, does this indicate that some other branch is getting read and I missed it. And if so, how can I tell which branch it is (as this would be a bug in my framework that I’d like to solve!!).

4

Hi Gordon,

This code looks fine. Is it plausible that there is a call to GetEntry before does like are executed?

[quote]Or if it does, does this indicate that some other branch is getting read and I missed it. And if so, how can I tell which branch it is (as this would be a bug in my framework that I’d like to solve!!).
[/quote]It is not impossible :slight_smile:. There is no direct way to know which branch is missing from the cache. For now you would need to put a break point in TBasket::ReadBasketBuffers around line 480 ( // Read directly from file, not from the cache).

Cheers,
Philippe.

5

No, no way GetEentry is called first. Ok. Perhaps I turn on the learning phase and see if that makes the number of single reads go down.

6

Wait. In 17 gb, is 23 reads a lot? Well, half a meg…

7

Hi Gordon,

Yes, this is a small effect in this case, probably nothing to worry about but is still some unexpected (zero is expected :slight_smile:).

Cheers,
Philippe.

PS. Are you calling SetEntryRange?

8

I just searched my code base - it is never called. I assume that is as desired, right? I will do some other tests - for example, read only one leaf and see if things work well. Oh, is there a way I can tell the TTreeCache to dump out the leaves it is reading as a diagnostic? That way I could really check my code was predicting the reading…

9

Hi Gordon,

That you can do by calling TTreeCache::GetCachedBranches (grab the TTreeCache with TTreeCache tc = (TTreeCache)ttree->GetFile()->GetCacheRead(tree):wink:

Cheers,
Philippe.