Replacing TTree I/O Traversal with Arithmetic: A Coordinate-Indexed Read Path for CMS Open Data


The SSCCS Foundation is a non-profit organization developing open-source infrastructure to lower the barriers to scientific computing. One of our foundational projects, synTagma, is a coordinate engine that maps multidimensional coordinates, such as run, luminosity block, and event number, directly to memory addresses using closed-form arithmetic. It computes addresses in O(1) with no hash tables, B-trees, or index scans.

As we sought to validate synTagma at a production level, focusing on real-world impact and public utility, we came across the well-documented I/O bottleneck in CERN ROOT’s TTree.

The CERN ROOT TTree Problem

ROOT is the de facto data analysis framework for high-energy physics. Its TTree format stores billions of collision events in a columnar layout. The bottleneck lies in how a single event is read.

The legacy path follows a complex traversal: Branch, Basket, Cache, File. Each step accumulates system calls, cache misses, and context switches. The 2025 Fermilab CCESOP analysis documents the production signature: 372,000 singular reads averaging 4.6 KB, at an effective 33 KB/s, over roughly 14 hours. The cost is per request, not per byte; about 1.6 GB moves in seconds at raw bandwidth.

Two bottleneck regimes matter here. In the remote production pattern, per-request I/O wait dominates, around 98.8 percent of wall time, while the CPU stays mostly idle. On a local full-file measurement, the baseline is decompression CPU bound (CPU 182.9 s of 192.8 s wall). Both regimes are structural properties of the traversal, and both disappear when the traversal is replaced.

Replacing Traversal with Arithmetic

We integrated the synTagma C++ core into a ROOT fork and replaced the traversal path with a single arithmetic operation:

Index = (r × L + ℓ) × E + e, where L is the maximum number of luminosity blocks and E the maximum number of events per block. Byte Offset = Index × record size.

Given (Run, Luminosity Block, Event Number), the byte offset resolves in one step. No hash, no B-trees, no cache lookups.

Crucially, the TTree API remains unchanged. Analysis code, build chains, and researcher workflows stay exactly the same. We replaced only the interior read path, leaving the researcher interface untouched.

Measured Results: Validated on CMS Open Data

We measured the full CMS Run2016G DoubleMuon NanoAOD first file (2,315,223 events, about 2 GB) on the same local medium.

Read Path Wall Time Speedup
Legacy ROOT 192.8 s
Coordinate (standard I/O) 2.79 s 69.0×
Coordinate (memory-mapped) 1.06 s 181.7×

The memory-mapped path issues zero application-level read system calls; every event is copied directly from kernel-mapped memory, and the mapped pages are demand-paged once by the kernel.

Correctness was verified at two levels: the served bytes match a sidecar checksum, and an analysis workload over the same events reproduced identical results on both paths, 20,861 selected events with exact histogram equality, at 60.9×. The one-time store conversion cost 226.3 s, comparable to a single baseline read pass and amortized from the second analysis pass onward.

One boundary, stated plainly: the documented 14-hour production workload is the reference pattern, and the measured scale is the full local file. We reproduced the access signature, reads of 1.37 to 2.76 per event at 2.6 to 0.8 KB per read, rather than the remote end-to-end run. The recorded rows are the committed benchmark artifact in the fork, reproducible from the runbook.

Public Contribution and Open Access

Beyond publishing the code, we wanted a permanent academic record. We registered the technical report on Zenodo, CERN’s open research repository, which assigns a free DOI to every research output. It closes a loop: using CERN’s infrastructure to share an open solution to a CERN problem.

Why This Matters

The pattern repeats beyond ROOT. Genomics, astrophysics, and climate modeling all share pointer chasing, cache misses, and many tiny reads. What we demonstrated is a structural principle: when traversal is replaced with arithmetic, per-request overhead disappears. The address becomes the traversal.

The technical report, implementation, and benchmarks are all open and linked below.

  • Technical report (DOI): doi.org/10.5281/zenodo.21888670
  • Technical report: docs.ssccs.org/works/cern/root-ttree
  • ROOT fork: github.com/ssccsorg/root
  • synTagma C++ core: github.com/ssccsorg/syntagma

ROOT Version: 6.41/01 (development version, master)
Platform: macOS 15.7.1 (arm64), Apple M1 Max, single Arm Firestorm 3.2 GHz core
Compiler: AppleClang 17.0.0 (clang-1700.6.3.2), C++17


1 Like