KV Reuse Hazard Exponent (alpha)
1.53 dimensionless 2 se 1.5–1.56
The decay exponent of the KV/prefix-cache reuse hazard, measured at 1.53 (2 se: 1.50-1.56) on a public serving trace. It is the single free factor deciding whether flash tiering buys hours or months of cache residency, and therefore whether a 252 EB KV-NAND TAM closes. Nobody on either side of that debate had named the quantity.
1.53, against the 0.77 the disclosed TAM requires. On this workload flash buys about eight hours of residency, not the 2.6 days the disclosure needs.
Why it matters
Economic cache residency is set by a break-even, not by a product decision: hold the marginal byte while P(reuse) x recompute_saving > storage_cost. The RATIO of two tiers' windows collapses to (price ratio)^(1/alpha), with the hazard's scale and origin cancelling entirely. TLC is ~25x cheaper per GB-day than DRAM and QLC ~44x, so at alpha 2.0 a flash window is five hours and at alpha 0.4 it is months. Nothing else in the flash-tiering debate moves the answer as much, and every other factor in the break-even is public or an arithmetic bound.
Method
Moonshot's open-sourced Kimi serving trace — 12,031 requests, 182,790 distinct 512-token prefix blocks, a 3,537 s window.
Life-table hazard with CENSORED intervals kept in the risk set. The interval from a block's last access to the end of the trace is not 'never reused', it is 'not yet reused when the recording stopped'. Treating it as a completed observation manufactures decay out of the recording window and biases alpha upward — which biases the flash window down, which would have made this desk more confident against the TAM for a purely methodological reason.
The hazard PEAKS at 77 seconds, which is the human read-and-reply interval. The estimator recovers a physical timescale it was never told about.
What had to come out first
- timestamp quantisation12,031 requests carry only 1,180 distinct timestamps, up to 28 sharing one. The grid is ~3 s, so nothing below that is a measurement.
- concurrent fan-out10,852 intervals (10.3%) are exactly zero — two requests touching the same prefix at the same instant. That is fan-out across simultaneous sessions, not a block aging.
- the universal root blockBlock 0 appears in all 12,031 requests (the shared system prompt) and contributes ~11% of intervals, every one short. It is also economically degenerate: a block every request touches never leaves the top tier, so it is never the marginal byte.
- fitting across the hazard peakThe hazard rises to 77 s before decaying. A single power law fitted across the hump averages the rising limb into the falling one and returns a slope far too shallow.
Robustness
The verdict survives a 20x swing in the assumed token denominator (1e17 to 2e18 tokens/yr). It flips only at 3.83e18 — about 100x the one disclosed single-operator figure — and at that volume the required residency is 8.2 hours, which IS reachable. So the TAM has a VOLUME route even where the residency route fails, and that is the honest form of the caveat.
What it does not settle
ONE OPERATOR, ONE HOUR, CONVERSATION TRAFFIC. Gaps beyond 3,537 s are unobservable, so carrying the power law to a 2.6-day residency is a ~60x extrapolation beyond the observation window. A chat session reuses its prefix on a human turn-taking clock; an agent re-reading a document corpus is a different process. Alpha should be held as a PER-WORKLOAD quantity, not a constant.
What would refute it
- an agentic or document-retrieval trace fitting alpha at or below 0.77 on its decaying tail — this is the single measurement that could reopen the capacity-grade leg, and on the workload that is growing
- a multi-day trace showing the power law breaking to something flatter beyond the one-hour window this was fitted in
- a hyperscaler RFQ for QLC KV or context-lake capacity, or a QLC contract-price inflection, either of which would say the long-residency window opened despite the hazard
Where it converges
The TLC/QLC price tape. Economics say the QLC window only opens if the hazard is flat; the tape shows TLC firming and QLC contract pricing flat, i.e. the window has not opened. A derivation and a price agreeing from entirely unshared inputs.
Computed by
- tools/reuse_hazard.py
- tools/residency_economics.py
Sources
- Open-sourced Kimi serving trace: request arrival times and 512-token prefix block hashes — https://github.com/kvcache-ai/Mooncake (accessed 2026-08-15)
- H100-class rental $2-4/GPU-hr late 2025, used for the recompute side of the break-even — no URL (accessed 2026-08-15)