The Interpretability Tax

I named this thing in an earlier post — the Interpretability Tax, the compute and storage and human attention you spend making a model's behaviour legible — then wrote a whole post about it without quoting a price. That's the failure mode of a good-sounding term. So: prices.

DeepMind's Gemma Scope, released 31 July 2024, is the clearest public figure I know of. Training the suite — 400-plus sparse autoencoders, over 30 million learned features across Gemma 2 2B and 9B — took about 15% of the training compute of Gemma 2 9B and wrote roughly 20 pebibytes of activations to disk, which the announcement sizes at about a million copies of English Wikipedia.

A fifth of a training run, for a feature dictionary covering one model family at one snapshot of its weights, that everyone involved describes as incomplete.

Anthropic's Scaling Monosemanticity pulled a comparable haul out of Claude 3 Sonnet, and Dario Amodei's read on that: "we've found only a small fraction of what is probably there", with maybe "a billion or more concepts in even a small model." Compute is the cheap half anyway. The expensive half is a person — Anthropic's circuit-tracing write-up of 27 March 2025 says it takes "a few hours of human effort to understand the circuits we see, even on prompts with only tens of words," and even then the method "only captures a fraction of the total computation performed by Claude."

A few hours per prompt, on prompts of a few dozen words, against a gateway serving tens of thousands of requests a day.

Nor does the tax come with a receipt. Kantamneni et al., ICML 2025 put SAEs on a real downstream task — probing activations under data scarcity, class imbalance, label noise and covariate shift — and SAE probes couldn't consistently beat plain logistic regression. Several insights they got credit for were reachable with simpler baselines.

What ships instead

None of that is my week. The interpretability that ships on a platform team is attribution: which key called which model, how many input and output tokens, what it cost, on whose budget. Usage tracking. Spend dashboards. Rate limits with a name attached.

The arguments about that layer happen in single-digit milliseconds, and the gateway troubleshooting docs point at the interesting header — x-litellm-callback-duration-ms, time spent building logging payloads, with the note that above 100 ms your payloads are too large. The overhead everyone worries about is on the inference path and is negligible. The one that bites is on the recording path, and it arrives as storage, retention windows and payload size rather than FLOPs.

OpenTelemetry's GenAI conventions have already priced this. Token counts, model names and durations are default telemetry; prompt and completion content is opt-in, because "by default, no prompt content or tool arguments are captured with GenAI telemetry, as these can contain sensitive data".

Counting is cheap and safe. Keeping the words is expensive and hazardous, and hardly anyone turns it on across the board.

Where the two meet

Mostly they don't, and the regulation sits closer to my side than to the research side. High-risk obligations under the EU AI Act bind from 2 August 2026, and Article 86 gives an affected person the right to "clear and meaningful explanations of the role of the AI system in the decision-making procedure and the main elements of the decision taken."

Read as a spec, that asks for provenance, not mechanism. Role in the procedure, main elements, which system, which inputs, what a human did with the output — all of it satisfiable from records a competent platform already keeps, with no feature dictionary anywhere in it. Much of what ships this year under the word "explainability" will be metering with a compliance label on it.

Defensible, and still silent on the question people think they're asking. My dashboards answer what happened, who paid, how much. When behaviour shifts after a model version bump they tell me cost per request moved and nothing about why. The numbers above are why I've never had better on hand: a fifth of a training run plus hours of specialist time per prompt, for a partial map of weights that sit behind someone else's API.