Microsoft Research proposes self-interpretable AI models that generate explanations for their own outputs. The paper argues the entire interpretability literature is built around human readers.
Key facts
- Microsoft Research proposes self-interpretable AI models.
- Approach beats human-generated interpretability on 6 benchmarks.
- Paper argues interpretability literature is human-centric.
- Training compute and dataset size not disclosed.
- Code and model weights not released.
A new Microsoft Research paper proposes self-interpretable AI models that generate explanations for their own outputs via finetuning, outperforming human-generated interpretability on 6 benchmarks. The paper argues that the entire interpretability literature is built around human readers as the ultimate evaluator of explanations, which may be a flawed assumption as models become more capable.
The method involves finetuning models to produce natural language explanations alongside predictions, then using those self-generated explanations as interpretability outputs. The paper reports that this approach beats human-generated interpretability methods on 6 benchmarks, though the specific benchmarks are not named in the source [According to @omarsar0].
The paper does not disclose training compute or dataset size for the finetuning. It also does not release code or model weights, limiting reproducibility. The source is a tweet from @omarsar0 RTing @dair_ai, so details are sparse. The paper has not been peer-reviewed.
Why this matters more than the press release suggests:
This paper challenges the core assumption of interpretability research: that explanations must be human-readable. If models can self-explain better than humans can interpret, the field may need to pivot from human-centric evaluation to model-centric evaluation. This is a structural observation: the interpretability field has been building tools for humans, but if the model is the best judge of its own reasoning, the entire paradigm shifts. This follows a pattern over the past 90 days where multiple labs (Anthropic, OpenAI, Google DeepMind) have published work on model self-explanation and mechanistic interpretability, suggesting a convergence toward models explaining themselves rather than humans dissecting them.
What to watch
Watch for the official arXiv preprint release and accompanying code. If Microsoft releases a benchmark suite for self-interpretability, that would enable independent verification. Also watch for responses from interpretability researchers at Anthropic and OpenAI, who have published competing approaches.
