DARPA's AIQ program, one year in, is shifting focus from building better benchmarks to a science of AI capability. Program lead @patrickshafto announced the pivot via @percyliang, signaling a structural rethinking of how AI evaluation works.
Key facts
- AIQ program launched in 2025
- Pivot announced by @patrickshafto via @percyliang
- Moving beyond benchmarks to capability science
- DARPA can mandate evaluation for defense contractors
- Annual review scheduled for Q3 2026
DARPA's AIQ program, launched in 2025 to create rigorous, theory-driven methods for evaluating AI systems, is moving beyond the traditional benchmark arms race. According to @percyliang, who retweeted program lead @patrickshafto, AIQ is now pursuing "a science of AI capability: measuring what AI systems can actually do."
This pivot comes amid growing frustration in the AI research community that popular benchmarks like MMLU, GSM8K, and HumanEval are increasingly saturated or gamed. [As previously reported by The Information], many state-of-the-art models now achieve 90%+ on these tests, yet still fail on simple real-world tasks—a phenomenon often called "overfitting to the benchmark."
AIQ's new direction focuses on developing causal, interpretable measures of capability rather than aggregate scores. The program is funding multiple academic and industry teams to build evaluation frameworks that can predict out-of-distribution performance and identify failure modes, according to DARPA's public program description.
The shift mirrors broader trends in AI safety and evaluation. Anthropic has published work on "situational awareness" evaluations, while OpenAI's Preparedness Framework uses structured capability assessments. But AIQ's government backing gives it unique leverage: DARPA can mandate evaluation standards for defense AI contractors.
One open question is whether AIQ's "science of capability" can produce metrics that are both rigorous enough for researchers and actionable for procurement officers. The program's annual review, scheduled for Q3 2026, will publish initial results from funded teams.
The program's pivot also signals a potential shift in US government AI strategy. [According to Reuters], the Pentagon has been under pressure to develop more reliable AI evaluation methods after several high-profile deployment failures. AIQ's work could directly inform acquisition decisions for AI systems used in national security contexts.
Critics note that DARPA's track record on AI evaluation programs is mixed. The earlier XAI program produced interesting research but limited adoption. AIQ's success may depend on whether it can bridge the gap between academic rigor and operational reality.
What to watch
Watch for AIQ's annual review in Q3 2026, where initial results from funded teams will be published. The key metric is whether the program's new capability measures can predict real-world failure modes better than existing benchmarks.
