A new survey of 239 papers reveals two distinct paths for AI agent self-improvement: model updates and scaffold updates. Scaffold updates — modifying prompts, memory, or tools — account for 68% of recent work, per @HuggingPapers.
Key facts
- Survey covers 239 papers on AI agent self-improvement.
- 68% of papers focus on scaffold updates (prompts, memory, tools).
- Model updates (fine-tuning, RLHF) account for 32% of approaches.
- Only 12% evaluate on held-out benchmarks.
- Scaffold updates yield faster iteration and lower compute costs.
A survey of 239 papers on how AI agents self-improve — by updating the model itself or the scaffold (prompts, memory, tools) — has been released by @HuggingPapers According to @HuggingPapers. The analysis categorizes self-improvement methods into two broad families: model updates and scaffold updates.
Scaffold Updates Dominate
Scaffold updates dominate recent work, with 68% of papers focusing on prompt and tool modifications rather than model retraining. These include iterative prompt engineering, dynamic memory retrieval, and tool-use refinement loops. The survey finds that scaffold-based improvements yield faster iteration cycles and lower compute costs, making them attractive for production deployments where model weights are frozen.
Model Updates Persist
Model updates, including fine-tuning and RLHF, account for only 32% of surveyed approaches. These methods modify the model's weights but require substantial compute and risk catastrophic forgetting. The survey notes a trend toward smaller, targeted fine-tuning runs rather than full retraining.
Benchmarking Gaps
Only 12% of papers evaluate self-improvement on held-out benchmarks, raising questions about overfitting. Most evaluations use in-distribution tasks or synthetic test sets, limiting generalizability claims. The survey calls for standardized evaluation protocols across the field.
The unique take: the field is quietly pivoting from model-centric improvement (fine-tuning, RLHF) to scaffold-centric improvement (prompts, memory, tools). This mirrors the broader industry shift toward agentic systems where the model is a fixed compute engine and the intelligence lives in the orchestration layer. The survey's 68-32 split suggests the scaffold approach is winning in practice, even as the research spotlight remains on model updates.
What to watch
Watch for a follow-up survey in late 2026 tracking whether held-out benchmark adoption crosses 30%, and whether scaffold updates maintain their dominance as model fine-tuning costs drop with new techniques like LoRA variants.
