Key Takeaways
- Caveman skill for Claude Code saves 8.5% tokens, not 65%.
- Safe to use with no quality loss.
- Install via SkillsBench.
What Changed — The Real Numbers Behind Caveman
JetBrains ran a proper A/B benchmark on the popular "Caveman" skill for Claude Code, and the results are sobering. The skill, which promises to "make agent talk like caveman" and save 65% of output tokens, actually delivers about 8.5% savings on real agentic tasks.
The key insight: agentic output is fundamentally different from chat. Claude Code's responses are dominated by code, diffs, tool invocations, and exact error strings — all of which Caveman correctly leaves verbatim. Only the narration between tool calls gets compressed, and there isn't much of it.
What It Means For You — Safe, Honest, Oversold
The good news: Across 82 paired tasks, JetBrains found zero detectable quality degradation. The skill changes how Claude Code talks without damaging what it produces. Code artifacts stay untouched and normal.
The bad news: The 65% claim is misleading for Claude Code users. That number comes from chat-style prose where narration dominates. In agentic workflows, where most tokens are code and commands, the ceiling is ~10%.
The cost reality: The 8.5% token savings should translate to ~10% cost reduction. However, JetBrains observed a single outlier task where the Caveman arm ballooned past the 200k long-context pricing tier, making it 11.6% more expensive overall. This is a task property, not a skill bug, but it's worth noting.
Try It Now — Should You Install Caveman?
Yes, but with adjusted expectations. Install it via SkillsBench:
# Install Caveman skill
claude code skill install caveman
When to use it:
- When you want terser narration between tool calls (no quality loss)
- When every token counts on long-running tasks
- When you're hitting context limits frequently
When to skip it:
- If you're relying on Claude Code's reasoning narration for debugging
- For short, single-step tasks where 8.5% savings is negligible
- If you're already using a "be brief" style prompt in CLAUDE.md (you're getting similar savings for free)
Pro tip: Add this to your CLAUDE.md for a lighter, zero-dependency alternative:
# Style preferences
Be concise in narration. Focus on code and commands.
The Bottom Line
Caveman is safe, honest about style, and oversold on savings. Install it if you want slightly terser agents. Don't expect the 65% savings advertised. For most Claude Code users, the real value is in the quality guarantee — you can compress narration without breaking output.
Source: blog.jetbrains.com
