SWE-Pruner Pro achieves up to 39% token savings on coder LLMs. The method reads keep-or-prune signals directly from the model's own hidden states, eliminating external classifiers.
Key facts
- Up to 39% token savings on coder LLMs.
- Reads signals from model's own hidden states.
- Eliminates external classifiers or heuristics.
- Accuracy drop within 1–2 percentage points.
- Generalizes across 1B to 7B parameter models.
SWE-Pruner Pro, announced via @HuggingPapers, introduces a token pruning technique for coder LLMs that exploits information already computed during the forward pass. By reading keep-or-prune signals from the model's own hidden states, it achieves up to 39% token reduction while preserving task quality. This eliminates the need for external classifiers or heuristic rules, which often introduce latency and degrade performance.
How hidden-state pruning works
Standard pruning methods rely on auxiliary models or hand-crafted heuristics to decide which tokens to drop. SWE-Pruner Pro instead queries the LLM's internal representations at a given layer, extracting a binary keep-or-prune decision per token. Because these signals are computed as part of the existing forward pass, the overhead is minimal. The authors report that the pruning decisions align closely with attention patterns, suggesting the model already encodes relevance information internally.
Benchmarks and savings
The paper reports token savings of 28–39% across multiple coding benchmarks, including HumanEval and MBPP. Accuracy drops are within 1–2 percentage points, and in some cases pruning improves performance by removing noisy tokens. The method generalizes across model sizes from 1B to 7B parameters, though the exact architectures tested were not specified in the source.
Unique take: inference-time compression as a byproduct of better architecture
The broader implication is that many LLMs already possess the internal structure needed for efficient inference — we just haven't learned to read it. SWE-Pruner Pro is part of a growing trend (see recent work on early-exit models and speculative decoding) that treats hidden states as a first-class resource rather than a discardable intermediate. If this pattern holds, future models may be trained with explicit pruning heads, making inference cheaper without sacrificing quality.
What to watch
Watch for the release of the SWE-Pruner Pro code and checkpoints, which the authors have not yet made public. If the method reproduces across non-coding domains (e.g., general-purpose chat models), it could become a standard inference-time optimization.
