mistral-ai" class="entity-chip">Mistral AI released Leanstral 1.5, an open-source model that scores 100% on the miniF2F formal math benchmark. The model also found five previously unknown bugs in 57 open-source repositories.
Key facts
- 100% on miniF2F formal math benchmark
- Solves 587 of 672 Putnam problems
- Found 5 bugs in 57 open-source repos
- Apache 2.0 license, available on Hugging Face
- Trained with mid-training, SFT, and RL
Mistral AI released Leanstral 1.5, an open-source model (Apache 2.0) built for formal verification in the Lean 4 programming language. Lean 4 is designed to formally verify mathematical proofs and software correctness.
According to The Decoder, Mistral says the model hits 100 percent on miniF2F, a formal math benchmark covering problems from high school level up to math olympiad difficulty. On PutnamBench, which includes 672 problems from the Putnam math competition, it solves 587. On the algebra benchmarks FATE-H and FATE-X, which test master's and doctoral-level tasks in areas like group theory and ring theory, it scores top results of 87 and 34 percent.
The model was trained mainly for math, but Mistral says it also performs well at code verification. In a hands-on test, it scanned 57 open-source repositories and caught five previously unknown bugs, including an overflow bug in the Rust library varinteger. The model is available through Hugging Face and a free API. Training involved mid-training, supervised fine-tuning, and reinforcement learning.
One unique take: Formal verification as a practical bug-finding tool
Most formal verification models focus on math proofs, but Leanstral 1.5's discovery of real bugs in production Rust code shifts the narrative. The model caught an overflow bug in varinteger, a Rust library — not a toy example. This suggests that Lean 4 models, previously confined to academic math problems, can serve as practical software verification tools. Mistral's open-source release under Apache 2.0 lowers the barrier for developers to integrate formal verification into CI/CD pipelines, potentially reducing reliance on traditional fuzzing or manual audits.
What to watch
Watch for Mistral's next release — likely a larger variant of Leanstral or integration into their API. Also track whether open-source adoption of Lean 4 verification in CI/CD pipelines increases, with the first public case study from a major Rust project.
Source: the-decoder.com
