Mistral's Leanstral 1.5 Achieves 100% on Formal Math Benchmarks — Open-Source Breakthrough

Mistral AI has taken another significant step in the artificial intelligence landscape with the release of Leanstral 1.5, an open-source model purpose-built for formal verification. This is not just another language model — it is a tool capable of proving mathematical theorems and catching software bugs with a level of rigor that no open-source model has achieved before.

What Is Leanstral 1.5?

Leanstral 1.5 is an open-source model developed by Mistral AI, released under Apache 2.0 license. The model is specifically trained to work with the Lean 4 programming language, making it a powerful instrument for formal mathematical proofs and code verification.

miniF2F: A Perfect 100% Score

One of Leanstral 1.5's most impressive achievements is its perfect 100% score on the miniF2F benchmark. miniF2F covers problems ranging from high school level up to mathematical olympiad difficulty. Scoring 100% means the model solved every single problem correctly with formal proofs in Lean 4.

PutnamBench: 587 Out of 672 Problems

PutnamBench collects 672 problems from the prestigious Putnam competition. Leanstral 1.5 successfully solved 587 of these problems — approximately 87%, leading all open-source models.

Algebra Benchmarks: FATE-H and FATE-X

On FATE-H, Leanstral 1.5 achieved 87%, and on FATE-X, 34% — top scores among all open-source models.

Understanding Lean 4

Lean 4 is a functional programming language and theorem prover developed by Microsoft Research for formal verification of mathematical proofs and software correctness.

Catching Real Bugs in Production Code

Leanstral 1.5 scanned 57 open-source repositories and discovered five previously unknown bugs, including an overflow bug in a Rust library.

Availability

The model is available on Hugging Face under Apache 2.0 license with a free API for testing.