one year on
Mistral AI releases open-source Mistral 7B model, outperforms larger Llama 2
The four-month-old French startup claims its 7.3B parameter model beats Meta's Llama 2 13B across all benchmarks, released under a permissive Apache 2.0 license.
Mistral AI, a Paris-based startup founded just four months ago, today released Mistral 7B, a 7.3 billion parameter open-source language model that it claims outperforms Meta’s Llama 2 13B on every standard benchmark. The model is available under the Apache 2.0 license on GitHub.
The model is released under the Apache 2.0 license, allowing unrestricted use for both research and commercial purposes. Mistral 7B uses Grouped-query attention for faster inference and a sliding window attention mechanism that the company says reduces compute cost while handling longer sequences. On code benchmarks, the model approaches CodeLlama 7B’s performance while remaining good at English tasks.
Mistral AI raised a €105 million funding round four weeks after launching in May at a €240 million valuation, backed by Lightspeed Venture Partners, Index Ventures, Redpoint Ventures and more than a half dozen other investors. The company has now delivered its first model months ahead of its original 2024 timeline. The company says the model took three months to develop, and that in that time its founders assembled an engineering team and built an MLOps stack. The release suggests the development effort is advancing faster than expected.
The record
Reported that Mistral AI's release comes faster than expected: the company had planned to introduce its first models in 2024.
One year later — open only if you can handle spoilers
Mistral 7B quickly became a foundational model for fine-tuning and deployment across the open-source ecosystem. The company's unorthodox release strategy and permissive license set a precedent for future model releases, cementing Mistral's reputation as Europe's leading open-weight AI lab.