one year on
Meta Releases Llama 3.1 405B, Claims It Matches Leading Closed Models
The first open-weights model to credibly rival GPT-4o and Claude 3.5 Sonnet on benchmarks, plus 128K-context updates to 8B and 70B models and a revised license allowing synthetic data distillation.
Meta today released Llama 3.1 405B, a 405-billion-parameter model it says is the first open-weights AI model to match the capabilities of leading proprietary systems like OpenAI’s GPT-4o and Anthropic’s Claude 3.5 Sonnet. Trained on over 15 trillion tokens using 16,000 Nvidia H100 GPUs, the model is text-only but achieves competitive results across coding, math, multilingual translation, and general knowledge benchmarks.
Alongside the flagship, Meta refreshed its 8B and 70B models with 128,000-token context windows — a 16x increase over the prior Llama 3 generation — and added support for eight languages and tool use. The company also updated its license to explicitly allow developers to use outputs from Llama models to train other models, which Meta says helps with synthetic data generation and model distillation, addressing a major criticism from the open-source community. However, the license still restricts developers with over 700 million monthly active users, who must seek special permission.
In a simultaneous open letter, CEO Mark Zuckerberg argues that open source is good for developers, Meta and the world, and frames the release as part of Meta’s open-access push. The release comes with a reference system, safety tools (Llama Guard 3, Prompt Guard), and a call for comment on a proposed Llama Stack API. More than 25 launch partners — including AWS, Nvidia, Databricks, Groq, and Google Cloud — announced support on day one.
The move intensifies the open-vs-closed AI debate. Critics note that the model still exhibits hallucinations, requires a server node for inference, and trails Claude 3.5 Sonnet on some reasoning tasks. But for many developers, the offer of a frontier-level model they can download, fine-tune, and deploy without sharing data with Meta is the strongest case yet for open-source AI’s viability.
The record
One year later — open only if you can handle spoilers
Llama 3.1 405B became a go-to base model for the open-source community, but its 16,000-GPU requirement limited widespread local use; it was eventually spun off into smaller distilled versions that achieved broader adoption. The debate over the license's user cap did not trigger major forks, but several startup lobbied Meta to relax the threshold.