MiniMax Open-Sources 456 Billion Parameter Model with Lightning Attention
MiniMax has open-sourced its new MiniMax-01 series, featuring the foundational language model MiniMax-Text-01 and the visual multi-modal model MiniMax-VL-01. These models incorporate a novel Lightning Attention mechanism, offering an alternative to the traditional Transformer architecture. Possessing 456 billion parameters, with 45.9 billion activated per inference, they achieve a context length of up to 4 million tokens, significantly exceeding other leading models.
This release marks the first large-scale commercial-grade model primarily relying on linear attention mechanisms, potentially setting new standards for long-context understanding and AI Agent development.
Learn one new AI thing every day.
Daily Deck sends you seven plain-English cards like this every morning. Free.
Start free