MiniMax-01 Leverages Lightning Attention for Unprecedented Context Lengths
MiniMax has open-sourced its MiniMax-01 series, which includes the foundational language model MiniMax-Text-01 and the multimodal MiniMax-VL-01. These models introduce a novel 'Lightning Attention' mechanism that offers a new alternative to the traditional Transformer architecture. With 456 billion parameters, MiniMax-01 efficiently handles the world's longest context length of up to 4 million tokens, which is 20 to 32 times more than other leading models.
This breakthrough in context length allows AI agents to maintain a much deeper understanding of ongoing conversations and complex tasks, which is crucial for advanced AI Agent systems. It could accelerate the development of more capable and autonomous AI.
Learn one new AI thing every day.
Daily Deck sends you seven plain-English cards like this every morning. Free.
Start free