MiniMax Open-Sources MiniMax-01 Model with Lightning Attention for Long Context
MiniMax has open-sourced its MiniMax-01 series of models, including the MiniMax-Text-01 language model, which introduces a novel Lightning Attention mechanism as an alternative to the traditional Transformer architecture. This foundational model, with 456 billion parameters, efficiently handles the world's longest context length of up to 4 million tokens, which is 20 to 32 times longer than other leading models. This significant context window is crucial for complex agent systems.
The long context length and open-source nature of MiniMax-01 can inspire new research and applications, accelerating the development of sophisticated AI agents that require extensive contextual understanding.
Learn one new AI thing every day.
Daily Deck sends you seven plain-English cards like this every morning. Free.
Start free