MiniMax Open-Sources MiniMax-01 Series Featuring Lightning Attention for Long Contexts
MiniMax has released and open-sourced the MiniMax-01 series, including MiniMax-Text-01 and MiniMax-VL-01, a foundational language model and a visual multi-modal model respectively. These models introduce a novel 'Lightning Attention' mechanism, an alternative to the traditional Transformer architecture. The 456-billion-parameter model efficiently handles context lengths up to 4 million tokens, significantly larger than other leading models.
The open-sourcing of models with such long context capabilities and a new attention mechanism could accelerate research into more complex and sustained AI agent interactions and memory systems, pushing the boundaries of what large language models can process.
Learn one new AI thing every day.
Daily Deck sends you seven plain-English cards like this every morning. Free.
Start free