MiniMax Open-Sources MiniMax-01 Models with Lightning Attention
MiniMax has released and open-sourced its MiniMax-01 series of models, including the foundational language model MiniMax-Text-01 and the visual multi-modal model MiniMax-VL-01. These models utilize a novel Lightning Attention mechanism, offering an alternative to the traditional Transformer architecture. MiniMax-01 boasts 456 billion parameters and can handle a context length of up to 4 million tokens, significantly more than other leading models.
The open-sourcing of such a large-scale model with a novel attention mechanism could drive significant advancements in long-context understanding and AI agent development. It provides developers with powerful tools for research and application building.
Learn one new AI thing every day.
Daily Deck sends you seven plain-English cards like this every morning. Free.
Start free