← Glossary · Architecture

transformer

Concept

Fact-checked May 20, 2026

Also called: Transformers

A neural network architecture, or blueprint, especially good at understanding sequences of data like text or speech. It processes all parts of a sequence at once, rather than one piece at a time.

The Transformer is a neural network architecture introduced in a 2017 paper by Google researchers titled 'Attention Is All You Need.' Before the Transformer, many AI models processed sequences of data, like words in a sentence, one after another. This made it difficult for them to connect parts of a sequence that were far apart. The Transformer changed this by introducing a mechanism called 'self-attention.'

Self-attention allows the model to weigh the importance of different parts of the input sequence when processing each element. For example, when reading a sentence, it can decide to pay more attention to specific words that are key to understanding the meaning of another word. This ability to assess context across an entire sequence simultaneously, rather than step-by-step, made Transformers incredibly powerful for tasks like language translation and text generation.

Transformers laid the foundation for many of today's most advanced large language models (LLMs), including popular ones like OpenAI's GPT series and Google's BERT and T5 models. Its design fundamentally changed how AI approaches sequence-to-sequence problems, leading to significant breakthroughs in natural language processing and other fields.

Related terms

attentionAttention is a powerful mechanism in AI, especially in models like Transformers, that helps the model focus on the most important parts of its input.CNNCNN stands for Convolutional Neural Network, a type of neural network especially good at processing images and other grid-like data.diffusion modelDiffusion models are a type of generative AI model that learn to create new data, like images or audio, by progressively removing noise from an initial random input.GANGAN stands for Generative Adversarial Network, a type of AI framework that learns to create new, realistic data like images, audio, or text.Graph Neural NetworkGraph Neural Networks, or GNNs, are a type of neural network designed to work directly with data structured as graphs, like social networks or molecular structures.GRUGRU stands for Gated Recurrent Unit.

Learn AI in 5 minutes a day.

Daily Deck explains terms like transformer as part of a free seven-card daily brief. No jargon. No fluff.

Start free