What is a transformer in AI?
A transformer is the neural network design behind modern language models like the ones in ChatGPT, Gemini and Claude. Its key idea, attention, lets the model weigh how every word in a passage relates to every other word, which makes it very good at understanding and generating text.
How does a transformer work?
Text is turned into tokens and then vectors. Stacked transformer layers repeatedly use attention to mix in information from relevant tokens, building up an understanding of context before predicting the next token.
Why does a transformer matter?
Transformers made today's AI boom possible because they learn from huge amounts of text efficiently and scale well with more data and computing power.
An example
Example: In 'The bank by the river flooded', a transformer uses the word 'river' to understand that 'bank' isn't a place to keep money.
Related terms
Learn it properly
A transformer is covered in AI Engineer, with short lessons, practice questions and instant feedback.