Transformer

Models and Architectures

The neural-network architecture behind modern AI language models, introduced in 2017. The T in GPT stands for Transformer.

The transformer is a neural-network design that can weigh how every word in a text relates to every other word at once, using a mechanism called attention. This lets it grasp context far better than earlier designs.Nearly every headline AI system since 2018 - GPT, Gemini, Claude, Llama, and many image and audio models too - is built on transformers.