Techment — Site Header

What is Transformer (Model)

Definition

A transformer is a neural-network architecture that processes sequences using a self-attention mechanism to weigh the relevance of every element to every other, enabling efficient parallel training and powering most modern language and multi-modal models.

« Back to Glossary Index

Captures long-range relationships in sequences via self-attention
Trains efficiently in parallel, unlike older sequential architectures
Scales effectively to very large models and datasets
Underpins the breakthrough performance of modern LLMs and multi-modal AI

Real World Example

A translation system built on a transformer attends to the whole source sentence at once, correctly handling a word whose meaning depends on context far earlier in the sentence, which older sequential models often mishandled.

FAQs

What makes the transformer architecture special?

Its self-attention mechanism lets it weigh relationships between all elements of a sequence and train efficiently in parallel.

Why are transformers so widely used?

They scale well and handle long-range context, making them the foundation of modern LLMs and many vision and multi-modal models.

What is self-attention?

A mechanism that computes how much each element of a sequence should attend to every other element when forming its representation.