What is a transformer architecture?

Answer

The transformer is a neural network architecture, introduced in 2017, that processes sequences using a self-attention mechanism instead of recurrence. Self-attention lets each token weigh the relevance of every other token in the input, capturing long-range relationships in parallel rather than step by step. Transformers scale efficiently on modern hardware and underpin nearly all current large language models and many vision and multimodal models.