Transformer
6 items tagged with "transformer"
FAQs2
What is a large language model (LLM)?
A large language model is a neural network trained on vast amounts of text to predict the next token in a sequence, which lets it generate and underst...
What is a transformer architecture?
The transformer is a neural network architecture, introduced in 2017, that processes sequences using a self-attention mechanism instead of recurrence....
Glossaries4
Large Language Model (LLM)
A large language model is a neural network trained on vast text corpora to predict the next token, enabling it to generate and understand natural language across many tasks.
Transformer
A transformer is a neural network architecture built around self-attention that processes sequences in parallel, forming the basis of modern large language models.
Attention Mechanism
An attention mechanism lets a model weigh the relevance of different parts of its input when producing each output, focusing on the most pertinent tokens.
Foundation Model
A foundation model is a large model pretrained on broad data at scale that can be adapted, through fine-tuning or prompting, to a wide range of downstream tasks.