TRANSFORMERS & LLMs • 51 / 397
Follow text from characters to tokens, attention, transformers and generation.
Encoder
An encoder transforms an input sequence into contextual representations.
Think of it like
Think of Encoder as part of a very fast reader that repeatedly decides what information matters and what should come next.
Real life
Autocomplete, translation and chat assistants depend on concepts such as Encoder.
SRE lens
Encoder-style models are often useful for classification or understanding tasks.
Remember thisAn encoder transforms an input sequence into contextual representations.