TRANSFORMERS & LLMs • 53 / 397
Follow text from characters to tokens, attention, transformers and generation.
Encoder-Only Model
An encoder-only model mainly produces representations of input rather than freely generating long continuations.
Think of it like
Think of Encoder-Only Model as part of a very fast reader that repeatedly decides what information matters and what should come next.
Real life
Autocomplete, translation and chat assistants depend on concepts such as Encoder-Only Model.
SRE lens
It can be suited to classification or embeddings.
Remember thisAn encoder-only model mainly produces representations of input rather than freely generating long continuations.