TRANSFORMERS & LLMs • 56 / 397
Follow text from characters to tokens, attention, transformers and generation.
Autoregressive Generation
Autoregressive generation produces one token, appends it to context, then predicts the next token repeatedly.
Think of it like
Think of Autoregressive Generation as part of a very fast reader that repeatedly decides what information matters and what should come next.
Real life
Autocomplete, translation and chat assistants depend on concepts such as Autoregressive Generation.
SRE lens
Long answers take time because generation happens token by token.
Remember thisAutoregressive generation produces one token, appends it to context, then predicts the next token repeatedly.