TRANSFORMERS & LLMs • 72 / 397
Follow text from characters to tokens, attention, transformers and generation.
Sampling
Sampling is the method used to choose the next token from the model’s probability distribution.
Think of it like
Think of Sampling as part of a very fast reader that repeatedly decides what information matters and what should come next.
Real life
Autocomplete, translation and chat assistants depend on concepts such as Sampling.
SRE lens
Generation settings affect repeatability but do not magically improve model knowledge.
Remember thisSampling is the method used to choose the next token from the model’s probability distribution.