TRANSFORMERS & LLMs • 73 / 397
Follow text from characters to tokens, attention, transformers and generation.
Top-p
Top-p limits token selection to a probability mass containing the most likely candidates before sampling.
Think of it like
Think of Top-p as part of a very fast reader that repeatedly decides what information matters and what should come next.
Real life
Autocomplete, translation and chat assistants depend on concepts such as Top-p.
SRE lens
It is another decoding control alongside temperature.
Remember thisTop-p limits token selection to a probability mass containing the most likely candidates before sampling.