TRANSFORMERS & LLMs • 68 / 397
Follow text from characters to tokens, attention, transformers and generation.
Input Tokens
Input tokens are the tokens sent into the model in prompts, history, retrieved context and tool results.
Think of it like
Think of Input Tokens as part of a very fast reader that repeatedly decides what information matters and what should come next.
Real life
Autocomplete, translation and chat assistants depend on concepts such as Input Tokens.
SRE lens
Large input token counts raise cost and often latency.
Remember thisInput tokens are the tokens sent into the model in prompts, history, retrieved context and tool results.