TRANSFORMERS & LLMs • 62 / 397
Follow text from characters to tokens, attention, transformers and generation.
Inference
Inference is running a trained model to produce a prediction or response.
Think of it like
Think of Inference as part of a very fast reader that repeatedly decides what information matters and what should come next.
Real life
Autocomplete, translation and chat assistants depend on concepts such as Inference.
SRE lens
Every production LLM API request triggers inference and consumes compute.
Remember thisInference is running a trained model to produce a prediction or response.