TRANSFORMERS & LLMs • 42 / 397
Follow text from characters to tokens, attention, transformers and generation.
Embedding
An embedding is a numerical vector that represents information such as the meaning of a token, sentence or document.
Think of it like
Think of Embedding as part of a very fast reader that repeatedly decides what information matters and what should come next.
Real life
Autocomplete, translation and chat assistants depend on concepts such as Embedding.
SRE lens
Embedding vectors let an incident search find “RAM exhausted” when the query says “out of memory”.
Remember thisAn embedding is a numerical vector that represents information such as the meaning of a token, sentence or document.