LLMOPS & AI INFRASTRUCTURE • 268 / 397
Operate AI like production: serving, GPUs, latency, capacity, routing and reliability.
VRAM
VRAM is memory attached to a GPU and stores model weights, activations and caches during inference.
Think of it like
Think of VRAM like familiar infrastructure capacity and traffic engineering, except the scarce resources are often tokens, GPU memory and model latency.
Real life
High-scale AI services depend on VRAM to keep cost and latency under control.
SRE lens
GPU OOM often means VRAM, not system RAM, is exhausted.
Remember thisVRAM is memory attached to a GPU and stores model weights, activations and caches during inference.