LLMOPS & AI INFRASTRUCTURE • 300 / 397
Operate AI like production: serving, GPUs, latency, capacity, routing and reliability.
Cold Start
A cold start is the delay when required model or service state is not already loaded and ready.
Think of it like
Think of Cold Start like familiar infrastructure capacity and traffic engineering, except the scarce resources are often tokens, GPU memory and model latency.
Real life
High-scale AI services depend on Cold Start to keep cost and latency under control.
SRE lens
Loading large weights into GPU memory can take much longer than starting a normal web process.
Remember thisA cold start is the delay when required model or service state is not already loaded and ready.