Restarting infrastructure blindly may hide the actual bottleneck or make queues worse.
INCIDENT RESPONSEAI SRE
Incident: TTFT Suddenly Spikes
A TTFT incident is usually investigated through queueing, prompt growth, prefill cost, model startup and provider latency.
TTFT moves from 900ms to 7s while TPOT stays stable; queue depth and prompt-token p95 both rise.
Which evidence tells you whether the issue is queueing, larger prompts or provider slowdown?
REMEMBERSeparate 'waiting to start' from 'slow once started.'
No uploads · No company data · No account required