RELIABILITY & SRE • 328 / 397
Apply familiar SRE thinking to AI dependencies and workflows.
Thundering Herd
A thundering herd is a burst of many clients doing the same work at once, often after recovery or cache expiry.
Think of it like
Think of Thundering Herd exactly as you would in production infrastructure: assume components fail and design the user experience around controlled failure.
Real life
Banks, airlines and large online services use Thundering Herd to keep critical systems dependable.
SRE lens
Stagger retries and cache refreshes after a model-provider recovery.
Remember thisA thundering herd is a burst of many clients doing the same work at once, often after recovery or cache expiry.