EVALS & OBSERVABILITY • 248 / 397
Measure whether the AI is good and trace where it fails.
A/B Test
An A/B test sends different users or requests to different variants and compares outcomes.
Think of it like
Think of A/B Test like testing and telemetry for any production service: quality must be measured, not guessed.
Real life
Teams use A/B Test to decide whether a new release is safer, faster or more useful.
SRE lens
Compare a new prompt or model routing policy against the current one.
Remember thisAn A/B test sends different users or requests to different variants and compares outcomes.