Skip to content
anythingai.space
Deployment & Deliveryintermediate12 min · 9 steps

Eval Gates in CI

Turn eval suites into required checkpoints that block a change — code, prompt, model or index — from advancing when quality, safety, cost or latency slip.

tool-call acc Δ-4 pts
cost/turn Δ-22%
users exposed100%
merged anywaybypassChangecode · prompt · modelEval reporttool-call acc −4 ptsadvisoryModel upgradeno PR, no evalEveryonefailed ticket updates
requesterrorresponsedatasignal
metrics are illustrative
1 / 9
ProblemStep 1 of 9

An eval nobody has to pass is just advice

The team has a good golden set. It runs in CI and posts a score… and the PR merges anyway, because the check isn't required and the deadline is today.

And the riskiest changes don't even trigger it: a model upgrade, a re-indexed knowledge base or a new tool never opens a pull request against the code.

Watch outA suite nobody must pass is advice. A required status check is a control.

Lesson 6 of 20 in Deployment & Delivery