Stop Calling It an Eval: Why Agent Quality Must Live in CI

This article challenges a common shortcut in AI teams: evaluating agent behavior manually instead of gating it in CI. The critical insight is that manual evaluations degrade precisely when they're needed most—when prompts change, model checkpoints roll, or dependencies shift—making them unreliable as quality safeguards. The solution is treating agent behavior like any other production concern: automated, trace-enabled CI checks that actually block bad deployments.