DocsAgents & automations
Agent CI
Once a shared team agent is doing real work, you want to know before you change it whether the change made things worse - not after a member notices. Agent CI is a lightweight test suite for a shared agent: a set of example tasks, what each one should produce, and a green/red readout every time you run it.
Eval sets
An eval set is a small collection of cases attached to one shared agent. Each case has an input (a task, the same as you'd hand the agent directly) and, optionally, what to expect back:
- Contains / doesn't contain - substrings the reply must or must not include.
- Exact match - the reply must equal a given text exactly.
- Pattern match - the reply must match a given pattern.
- Judged - a short description of what a good reply looks like ("apologizes and offers a concrete next step"), graded by a second, independent model rather than a literal string check.
- Cost and time budgets - the case fails if the run cost more or took longer than you allow.
A case with no expectations at all still catches the basics: did the agent run without error, and how much did it cost.
Run a set from Settings > Governance > Evals, or leave "Run nightly" turned on so it checks itself automatically, roughly once a day, without anyone needing to remember. Each run shows a pass/fail count per case, and a diff view highlights exactly which cases changed since the previous run - useful right after a change to confirm nothing regressed.
Release gates
A release gate ties a shared agent (or, for an organization-owned company, one department's autonomy level) to an eval set: turn it on, and that agent can't be promoted to live until the linked set's most recent run is fully green. This catches the easy mistake of shipping a change that quietly broke something the old version handled fine - the promotion is refused, with the reason, until the suite passes again.
Gates apply where an agent actually goes live: promoting a canary version to current, and (for a company department) turning autonomy up to "autonomous" for real, unattended actions.
Automatic promotion
Combine a gate with a canary: start a canary of a new version, link it to an eval set, and turn on auto-promote. Once a run finishes testing that exact canary version and it's at least as good as the currently-live version - and, if a gate is on, it's fully green - the canary is promoted automatically. No gate, no suite, no surprise promotion: this only ever happens when you've explicitly wired it up.
Regressions
If a suite that was recently green comes back red, your team gets notified - the same way any other governance event does - so a quiet regression doesn't sit unnoticed until a member complains.