eval-kit

The EU AI Act's Article 14 requires human oversight of high-risk systems to be effective, and explicitly names automation bias as something deployers must counter. It does not say how effectiveness is measured, because nothing measures it.

That gap widens as models improve, which is the counterintuitive part. Bainbridge's "Ironies of Automation" (1983) is the canonical statement: the more reliable the automation, the less practiced the human operator, and the worse they perform in exactly the rare cases where they are the last line of defence. The automation-bias literature that follows: Parasuraman and Riley, Skitka, Parasuraman and Manzey, shows vigilance decays in proportion to observed accuracy.

So the better your agent gets, the more the residual error concentrates in cases a fatigued reviewer waves through, and the less any aggregate accuracy number tells you about it. The value of a gate is inversely proportional to how often it fires: one firing on 30% of cases is a bottleneck people route around; one firing on 0.5% is where all the risk lives, where measurement is hardest, and where human skill has most decayed.

Mandated gates are policy: approval must precede this action. Compliance is binary and ordering-sensitive: confidence is irrelevant, and a 94% compliance rate is not a good score, it is 6% unauthorised actions. The schema records which gates a step triggered, which were honoured, and which were violated.

Discretionary gates are judgment: should the agent have asked here? That is a precision/recall problem, because asking about everything is as much a failure as asking about nothing, and the two error directions carry different costs.

They roll up as three separate numbers: mandated compliance, ask precision, blocker recall, and never into one. Two asymmetries sit underneath, and they resolve differently on purpose. If gate ordering was not captured in the trace, the scorer refuses to score at all, because an instrument that cannot see must not report success. But an approval that does not name what it authorises scores as a violation rather than an error, because that trace can be read, and the honest reading is that nothing was authorised.

Ordering is the compliance claim, so ordering is drawn rather than summarised. The review surface interleaves the agent's tool calls with its gate events in trace order, and marks an unauthorised call at the row where it happens, naming the gate that covers it.

The two demo runs shipped in the repo make the point without commentary: their task tools, tool-match scores and final outputs are byte-identical, and they differ only in whether authorisation happened. Six gated calls authorised versus six unauthorised. A benchmark scores those two runs the same.

Where a suite declares no gates, the interface says so, "authorization was not assessed", rather than rendering a blank. Absence of measurement is stated, never left to look like a clean result.

Anywhere a decision creates an obligation or a record that outlives it: content moderation, customer support, public-sector determinations, and scientific or academic review. The gate is the unit; the domain is an instance.

Known limits are stated up front rather than left implicit. Rare events need large denominators, at a 1% error rate, a hundred error cases means reviewing ten thousand decisions, so you sample at the gate, where escalated cases are already enriched for error. And errors are not randomly distributed: they concentrate in a dialect, a demographic, a document format, so uniform sampling will miss a subgroup while the aggregate looks excellent. Stratified audit is a requirement, not a nice-to-have.