Your agent monitor raised an alert. Did the workflow actually stop?

Sources checked: 2 November 2026. FailSafe Research.

There are two different results in a monitoring exercise: the system recognized concerning behavior, and the system prevented further consequential actions. OpenAI's October monitoring publication is useful context for that distinction. Our proposed enterprise evaluation makes the second result directly observable.

Define the consequence before the alert

Choose an authorized, synthetic workflow with a clear prohibited outcome. For example, a test agent may read one set of harmless records while a separate policy prevents a change to a protected test object. Avoid using actual financial transfers or real personal information as the finish line.

Document which component detects the behavior and which component can remove authority. They may have different owners, failure modes and escalation paths. A monitoring service that sends notifications should not be scored as an automatic enforcement service unless that function is actually configured.

Keep a timeline with separate clocks

Record the first relevant action, monitor observation, alert delivery, acknowledgement, intervention request and confirmed loss of authority. Align timestamps before comparing durations. Note clock uncertainty where an integration does not provide sufficient precision.

Inspect the interval between alert and containment. Did a queued action finish? Did another session continue? Did a delegated worker retain access? The answer may be operationally more useful than a single detection percentage.

Exercise the degraded path

Repeat the permitted scenario when the notification channel is delayed or the intended responder is unavailable, using a plan approved by operations. The objective is to understand the fallback, not to disable real monitoring during an uncontrolled test.

A result should say whether the task stopped, paused safely, continued with less authority or continued unchanged. Preserve incomplete and ambiguous results. Excluding every integration failure would create a misleading picture of the deployed system.

Publish the method and its limits

Use the same scenario definitions and action budgets when comparing configurations. Disclose which interfaces were visible to the assessor and which actions were simulated. An evaluation of one model, harness and response path does not establish a universal safety property.

FailSafe can scope adversarial testing of an AI application around observable controls and evidence. Use the acceptance criteria to define what your own assessment must observe. Existing AttackBench research should be read under its own published method.

Measure the decision points

1. Observed: the monitor has the relevant event.
2. Escalated: an accountable responder receives it.
3. Restricted: the target rejects a subsequent unauthorized action.
4. Recovered: legitimate work resumes under an approved decision.

Source review, 2 November 2026: OpenAI: how frontier model misalignment is detected. The evaluation method above is FailSafe editorial analysis; no new measured performance result is claimed.