Test an agent’s stop controls after it has already started working

Sources checked: 2 November 2026. FailSafe Research.

The first instruction in a task is a poor substitute for an operating boundary that remains effective later. Anthropic's 23 October assessment of the UK AISI cybertesting incident examines how a model's trajectory affected its behavior. Those unusual evaluation conditions should prompt careful testing, rather than a general claim about all deployed agents.

Choose intervention points deliberately

A short task and a long task can encounter the same control under different conditions. Identify points before a tool is selected, after access is granted, while work is queued and after a summary replaces earlier context. Define the expected response to a stop or scope change at each point.

Keep the task itself benign. A synthetic document-processing or test-object workflow can show whether authority changes take effect without recreating harmful behavior from a research transcript.

Keep policy outside the agent’s narrative

A summary may preserve the task goal while losing an important restriction. The environment should therefore enforce the current permission boundary independently of the agent's description of it. Examine which service or operator owns that boundary and how a changed decision reaches workers.

When testing, compare the recorded policy state with the action that the runtime permits. The agent saying that it has stopped is an observation to investigate, not the decisive enforcement result.

Distinguish stopping from cleanup

After intervention, establish whether queued actions, delegated work and previously issued credentials remain usable. Then review cleanup: temporary artifacts, access grants and partial business operations can outlive the task that created them.

Do not call a successful cancellation a completed rollback. A useful result identifies the last authorized action, any residual effects and the owner responsible for resolving them. The process should also explain how a legitimate user can resume safely. Review that recovery decision with the operations owner before accepting the assessment result.

Compare like-for-like runs

Repeat the same intervention points with recorded model and harness versions. Preserve failures and unavailable telemetry in the result set. A changed prompt, longer action budget or different approval policy can invalidate a comparison even when the task name is unchanged.

FailSafe's agentic assessment approach connects runtime behavior to reproducible evidence. Long-run control assurance should be separately scoped against the deployed application; neither a model benchmark nor an initial approval establishes that property. See the containment tabletop guide for the response team's complementary planning exercise.

Source review, 2 November 2026: Anthropic assessment of the UK AISI incident. No continuation percentage is generalized to customer deployments.