Keeping Autonomous Agents In Bounds: How Swarm Protects Target Infrastructure

Running autonomous penetration testing agents requires pointing an AI directly at infrastructure and allowing it to probe for vulnerabilities. Recent high-profile industry incidents have shown unconstrained AI agents deleting production data and executing instructions planted by third parties. FailSafe Swarm prevents these failure modes through deterministic controls enforced below the model.

The Core Security Principle: The Model Is Untrusted

Swarm is built on one foundational rule: we treat our own AI as an untrusted component. The safety of a pentest run does not depend on model alignment, prompt adherence, or good behavior. Authority is constrained by hard gates in code that the model cannot alter or bypass. If the model makes an error or encounters an adversarial prompt, the instruction cannot escalate into infrastructure damage.

1. Safety Modes Tailored to the Environment

Operators select a Safety Mode prior to execution based on the target environment. This mode establishes the boundary of what actions the agent is permitted to execute:

Capability	Safe	Authorized	Cyber
Read, probe, enumerate	Yes	Yes	Yes
Reversible writes (UPDATE, INSERT, PUT, PATCH)	No	Yes	Yes
Deletes, form submissions, credential brute-force	No	No	Yes
Irreversible destruction (DROP/TRUNCATE TABLE, DELETE FROM, rm -rf)	Never	Never	Never

• Safe (Production targets): The agent reads and probes without altering system state. Flaws proven through read operations arrive confirmed. Findings requiring state mutations are flagged as candidate findings for manual verification.
• Authorized (Staging environments): The agent executes non-destructive state changes to validate proof-of-concepts while preventing destructive deletions.
• Cyber (Isolated test environments / replicas): The agent executes complete exploit chains (privilege escalation, record modification, account takeover) to eliminate false positives in an isolated replica.

Regardless of the active mode, irreversible operations such as dropping tables or clearing root directories are blocked in all circumstances.

2. Five Layers of Defense

Safety does not rely on any single defensive check. An errant or manipulated agent must bypass five distinct control layers to cause unintended side effects:

1. SCREEN (Prompt-Injection Detection): Target content retrieved by the agent is scanned for adversarial instructions before the agent processes it. Injected instructions attempting to hijack execution flow are stripped.
2. PREVENT (Deterministic Command Gate): Every shell command emitted by the agent is validated by an independent code gate prior to execution. The gate rejects destructive operations, flood tools, aggressive scanner flags, and concurrency violations.
3. CONFINE (Operator-Defined Scope): Execution is restricted to explicit host allowlists and path boundaries. Out-of-scope network calls fail closed and are blocked immediately at runtime.
4. CONTAIN (Disposable, Credential-less Sandbox): Each engagement runs within an ephemeral container with zero standing access to target infrastructure. Target networks are reached externally using only explicitly provisioned test credentials.
5. DETECT (Immutable Audit Trail): Every executed command and blocked attempt is recorded in sequence for post-run review and verification.

3. Verifiable Artifacts

Every run produces verifiable output artifacts that security teams can audit directly:

• The Command Log: The ordered record of every network request and shell command dispatched during the engagement.
• The Block Log: The explicit refusal record for commands rejected by the runtime gate. Examples from production runs include:

  COMMAND REFUSED: out-of-scope host crt.sh (Restricted to scope allowlist)
  COMMAND REFUSED: SQL DELETE FROM (Data loss prevention, blocked in all modes)
  COMMAND REFUSED: HTTP DELETE (Resource deletion blocked unless explicitly permitted)
  COMMAND REFUSED: FORM SUBMISSION (External message generation blocked)
  COMMAND REFUSED: High-volume HTTP (Capped at 5 concurrent requests)

• Graded Restraint: When proving a vulnerability requires high volume (such as brute-forcing authentication), the agent demonstrates that the weakness exists and halts before generating disruptive traffic.