AI Code Review Agents: Separate Untrusted PR Content from CI Authority

Sources checked: 29 October 2026. This article contains FailSafe's analysis and proposed validation workflow; linked sources support the reported facts.

An AI reviewer may read code from an untrusted pull request while running inside a CI environment with trusted permissions. The security boundary is the relationship between that content and the authority of the review job. Consider a bot that summarizes a contributor’s change and posts a review comment. It may not need repository write access or deployment secrets at all. This assessment asks whether the configured workflow grants only the access required for review and whether proposed edits require an independent gate.

Define the boundary before testing

Inventory event triggers, trusted and untrusted branches, job permissions, secrets, tool access, build artifacts, and the identity used for comments. Identify whether the agent can edit files, execute repository scripts, install dependencies, change workflow configuration, or push a commit. Use a disposable repository with harmless fixtures and no production secrets. Define the expected behavior for external contributors and internal branches separately. A repository README or PR description can be useful evidence without becoming an instruction to grant new authority.

Run a bounded assessment

Run a normal test pull request and observe the bot’s full job permissions and artifacts. Then place an approved harmless instruction fixture in reviewable content, asking for an out-of-scope mock operation. Inspect whether the bot proposes or executes the operation and whether CI permissions independently prevent it. Include a generated build log or test result if the reviewer reads those channels. Do not run untrusted code against production credentials. Use inert substitutes for deployment, external messaging, and privileged repository actions.

Keep evidence an engineer can use

Keep the workflow revision, triggering event, token permission summary, fixture location, proposed operation, policy result, and resulting repository state. Redact secret names where they reveal internal infrastructure. A useful finding distinguishes unsafe runner privileges from a model’s attempt to follow repository text. If a low-privilege review bot produces a bad suggestion but cannot change code, record that containment. If its job can modify protected configuration, identify the actual enforcement gap rather than describing all AI review tools as equivalent.

Interpret the result and retest

Retest external and internal contributor paths after narrowing permissions or separating jobs. Ensure useful review comments still work and privileged follow-up actions use an explicit trusted approval path. OWASP secure-coding-with-AI guidance is background for the development-loop risks. This review does not replace code review, dependency controls, or branch protection. Its value is a concrete answer about what untrusted pull-request content can influence under the configured agent and CI identities.

Assessment scope and next steps

This is a proposed defensive assessment workflow, not a finding about a particular product. Use a staging environment, synthetic records, approved test identities, and an agreed stop condition. Record the model, connector, policy, and application versions so a later result can be compared with the same configuration. Unavailable evidence should remain an explicit coverage gap.

Primary reference: OWASP security guidance. The scenarios and acceptance criteria here are editorial recommendations for adapting that background to an authorized assessment.

For a scoped review of these boundaries, explore FailSafe AI security testing and agentic penetration testing. Agree the specific tools, identities, evidence, and exclusions before purchasing an assessment.