Sources checked: . FailSafe Research.
Vetting an MCP server means answering one question: what can this server reach, and what is the caller allowed to do through it? A tool list and a successful demo do not answer that. The review should connect the server's provenance, the tools it exposes, the identities it uses, and the data and systems on the other side. Use this checklist before deployment or procurement, and keep the evidence with the decision. The MCP assessment test plan describes how to test the same boundaries in a synthetic lab.
Start with a threat model that names the boundaries
An MCP deployment moves data and authority across several components. Write down each boundary before scoring the server. The MCP security overview covers the wider risk model.
| Boundary | Question to answer |
|---|---|
| Client to host runtime | Which application authenticates the user, and what session or consent record exists? |
| Host runtime to server | How is the server identified, launched and constrained, and who owns its configuration? |
| Server to tool | Which caller and service credential may invoke each tool, and on which objects? |
| Tool to upstream service | What network, tenant and data boundaries does the tool cross when it acts? |
| Server to log and evidence store | What is recorded, where, for how long, and who can read it? |
The vetting checklist
Ask for the artifact, not only the description. A vendor or internal owner should be able to show each item in a synthetic or redacted review.
| Area | Question | Evidence to request | Stop condition |
|---|---|---|---|
| Provenance and version | Where did the server come from, and which version is running? | Repository or package source, image digest, dependency lockfile, owner. | Unpinned source or no owner. |
| Tool inventory | Which tools exist, what do they change, and which are read-only? | Tool definitions, side-effect classification, approval rules. | Undocumented write or destructive tool. |
| Caller authorization | Which user or service may call each tool? | Role matrix, policy decision, allowed and denied test cases. | Shared account with broad access. |
| Token audience | Does the server reject a token issued for another service? | Issuer and audience check, rejection point, valid-token control. | Token accepted without audience validation. |
| Tenant and object isolation | Can one workspace reach another workspace's record? | Synthetic cross-tenant test, object-owner check, canary result. | Cross-tenant read succeeds. |
| Secret handling | Where are credentials stored, scoped and rotated? | Secret manager, token scopes, rotation and revocation test. | Long-lived broad credential in configuration. |
| Network egress | Which destinations can a tool reach? | Allowlist, egress policy, denied-destination test. | Unrestricted outbound access. |
| Side effects and approvals | Which actions require human approval? | Approval gate, named approver, audit record. | Destructive action without approval. |
| Logging and retention | Can the owner reconstruct a tool decision? | Decision log, sanitized arguments, retention and access policy. | No record of permission decisions. |
| Update path | How are tool definitions, dependencies and policies changed? | Change review, version pinning, rollback procedure. | Silent tool or policy changes. |
A scoring worksheet, not a universal risk score
A numeric score is useful only when the inputs, scale and limits are visible. Use the same three-point scale for every area. Zero means no evidence. One means the answer is described but not shown. Two means the owner showed the artifact in a synthetic or redacted review and an engineer could repeat the check. Do not add the numbers into a single marketing claim or compare scores across different scopes.
| Area | Checks | What a two requires |
|---|---|---|
| Control and authority | Caller authorization, approvals, side effects | Role matrix, named approver, denied write case. |
| Identity and isolation | Token audience, tenant and object isolation | Rejected wrong-audience token, denied cross-tenant read, valid control. |
| Data and network | Secrets, egress, data retention | Scoped credential, allowlist, retention record. |
| Evidence and change | Logging, provenance, update path | Sanitized decision log, pinned version, reviewed change. |
Keep the evidence with the decision
Record the server version, configuration digest, identity used, fixture identifiers, observed decision and any coverage gap. If the deployment cannot show whether denied content reached the model or a downstream service, label that result inconclusive. A model refusal after retrieval is not evidence that retrieval was denied.
What this vetting does not establish
This is a review aid, not a certification or a guarantee of safety. It does not cover every deployment, model or upstream integration. Use it to decide what to inspect next, then test the highest-consequence paths under an agreed scope. Teams that need an active test can use the test plan or discuss an MCP assessment with FailSafe.
Questions & answers
Questions about this topic
Direct answers to common questions about this page.
No. A checklist shows whether the owner can evidence specific controls. It does not prove that every tool path, tenant boundary or upstream integration is safe, so test the highest-consequence paths under an agreed scope.
Only if the score exposes its inputs, scale, date and limits. A single number can hide which boundary failed. Use a scored worksheet per area and keep the supporting evidence beside it.
Ask for the server version and provenance, tool inventory, role matrix, token audience check, a denied cross-tenant case, credential scope, egress policy, approval gate and sanitized decision log. A description without an artifact is a one, not a two.
Related Articles
ARTEX Threat Report: How Agentic Intrusions Cross Web, Identity and Cloud Boundaries
A detailed ARTEX report separating observed campaigns, access paths, cloud behavior and attribution limits, with a defensive containment and retest framework....
Buying an AI Agent Pentest: Permissions, Tools, and Evidence
Scope an AI-agent assessment around authorized capabilities, tool permissions, data boundaries, human approval, and observable outcomes....
MCP Security Assessment: A Practical Test Plan and Safe Lab
A practical MCP assessment test plan and synthetic lab covering tool authorization, confused-deputy paths, token audience, tenant isolation, unsafe side effects...
Put this guidance to work
Bring the deployed servers, tools, identities and permitted actions. FailSafe can turn the vetting questions into an authorized test with reproduced evidence and a retest.
Startup offer: $999 USD for 250 credits and one authorized black-box run. Sign in to SWARM before Stripe checkout. Custom scope and sovereign deployments are separate.