Skip to content
Beta
All articles

MCP tool poisoning across 12 frameworks: 22.9% success and no framework defence

A Dynatrace study ran 9,720 poisoned MCP tests on 12 agent frameworks and 18 models. What 22.9% means, what it doesn't, and the controls the authors suggest.

By The SpanDock team

5 min read

A bar showing 9,720 test cells, of which 2,113 were vulnerable, split into 1,371 where the attacker's tool was called, 487 with network contact and 255 with a secret verifiably sent out

An MCP server describes its tools in plain text, and agent frameworks pass that text to the model unchanged. If the server is hostile, the description can carry instructions. Researchers at Dynatrace have now measured how often that works. Their paper, posted to arXiv on 2 November 2026 and due at the AISec workshop on 15 to 19 November, ran 9,720 tests with real tool execution. Poisoning succeeded in 22.9% of them. The more useful finding for operators is where the resistance came from: the model, not the framework.

What the study did

The authors, Farooq Shaikh and Biagio Montaruli, built an open-source harness called mcpvenom. It combines 12 agent frameworks, 18 models and 45 attack scenarios, which gives the 9,720 cells of the study's results matrix. The frameworks were chosen by download count and include LangChain, Pydantic AI, CrewAI, LlamaIndex and Semantic Kernel.

The attacker model is narrow and realistic. The attacker controls an MCP server that the victim has added to their client configuration. It can change tool descriptions, parameter descriptions in the input schema, and tool responses. It can't change the system prompt, the user's messages, the framework code or the model. The paper says such a server reaches a client through the supply chain, as a package or registry entry, or through a shared configuration file.

What makes the study different from earlier benchmarks is the test for success. It doesn't just check that the model wrote a matching tool call. A canary server records real requests, and the authors checked 1,259 transcripts by hand.

The headline numbers, read carefully

Of 9,720 cells, 2,113 were vulnerable, which is the 22.9%. That figure counts three levels of evidence, and the strongest level is much smaller:

EvidenceCellsShare of vulnerable cells
The attacker's tool was called with the canary payload1,37164.9%
A request reached the attacker's endpoint, without a secret48723.0%
A secret was verifiably sent out25512.1%

The three levels are exclusive, so each vulnerable cell sits in one row. By simple arithmetic, 255 of 9,720 is about 2.6% of all cells with verified exfiltration. The paper also explains why the first level is large. In 1,067 of those cells the model sent the canary to a tool served by the malicious server itself, which logs the call and returns a stub without making a network request. In a real attack the attacker's own server would have received that call.

The other results that stand out:

  • The model decides. Success by model ran from 0.4% to 49.7%. The nine core frameworks clustered between 23% and 26%, and the paper found no sign that framework choice made a difference.
  • No framework had an intentional defence. A code audit of all twelve found tool descriptions passed to the model unvalidated. Two incidental properties helped: Pydantic AI rejects duplicate tool names (16.6% success) and AG2 constrains the executor (4.6%).
  • One framework did worse than the rest. smolagents had the highest rate at 34.4%. The authors suggest its code-generation design, which presents tools as Python objects with docstrings, may be why.
  • Some attacks failed completely. Payloads hidden in invisible Unicode characters never worked in 209 cells. Parameter-binding attacks had the highest class rate, 51.9%.
  • A simple scanner catches less than half. A regex check for instruction-like text in descriptions rejected 20 of 45 scenarios, and flagged 9 of 125 benign descriptions (7.2%). Six of those nine legitimately named another tool.

What the paper does not show

The authors list their own limits. Models were tested as served on 22 to 24 July 2026, and eight had no dated version, so results may shift with a model update. The tests used single-turn interactions and servers run over stdio. Twelve of roughly 50 MCP-capable frameworks were tested. The scenarios come from published techniques, so a model's score may partly reflect having seen similar attacks in training. The paper measures exfiltration only, not other kinds of harm. This is a preprint for a workshop, not a settled result, and the numbers describe a research harness rather than your deployment.

The vendor responses are a finding of their own. The authors filed five disclosures with four organisations, and all were rejected. LangChain's reply to one was that the library "is expected to be passing the information exactly as specified". Microsoft's response, through its security response centre, said the attacker "must already control the MCP server, and AutoGen explicitly documents MCP servers as trusted". The paper notes that this concerns Microsoft's own AutoGen project, not the AG2 fork it tested. The deepset response called it "trusted configuration behavior", though the company agreed to clarify its documentation.

Those positions are consistent. If MCP servers are trusted configuration, the burden falls on whoever chooses the servers.

What it means for engineering and security leaders

Three conclusions follow for people running agents.

First, do not choose a framework for its security. In this data, no framework offered protection a team could rely on, so the protections have to sit around it.

Second, model choice matters a great deal and is not a guarantee. A spread of 0.4% to 49.7% is large. But the paper notes that no model provider offers a service level for injection resistance, and that results could shift with any model update. Re-test when you change models.

Third, the server is the supply chain. The attacker in this study needs only one malicious entry in a client configuration. That makes the question of which MCP servers a machine can load as important as what the model does with their descriptions. An earlier article on MCP server entries as commands makes the same point from the other direction.

Practical takeaways

  1. Keep an approved list of MCP servers and review every addition to a client configuration like a change to a build script.
  2. Pin tool descriptions. The paper suggests storing a hash of each tool's description and schema at first registration and alerting on a change. That addresses a server that behaves at first and changes later.
  3. Scan descriptions, but don't rely on it. The authors' own test shows a partial control. Combine it with change detection and approval of new or changed descriptions.
  4. Limit what a tool can reach. A poisoned description can only use the tools the agent holds. Start tools read-only, and require approval for anything that writes, sends or deletes.
  5. Treat model choice as a security decision, and test your own configuration with the mcpvenom scenarios before and after a model change.
  6. Watch outbound requests from agent hosts. The study's strongest evidence was a request reaching an endpoint nobody expected.

Where a gateway fits

The SpanDock MCP gateway documentation says every connector starts read-only for everyone, that read-only access hides write and destructive tools completely, and that when you grant write access the destructive tools, such as deleting, merging and closing, still need approval. It also says Permissions → Effective permissions shows what one person, computer or user type gets for every tool, and why. The MCP gateway is on the Premium and Enterprise plans. These settings narrow what an agent's connectors can do through the gateway. The page doesn't describe detecting poisoned tool descriptions, and servers outside the gateway are a separate matter.

Sources

  1. Shaikh and Montaruli (Dynatrace): MCP Tool Poisoning in Agentic AI Frameworks, a Cross-Framework Measurement Study (arXiv 2611.03555, 2 November 2026)
  2. mcpvenom: the study's open-source test harness
Cookie categories

Details: Cookie Policy.