👿 MCP is all fun, until you add this one malicious MCP server and forget about it. We have discovered a critical flaw in the widely-used Model Context Protocol (MCP) that enables a new form of LLM attack we term 'Tool Poisoning'. Leaks SSH key, API keys, etc. Details below 👇
When an MCP server is added to an agent like @cursor_ai, Claude or the @OpenAI Agents SDK, its tool's descriptions included in the context of the agent. This opens the doors wide open for a novel type of indirect prompt injection, we coin tool poisoning.
What's concerning about this, is that AI models are trained to precisely follow those instructions, rather than be vary about them. This is new about MCP, as before, agent developers could be relatively trusted, now everything is fair game.
It's trivial to craft a malicious tool description like below, that completely hijacks the agent, while pretending towards the user everything is going great.
Lastly, not only can you expose malicious tools, tool descriptions can also be used to change the agent's behavior with respect to other tools, which we call 'shadowing'. This way all you emails suddenly go out to 'attacker@pwnd.com', rather than their actual receipient.
We wrote up a little report about this, to raise awareness. Please have a look for much more details and scenarios, and our code snippets. Blog: https://invariantlabs.ai/blog/...
Some extended thoughts after reading some of the reactions: https://x.com/lbeurerkellner/s...