Prompt Injection Is Not Solved: Design Your Agents Around It

1. The Problem: Instructions and Data Share One Channel

A classic program keeps code and data apart. An LLM does not: your system prompt, the user’s request and the body of an email arrive as one stream of tokens. OWASP’s LLM01:2025 splits the problem in two. Direct injection is when the user’s own input changes the model’s behaviour. Indirect injection is when the model ingests external content — a web page, a file, an email — that carries instructions of its own. Agents live in the second category all day, because reading outside content is their job.

So the engineering stance changes. Assume the model will sometimes obey the wrong text, and design so that a fooled model has nothing valuable within reach and no way to send it out. That is architecture, not prompt writing, and it is what the rest of this post is about.

2. Minimum Concepts Before Touching Code

Six ideas carry the whole post. Each card links to the primary source, so you can check my summary against the authors’ words.

3. Worked Audit: An Email Assistant Against the Three Legs

Meet MailPilot, a hypothetical assistant that stands in for the many that exist. It has three tools: read_inbox, search_mail and send_email. It summarises overnight mail, drafts replies and — in the version that worries me — sends them on its own. No attack is needed to see the problem; a design review on paper is enough.

Writing the audit as code makes it enforceable. Every tool declares which properties it grants, and a startup check refuses to boot an agent whose combined toolset covers all three — the Rule of Two as an assertion.

TypeScript — capability manifest plus a Rule of Two check

type Property = "untrusted_input" | "private_data" | "external_effect";

interface ToolSpec {
  name: string;
  grants: Property[];
}

const MAILPILOT_TOOLS: ToolSpec[] = [
  { name: "read_inbox", grants: ["untrusted_input", "private_data"] },
  { name: "search_mail", grants: ["untrusted_input", "private_data"] },
  { name: "send_email", grants: ["external_effect"] },
];

// Rule of Two as a startup assertion: at most two properties per session.
export function assertRuleOfTwo(tools: ToolSpec[]): void {
  const held = new Set(tools.flatMap((t) => t.grants));
  if (held.size === 3) {
    throw new Error(
      "Rule of Two violated: untrusted input + private data + external effect. " +
        "Drop a tool, split the session, or require human approval."
    );
  }
}

assertRuleOfTwo(MAILPILOT_TOOLS); // throws: this agent holds all three

4. Step 1 — Cut Leg 3 with an Egress Allow-List

Start where the damage leaves the building. One module owns every outbound effect and denies by default: recipients must be known contacts or sit on an allow-listed domain, HTTP goes only to hosts you operate, and redirects are not followed. The model proposes; this code disposes.

TypeScript — egress.ts, the only door out

// egress.ts - the only module allowed to talk to the outside world.
const RECIPIENT_DOMAINS = new Set(["mycompany.com"]);
const HTTP_HOSTS = new Set(["api.mycompany.com"]);

export class EgressDenied extends Error {}

export function assertRecipient(address: string, knownContacts: ReadonlySet<string>): string {
  const addr = address.trim().toLowerCase();
  const domain = addr.split("@").pop() ?? "";
  if (!knownContacts.has(addr) && !RECIPIENT_DOMAINS.has(domain)) {
    throw new EgressDenied("recipient not allow-listed: " + domain);
  }
  return addr;
}

export async function safeFetch(rawUrl: string, init: RequestInit = {}): Promise<Response> {
  const url = new URL(rawUrl);
  if (url.protocol !== "https:") throw new EgressDenied("https only");
  if (!HTTP_HOSTS.has(url.hostname)) {
    throw new EgressDenied("host not allow-listed: " + url.hostname);
  }
  // A redirect could leave the allow-list, so never follow one silently.
  return fetch(url, { ...init, redirect: "manual" });
}

Two details matter more than they look. Derive the recipient from structured data your own code parsed, never from model text. And count what your UI renders as egress too: an auto-loaded image or link preview is an outbound request that carries whatever is in its URL.

5. Step 2 — Quarantine Untrusted Text (Dual LLM, Typed Outputs)

Now cut leg 2 as far as the product allows. Reading is still necessary, so confine it: a quarantined model sees the raw email, has no tools and may only answer in a fixed schema. The controller stores the raw text under a handle such as $MAIL_1. The privileged planner — the model that owns the tools — receives handles and typed fields, never the body. This is Willison’s 2023 Dual LLM pattern in its smallest form.

Python — quarantined triage with a Pydantic schema and a variable store

from typing import Literal
from pydantic import BaseModel, Field


class Triage(BaseModel):
    category: Literal["invoice", "meeting", "newsletter", "personal", "other"]
    needs_reply: bool
    urgency: int = Field(ge=1, le=5)


class VarStore:
    """Controller-owned. Untrusted text lives here; the planner sees handles."""

    def __init__(self) -> None:
        self._values: dict[str, str] = {}

    def put(self, text: str) -> str:
        handle = "$MAIL_" + str(len(self._values) + 1)
        self._values[handle] = text
        return handle

    def get(self, handle: str) -> str:
        return self._values[handle]


def quarantined_triage(llm, raw_email: str) -> Triage:
    # No tools are attached to this call and the output is schema-constrained.
    # llm.parse stands for your structured-output call (OpenAI parse, AI SDK Output, instructor).
    return llm.parse(
        system="Classify the email into the schema. You have no tools.",
        user=raw_email,
        schema=Triage,
    )

Python — the planner only ever sees handles and typed fields

def planner_view(handle: str, triage: Triage) -> dict:
    # Typed, bounded fields only: no subject, no body, no free text.
    return {"handle": handle, **triage.model_dump()}


def morning_run(inbox, planner, quarantine_llm, store: VarStore):
    views = []
    for message in inbox.fetch_unread():
        handle = store.put(message.body)  # raw text never leaves the store
        triage = quarantined_triage(quarantine_llm, message.body)
        views.append(planner_view(handle, triage))
    # The privileged planner owns the tools but only ever receives views.
    return planner.plan(task="Order today's mail by urgency", mail=views)

Two limits to remember. Social engineering survives: if the quarantined summary reaches you and sounds persuasive, the human becomes the attack surface. And Willison himself says the pattern is not 100% reliable and that features added later can leak untrusted text into the privileged side — his post now notes that the CaMeL paper found a flaw in his original proposal.

6. Step 3 — Toolsets per Session, Approval Gates and an Audit Log

Last layer: constrain what each session can even attempt. The Rule of Two applies per session, so split MailPilot into modes. Triage reads and classifies but owns no outbound tool. Reply can save drafts locally. Only the send mode — which reads no mail — can transmit, and only after a human approves a card your code rendered.

TypeScript — toolsets per mode and a gate on side effects

type Effect = "read" | "write_local" | "external";
type Mode = "triage" | "reply" | "send";

interface Session {
  id: string;
  mode: Mode;
  untrustedIds: string[]; // ids of emails/pages that entered this context
  askHuman(card: string): Promise<boolean>;
}

interface Tool {
  name: string;
  effect: Effect;
  run(args: Record<string, unknown>): Promise<unknown>;
}

// One session = one mode = at most two of the three properties.
const TOOLSETS: Record<Mode, string[]> = {
  triage: ["read_inbox", "search_mail"], // no outbound tool exists here
  reply: ["read_thread", "save_draft"],  // drafts stay local
  send: ["send_email"],                  // reads no mail; runs after approval
};

// Built by our code from structured fields, never from model-written text.
function renderCard(name: string, args: Record<string, unknown>): string {
  const to = String(args.to ?? "");
  return name + " to domain " + to.split("@").pop() + "\nsubject: " + String(args.subject ?? "");
}

export async function callTool(s: Session, tool: Tool, args: Record<string, unknown>) {
  if (!TOOLSETS[s.mode].includes(tool.name)) {
    throw new Error("tool not in toolset for mode " + s.mode);
  }
  let decision: "auto" | "approved" | "denied" = "auto";
  if (tool.effect === "external") {
    decision = (await s.askHuman(renderCard(tool.name, args))) ? "approved" : "denied";
  }
  auditLog(s, tool, args, decision);
  if (decision === "denied") throw new Error("denied by human");
  return tool.run(args);
}

The log is how you notice an attack that worked. Record every tool call with its mode, an argument hash, the ids of untrusted items present in context and the human decision. Then alert on the one combination that should never be silent: an external effect issued while untrusted content was in context.

TypeScript — structured audit log with a taint alert

import { createHash } from "node:crypto";

declare function alertSecurity(reason: string, entry: object): void; // your pager

function auditLog(s: Session, tool: Tool, args: Record<string, unknown>, decision: string) {
  const entry = {
    ts: new Date().toISOString(),
    session: s.id,
    mode: s.mode,
    tool: tool.name,
    effect: tool.effect,
    argsSha256: createHash("sha256").update(JSON.stringify(args)).digest("hex"),
    untrustedIds: s.untrustedIds, // carry these from reply into send so taint survives
    decision,
  };
  console.log(JSON.stringify(entry));
  if (tool.effect === "external" && s.untrustedIds.length > 0) {
    alertSecurity("external effect with untrusted content in context", entry);
  }
}

7. What This Does Not Fix (Read This Before You Ship)

Everything above lowers blast radius. None of it makes prompt injection ‘solved’, and the primary sources are blunt about it.

8. Common Errors (and the Habits That Replace Them)

Almost every weak agent design I review fails in one of these buckets. The left column is what holds up when the model is fooled; the right column is what only works while it is not.

Conclusion

Prompt injection is not a bug waiting for a patch; it is a property of systems that mix instructions and data in one channel. What you control is the architecture: which legs of the trifecta an agent holds at once, how far untrusted text travels, and who confirms before something leaves.

This week, take your busiest agent and write its tool manifest like the audit above. If it covers all three properties, cut one leg for the riskiest session, route outbound effects through one allow-listed module and turn on the log. Then re-run the audit whenever someone adds a tool — especially an MCP server, since Willison notes the protocol makes it easy to mix tools from different sources.

Sources

• Simon Willison — The lethal trifecta for AI agents (16 Jun 2025)
• Meta AI — Agents Rule of Two: A Practical Approach to AI Agent Security (31 Oct 2025)
• Debenedetti et al. — Defeating Prompt Injections by Design (CaMeL), arXiv:2503.18813
• Beurer-Kellner et al. — Design Patterns for Securing LLM Agents against Prompt Injections, arXiv:2506.08837
• Simon Willison — Design Patterns for Securing LLM Agents against Prompt Injections (13 Jun 2025)
• Simon Willison — The Dual LLM pattern for building AI assistants that can resist prompt injection (25 Apr 2023)
• Nasr, Carlini et al. — The Attacker Moves Second, arXiv:2510.09023
• Simon Willison — New prompt injection papers: Agents Rule of Two and The Attacker Moves Second
• OWASP GenAI — LLM01:2025 Prompt Injection