How to Stop Prompt Injection from Poisoning an AI Agent’s Memory

AI agents that browse websites, read documents, or process emails can encounter malicious instructions hidden among legitimate content. The risk grows when the system stores what it learns in persistent memory: the instruction remains there, survives the end of the session, and may be activated days later. A demonstration built with Strands Agents reproduces this attack and shows why sensitive actions must be protected by rules the model cannot override.

Memory poisoning in AI agents: the key facts in 20 seconds

• The attack comes from external content: a website, PDF, or email can contain instructions aimed at the agent rather than the user.
• Memory increases the risk: malicious content can be stored and resurface in later sessions.
• A better prompt is not enough: telling the model to ignore suspicious instructions may help, but it is not a reliable security barrier.
• The demo reproduces a genuine cross-session attack: a new agent reloads poisoned memory from disk and attempts to send information to an attacker.
• The defence is enforced before the tool runs: a hook checks the recipient and cancels the email if the domain is not authorised.
• The memory remains poisoned: the control prevents one specific action, but it does not remove the malicious content from storage.
• Production systems need several layers: memory write and retrieval policies, least-privilege permissions, tool validation, audit logs, and human approval for sensitive actions.

The attack does not end when the conversation closes

Prompt injection occurs when a model interprets text that should be treated as information as an instruction to follow. It can be direct, when the user writes the instruction, or indirect, when it appears inside an external source consulted by the system.

An agent browsing the web to compare hotels could read a page containing a hidden sentence instructing it to send future booking details to an unauthorised address. The user has not given that order, and the attacker does not need to compromise the server. They only need to publish content that they expect the agent to process.

In an application without memory, the effect might be limited to that execution. The situation changes when the agent summarises the page and saves its findings for later use. The instruction no longer appears as external content. It resurfaces inside memory that the system regards as its own.

This behaviour is known as memory poisoning. The agent may retrieve the instruction weeks later, during an otherwise legitimate request, and act as though it were part of the user’s preferences or an internal operating procedure.

The academic paper Zombie Agents: Persistent Control of Self-Evolving LLM Agents via Self-Reinforcing Injections, published in February 2026, examines this persistence. Its authors show how an indirect injection can become embedded in an agent’s state, survive across sessions, and later trigger unauthorised tool use. The research also warns that filters applied only to each conversation prompt are insufficient once the dangerous content has already entered memory. (arxiv.org)

This does not mean system instructions, classifiers, or visual separation of external content are useless. They can reduce the number of successful attacks. The problem is relying on them as the only defence when the model can send messages, query databases, modify files, execute code, or authorise transactions.

OWASP continues to list prompt injection as the leading risk in its ranking for applications based on large language models. The organisation states that there is no infallible way to prevent it and recommends combining controls such as least privilege, isolation of untrusted content, deterministic validation, human approval, and adversarial testing. (genai.owasp.org)

A demo that infects, restarts, and blocks the agent

The resilient-agent-harness-sample-for-aws repository, published by Elizabeth Fuentes, contains several demonstrations of failures in AI agents. One of them focuses on memory poisoning and uses a hotel-booking assistant as its example.

The exercise is divided into three stages. In the first, a normal note and a poisoned note are written into the agent’s memory. The malicious entry contains an instruction to send booking information to an address controlled by a supposed attacker.

The memory is stored in agent.state, the native state mechanism used by Strands Agents, and persisted in the file system through FileSessionManager. Persistence is not simulated by keeping a variable alive within the same process. The demonstration later creates another agent, which reloads the state saved by the previous session from disk.

During the second stage, the new agent receives an ordinary booking request. After retrieving the poisoned memory, it attempts to use the send_email tool to send the details to [email protected]. The repository records the tool’s actual execution rather than merely checking whether the model’s response contains wording such as “email sent”.

The third stage repeats the test with an additional defence. Before send_email runs, a hook attached to BeforeToolCallEvent extracts the recipient and checks whether its domain appears on an approved list. When the domain is not authorised, it assigns a message to event.cancel_tool, and the call is cancelled.

The outcome published in the demonstration is clear: without the rule, the email tool executes once against the attacker’s address; with the control enabled, no dangerous message reaches execution. The official Strands documentation confirms that FileSessionManager persists messages and agent state in the file system, while pre-tool hooks can cancel or modify an operation before it runs. (github.com)

The important difference lies in who makes the decision. A prompt such as “do not send information to unknown people” still depends on the model correctly interpreting the situation and following the instruction. A domain allowlist enforced through code does not ask the model to behave properly. It physically prevents the action when the policy is not met.

The technique is simple, but it represents a useful principle for any agentic architecture. The model may propose an operation, but an external layer must decide whether that operation is authorised. This separation resembles the traditional security controls of an application more than a conversation with an assistant.

Blocking one email does not clean poisoned memory

The demonstration proves that a rule at the tool boundary can contain one specific path for data exfiltration. It does not show that the injection has disappeared or provide universal protection against every malicious action.

The poisoned note remains in memory. If the agent has access to other tools, it could attempt to send the information through a messaging application, calendar invitation, HTTP request, shared file, or online form. An allowlist applied only to email would not cover those channels.

Even recipient validation requires more care in a real system. Addresses must be normalised, subdomains and aliases handled correctly, ambiguous comparisons avoided, and every recipient checked, including carbon-copy and blind-copy fields. It may also be necessary to restrict the content that can leave the system, the size of attachments, and the type of information each tool is permitted to access.

The policy should not be stored somewhere the agent itself can modify. If the memory can alter configuration, create tools, or edit the code containing the approved domains, an attacker may try to disable the protection. Sensitive rules must remain outside the context controlled by the model and be protected through independent permissions.

A production defence begins before the tool call. Content collected from the web, emails, and documents should retain information about its origin and trust level. It should not automatically become a permanent preference or be mixed with instructions that the user has explicitly verified.

Memory writes need their own policy. Some information may be stored directly, while other content should be quarantined, expire after a set period, or require confirmation. Instructions that attempt to change permissions, recipients, credentials, or internal procedures should not be stored as ordinary facts.

Memory retrieval also needs scrutiny. An old memory does not become trustworthy simply because it comes from an internal database. The system should preserve its provenance, distinguish between external observations and confirmed decisions, and prevent retrieved notes from gaining the same priority as security rules.

The next layer is the tool boundary. Each operation should use credentials with the minimum permissions required and accept parameters narrowly matched to its purpose. An assistant that prepares drafts does not need permission to send them. One that reads the calendar does not necessarily need permission to edit it. An agent that searches for hotels should not automatically have access to the entire contact list.

Irreversible actions, or those with financial, legal, or privacy consequences, require an additional step. Sending sensitive data, making payments, deleting files, publishing content, or changing permissions are strong candidates for explicit human approval.

Logs complete the design. The organisation should be able to determine which memory was retrieved, which tool the model requested, which parameters it proposed, which rule made the decision, and whether the operation was ultimately executed. Without that traceability, an incident can remain hidden behind an apparently normal response.

The main lesson from the example is not that an allowlist solves prompt injection. Its value lies in showing that an AI agent should be treated as an untrusted component of the system, even when it is consulting its own memory. It may reason and propose actions, but effective boundaries must exist in code, policies, and services that do not depend on the model’s willingness to comply.

Frequently asked questions

What is the difference between indirect prompt injection and memory poisoning?
Indirect prompt injection arrives through external content read by the agent. Memory poisoning occurs when that content is stored and continues influencing the system in later sessions.

Can a strong system prompt completely prevent the attack?
No. It may stop some attempts, but the model still processes instructions and data within the same context. Sensitive actions require deterministic controls outside the model.

What exactly does the Strands Agents demo block?
It blocks the email tool when the recipient does not belong to one of the authorised domains. It does not remove the poisoned memory or automatically protect other tools.

What controls should an agent that reads the web or email include?
It should record content provenance, control what enters memory, inspect what is retrieved, restrict permissions, validate every tool call, require approval for sensitive operations, and maintain a complete audit trail.

You can access the code on GitHub from this repository: resilient-agent-harness-sample-for-aws. In this post, we also discuss the Memory Poisoning Defense demo (02-memory-poisoning-defense). Clone the repository and follow the step-by-step instructions.

via: dev.to