Stored Is Not the Same as Remembered: How AI Agents Keep Track

Quick answer

• Stored information isn't used automatically. It has to be selected for each model call.
• Keep memory (kept for later), task state (where the work stands) and context (what the model sees now) separate.
• Store the scope and date with each note, so a one-time instruction doesn't become a permanent rule.

You told your AI assistant something last week. Today it acted as if you never said it.

Or the opposite happens. You gave a one-time instruction, and the agent kept following it long after you stopped caring.

In both cases, the problem often isn't that the information was lost. It's that the system confused three different things: what was stored, what the agent tracks about its work, and what the model actually saw when it answered.

This post separates those three ideas, so you can design and debug agents with more confidence.

Why read this? Because almost every useful agent needs some form of memory, and most of the surprising behaviour people report comes from mixing these three ideas up.

Three words that get mixed up

In this post, we'll use three simple definitions. Other frameworks may use the words differently, so these are working distinctions, not universal rules.

Memory is information kept for possible later use. A user's preference, a saved note, or a past decision.

Application state is the record of where a task stands. What has been done, what's pending, and what the result was.

Context is the information actually given to the model for one particular call. Nothing more, nothing less.

The key idea is that something can be stored without being selected for the next call. Storing a note doesn't make the model read it.

Follow the information

Here's how information moves through a typical agent:

Stored preferences and notes     Task state
            \                      /
             ↓                    ↓
                  Selection
                      ↓
           Context for this call
                      ↓
                    Model
                      ↓
              Answer or action

Notice the selection step in the middle. This is where most surprises happen. Something can exist in your database and still never reach the model. Or the wrong thing can reach it.

Five questions to ask about any memory

Before you store anything, work through these questions:

What deserves to be kept? A temporary detail isn't automatically a lasting preference. "I'm travelling this week" shouldn't become a permanent fact about the user.

When should it be used? Define who the information belongs to, which task it applies to, and how far it should reach.

Where did it come from? Keep track of the source and any uncertainty. A note the user wrote is different from a guess the agent made.

Can it go out of date? Decide how information gets corrected, replaced or deleted. An old address or an old policy can quietly cause trouble for months.

Did it actually reach the call? Before you blame storage, check the selection and context assembly. Many "the agent forgot" problems are really "the agent never saw it."

These are good engineering habits, not a universal memory standard. Adapt them to your own application.

A worked example: a stored preference

Suppose your agent has this note saved from an earlier chat:

"The user prefers Python examples."

Now the user asks, "Show this one in Go."

What should happen? A helpful system follows the explicit request. The old general preference shouldn't quietly override what the user just asked for.

So the design question isn't only "should we store preferences?" It's "how should a current instruction and a stored preference interact?" Don't treat every saved note as permanently authoritative. Stored information is a suggestion for the current situation, unless your design says otherwise.

State is a different concern

Now imagine PolicyPal, a fictional company-policy assistant. It has already submitted a cancellation request for a customer. Its task record might look like this:

request_id: REQ-104
submission_status: confirmed
next_step: notify_user

This isn't a writing-style preference. It records real progress. It may be the thing that stops the agent from submitting the same cancellation twice.

Because state affects what actions happen, it usually needs more reliable storage than a casual note. How much reliability you need depends on what goes wrong if the record is missing or wrong.

Keeping track over long conversations

Longer tasks can overflow what a model can hold at once. Anthropic describes two approaches that help here:

• Compaction: summarising the conversation so far into a shorter context for the next step.
• Structured notes: saving key information outside the current context, so it can be retrieved later.

Both approaches depend on careful choices about what to keep. A summary can lose something important, and a note can go stale. The source link at the end has more detail.

Pause and think

Here's a common failure. The original instruction says:

"Use the India policy for this request only."

A later summary becomes:

"Use the India policy."

What was lost?

Pause before reading on.

The answer: the scope. "For this request only" disappeared. Now a later call may treat a one-time instruction as a permanent rule, and every future answer for that user could be wrong.

The fix isn't only in the storage system. Check how summaries are written and how information is written back to memory. The small words that limit scope are often the first to disappear.

A quick checklist

Before you trust an agent to remember something, ask yourself:

• Is this a lasting preference, or a detail for this task only?
• Does the stored information carry its scope, source and date?
• Would a current instruction override it, and does my design say so?
• Is this task state, and does it need more reliable storage?
• When the agent forgets, did the information reach the model at all?

Trade-off

Keeping more information helps continuity, but it can also keep stale, unnecessary or sensitive details. Keeping less lowers those risks, but the agent may lose useful context.

There's no single right answer. Decide deliberately what to keep, for how long, and who can see it.

A note on limits

This post explains a way to think about memory, state and context. It isn't a guide to building a specific system, and it isn't a privacy assessment. The examples are fictional.

Keep learning

Want to connect this to the rest of agent design?

• Read the trace before changing the prompt: https://roy-s-ai-lab.vercel.app/blog/read-the-agent-trace-before-changing-the-prompt
• Designing clear tool interfaces: https://roy-s-ai-lab.vercel.app/blog/your-agent-tool-interface-is-a-contract
• Free AI agent interview answers: https://roy-s-ai-lab.vercel.app/interview-qa/AI%20Agents
• Full guides and scenario packs: https://roy-s-ai-lab.vercel.app/products

Try this: explain the difference between memory, state and context to a friend, using your own example. If you can name what would go wrong with each one, you've got the idea.

Source

• Anthropic, "Effective context engineering for AI agents": https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents

The three definitions, the worked examples and the checklist are practical teaching material built on the source, not a formal standard from it.

Ritesh Rai