RAG and memory both put useful context in front of a model, but they govern different things. RAG selects external source evidence for the current request.
Memory governs state that must survive, change, expire, or return across requests.
That boundary matters more than the storage technology. A vector index can retrieve product documentation for RAG or past events for memory.
A conversation buffer can hold recent state, but it is not a complete memory system unless the application also controls what enters it, how long it remains valid, and when it may return.
RAG and memory have different owners
RAG owns evidence selection for one request
A RAG pipeline starts with an information need and a corpus. It forms a retrieval query, selects source records, and passes a bounded set of those records to the model.
The answer should be traceable to what the retriever returned, not to the wording of the query or to an unsupported model guess.
The corpus can change and the retriever can use more than one index. Neither fact turns RAG into memory.
Its defining job is selecting evidence that can answer the current request.
A vector store is therefore not synonymous with RAG. Dense vectors are one retrieval path.
Lexical indexes, metadata filters, rerankers, and direct database lookups can all take part in the same evidence-selection step.
Memory owns state across requests
A memory system decides which observations become durable state, where that state belongs, how it can be retrieved, and when it should be superseded or removed. A preference, a completed action, and a temporary tool result have different lifecycles even if each can be represented as text.
The five-layer memory hierarchy makes those lifecycle decisions explicit. The current event and working context can expire quickly, while episodic, semantic, and procedural records need separate inclusion, retrieval, and retention rules.
A recent-message buffer covers only one part of that design. It does not decide whether a record remains valid next week, whether a later record replaces it, or whether another user may read it.
The context window owns neither job
The context window is the container the model reads during one inference call. It can contain retrieved evidence, selected memories, instructions, tool results, and the current request.
It does not decide which of those inputs are authoritative or durable.
That distinction is easy to lose when a large window can hold a long transcript. The context-window guide explains why capacity alone does not guarantee that every included fact will be used reliably.
Selection and placement still belong to the application.
A hybrid request loop keeps evidence and state separate
Read relevant durable state first
The application begins with a scoped memory read. Identity and access rules apply before similarity search, and the result should include only state relevant to the current task.
The system can then recover a user-approved constraint, an earlier task outcome, or another record that must influence the next step.
Reading memory does not yet answer the request. The returned state helps the application understand the task and form a better retrieval query.
Retrieve source evidence for the current question
The application then searches the external corpus. Current task state can improve the query, but it is not a substitute for the source records the answer needs.
The two inputs should remain distinguishable in the assembled prompt. Memory can say which service and deployment the request concerns.
Retrieved documentation can say what that service supports. If they conflict, the application needs an explicit authority rule rather than one undifferentiated block of text.
Build a bounded context and answer from evidence
The host selects the useful memory records and retrieved passages, labels their provenance, and fits them within a deliberate context budget. More retrieved text is not automatically safer.
Irrelevant passages compete with the records that actually answer the question.
The model can now answer from the selected evidence while respecting the durable constraints supplied by memory. Citations should point to source records, not to a memory summary unless that memory record is itself the evidence being discussed.
Make the memory write a separate decision
An answer should not become durable memory merely because the model produced it. After the response, the application decides whether anything new is worth storing, which layer owns it, what source supports it, and when it expires.
Separating the write prevents retrieval noise and model wording from silently becoming future state. It also gives the system a clear boundary where validation, attribution, access control, and supersession rules can run.
Hybrid retrieval for agent memory starts with asymmetric queries
Agent queries come from partial task state
A human usually phrases the missing information directly. An agent often forms a query while partway through a plan, using the observations and vocabulary available at that step.
Consider a deployment record. A person might ask, "What were the deployment steps for this service?"
An agent that has already attempted the deployment may ask, "What failed last time?" Both queries can need the same record, but their language and starting state differ.
That is the retrieval asymmetry: the generated query and the stored record describe the same task from different positions. Evaluation data made only from polished human questions will not expose the mismatch.
Dense retrieval finds meaning while lexical retrieval protects exact terms
Dense retrieval helps when the query and record use different words for the same idea. Lexical retrieval protects error codes, configuration keys, service names, and other identifiers that should match exactly.
Anthropic's primary Contextual Retrieval explanation uses an error-code example to show this split. An embedding can return material about error codes and miss the exact code.
BM25 can match the identifier.
The site's hybrid-search guide covers how dense and lexical result lists can be combined without pretending their raw scores share one scale.
For agent memory, the useful question is not whether dense or lexical search wins in general. It is whether their combined candidate set recovers the records required by the queries the agent actually forms.
Metadata filters enforce boundaries similarity cannot
Some requirements are constraints, not relevance signals. Identity, service, event type, time, and access boundaries should be represented as typed fields and enforced before a result reaches the model.
An embedding score cannot prove that a record belongs to the current tenant. A lexical match on a service name cannot prove that the record was valid during the requested time range.
Search can rank candidates only after the host has applied the boundaries that must never be approximate.
Hybrid retrieval for agent memory therefore uses dense search for semantic similarity, lexical search for exact terms, and metadata filters for enforceable scope.
Query transformation must match the kind of mismatch
HyDE moves a query toward document language
Hypothetical Document Embeddings, or HyDE, asks a model to draft an answer-like passage and embeds that passage for retrieval. The extra language can bring a short or indirect query closer to the language used by stored documents.
The hypothetical passage is a search representation, never evidence for the final answer. The system still has to retrieve real source records and ground its response in them.
HyDE is useful when the query lacks the terms a relevant passage is likely to contain. It does not enforce tenant identity, dates, permissions, or other exact constraints.
Self-querying separates semantic text from typed constraints
Self-querying retrieval parses a natural-language request into semantic search text and structured filters. A request for a tenant's failed deployments during a particular time range can keep the failure description in the search text while moving tenant, event type, service, and time into validated fields.
The host should validate those fields against an allowed schema and the caller's permissions. The model can propose a filter, but it cannot grant itself access or invent a field the retrieval layer does not support.
HyDE and self-querying solve different problems. HyDE shifts vocabulary toward document language.
Self-querying moves exact boundaries out of fuzzy similarity and into enforceable structure. A pipeline can use either or both, but their outputs need different validation.
Evaluation has to reproduce production query formation
Capture the state present when each query is formed
A useful retrieval fixture includes the agent's goal, the partial task state available at that step, the exact generated query, and the source records expected to satisfy it. Keeping only the final human-readable question removes the conditions that caused the agent to phrase the query as it did.
The fixture should also retain the identity and access scope that governs the search. A result can be semantically relevant and still be wrong because it belongs to another user, service, event type, or time period.
Score retrieval and answer use separately
Retrieval evaluation asks whether the expected records appear and whether forbidden records stay out. Answer evaluation asks whether the model uses the returned evidence correctly, preserves the memory constraints, and avoids treating a HyDE passage or generated query as a source.
Those checks catch different failures. A strong answer cannot repair a missing source record, and high retrieval recall does not prove that the final response used the evidence rather than a plausible prior.
Human-authored questions can supplement this set. They cannot substitute for agent-generated queries because they omit the partial task state that creates the asymmetry.
Test memory writes independently
The same fixture can end with a proposed memory update, but that update needs its own assertions. The test should check whether the record is eligible for storage, whether it has a source, which layer owns it, and what would supersede or expire it.
Separate assertions keep answer quality from becoming a proxy for memory quality. A correct answer may produce no durable update at all.
Choose the owner before choosing the storage
Use RAG when the request needs external evidence
If the answer depends on documentation, code, policy, or another corpus, RAG owns the evidence-selection step. The retrieval implementation can be dense, lexical, filtered, reranked, or a combination of those methods.
The important property is provenance: the final answer can be checked against the records selected for that request.
Use memory when state must survive or change
If the application needs a preference, prior outcome, task constraint, or other state to return later, memory owns the lifecycle. Storage is only one part of that job.
The application also needs inclusion, retrieval, supersession, expiry, deletion, and access rules.
A vector index may help retrieve those records, but using vectors does not turn the memory layer into ordinary RAG.
Use the hybrid loop when one request needs both
Many agent requests need durable state to form the right question and external evidence to answer it. Read scoped memory, retrieve sources with a query shaped by current task state, assemble a bounded context, answer from evidence, and evaluate any memory write separately.
That sequence preserves the boundary even when both systems share infrastructure. RAG determines what evidence the model sees now.
Memory determines what state is allowed to matter again later.