Stateful Agents

Context Hierarchy

How to manage different types of information for Letta agents

Letta offers multiple abstractions for how to contextualize agents with additional external context and long-term memory:

  • You can create a memory block that persists information in-context
  • You can create a file which the agent can read segments of and search
  • You can write to archival memory for the agent to later query via built-in tools
  • You can use an external DB (e.g. vector DB, RAG DB) to store data, and make the data accessible to your agent via tool calling (e.g. MCP)

In general, which abstraction to use depends on the scale of data and how important it is for the agent. For smaller amounts of data, it is best to simply place everything into the context window with memory blocks. For larger amounts of data, you may need to store data externally and retrieve it.

See the feature sets and recommended size limit (per block/files/archival memory) and count limits (total blocks/files/archival memories) below:

AccessIn-ContextToolsSize LimitCount Limit
Memory BlocksEditable (optional read-only)Yesmemory_rethink
memory_replace
memory_insert
& custom tools		Recommended <50k charactersRecommended <20 blocks per agent
FilesRead-onlyPartial (files can be opened/closed)open
close
semantic_search
grep		5MBRecommended <100 files per agent
Archival MemoryRead-writeNoarchival_memory_insert
archival_memory_search
& custom tools		300 tokensUnlimited
External RAGRead-writeNoCustom tools or MCPUnlimitedUnlimited

Examples

Below are examples of when to use which abstraction type:

Example Use CaseRecommended Abstraction
Storing very important memories formed by the agent that always need to be remembered (e.g. “user’s name is Sarah”)Memory Blocks
Giving your agent access to company communication guidelines that is a 1-2 pages longMemory Blocks
Giving your agent access to company documentation that is 100s of pages long or consists of dozens of filesFiles
Storing less important memories formed by the agent that do not always need to be recalled (e.g. “Today Sarah and I talked about our favorite foods and it was pretty funny”)Archival Memory
Giving your agent access to millions of documents you have scrapedExternal RAG