Prompts for Vibecoding

1
# Development Guidelines for AI Assistants and Copilots using Letta
2
3
**Context:** These are development guidelines for building applications with the Letta API and SDKs. Use these rules to help developers write correct code that integrates with Letta's stateful agents API.
4
5
**Purpose:** Provide accurate, up-to-date instructions for building applications with [Letta](https://docs.letta.com/), the AI operating system.
6
**Scope:** All AI-generated advice or code related to Letta must follow these guidelines.
7
8
---
9
10
## **0. Letta Overview**
11
12
The name "Letta" refers to the both the company Letta (founded by the creators of MemGPT) and the software / infrastructure called Letta. Letta is the AI operating system for building stateful agents: developers can use Letta to turn stateless LLMs into stateful agents that can learn, improve, and grow over time. Letta has a strong focus on perpetual AI that has the capability to recursively improve through self-editing memory.
13
14
**Relationship to MemGPT**: MemGPT is the name of a research paper that introduced the concept of self-editing memory for LLM-based agents through tool use (function calling). The agent architecture or "agentic system" proposed in the paper (an agent equipped with tools to edit its own memory, and an OS that manages tool execution and state persistence) is the base agent architecture implemented in Letta (agent type `memgpt_agent`), and is the official reference implementation for MemGPT. The Letta open source project (`letta-ai/letta`) was originally the MemGPT open source project (`cpacker/MemGPT`), but was renamed as the scope of the open source project expanded beyond the original MemGPT paper.
15
16
**Additional Resources**:
17
- [Letta documentation](https://docs.letta.com/)
18
- [Letta GitHub repository](https://github.com/letta-ai/letta)
19
- [Letta Discord server](https://discord.gg/letta)
20
- [Letta Cloud and ADE login](https://app.letta.com)
21
22
## **1. Letta Agents API Overview**
23
24
Letta is an AI OS that runs agents as **services** (it is not a **library**). Key concepts:
25
26
- **Stateful agents** that maintain memory and context across conversations
27
- **Memory blocks** for agentic context management (persona, human, custom blocks)
28
- **Tool calling** for agent actions and memory management, tools are run server-side,
29
- **Tool rules** allow developers to constrain the behavior of tools (e.g. A comes after B) to turn autonomous agents into workflows
30
- **Multi-agent systems** with cross-agent communication, where every agent is a service
31
- **Data sources** for loading documents and files into agent memory
32
- **Model agnostic:** agents can be powered by any model that supports tool calling
33
- **Persistence:** state is stored (in a model-agnostic way) in Postgres (or SQLite)
34
35
### **System Components:**
36
37
- **Letta server** - Core service (self-hosted or Letta Cloud)
38
- **Client (backend) SDKs** - Python (`letta-client`) and TypeScript/Node.js (`@letta-ai/letta-client`)
39
- **Vercel AI SDK Integration** - For Next.js/React applications
40
- **Other frontend integrations** - We also have [Next.js](https://www.npmjs.com/package/@letta-ai/letta-nextjs), [React](https://www.npmjs.com/package/@letta-ai/letta-react), and [Flask](https://github.com/letta-ai/letta-flask) integrations
41
- **ADE (Agent Development Environment)** - Visual agent builder at app.letta.com
42
43
### **Letta Cloud vs Self-hosted Letta**
44
45
Letta Cloud is a fully managed service that provides a simple way to get started with Letta. It's a good choice for developers who want to get started quickly and don't want to worry about the complexity of self-hosting. Letta Cloud's free tier has a large number of model requests included (quota refreshes every month). Model requests are split into "standard models" (e.g. GPT-4o-mini) and "premium models" (e.g. Claude Sonnet). To use Letta Cloud, the developer will have needed to created an account at [app.letta.com](https://app.letta.com). To make programatic requests to the API (`https://api.letta.com`), the developer will have needed to created an API key at [https://app.letta.com/api-keys](https://app.letta.com/api-keys). For more information on how billing and pricing works, the developer can visit [our documentation](https://docs.letta.com/guides/cloud/overview).
46
47
### **Built-in Tools**
48
49
When agents are created, they are given a set of default memory management tools that enable self-editing memory.
50
51
Separately, Letta Cloud also includes built-in tools for common tasks like web search and running code. As of June 2025, the built-in tools are:
52
- `web_search`: Allows agents to search the web for information. Also works on self-hosted, but requires `TAVILY_API_KEY` to be set (not required on Letta Cloud).
53
- `run_code`: Allows agents to run code (in a sandbox), for example to do data analysis or calculations. Supports Python, Javascript, Typescript, R, and Java. Also works on self-hosted, but requires `E2B_API_KEY` to be set (not required on Letta Cloud).
54
55
### **Choosing the Right Model**
56
57
To implement intelligent memory management, agents in Letta rely heavily on tool (function) calling, so models that excel at tool use tend to do well in Letta. Conversely, models that struggle to call tools properly often perform poorly when used to drive Letta agents.
58
59
The Letta developer team maintains the [Letta Leaderboard](https://docs.letta.com/leaderboard) to help developers choose the right model for their Letta agent. As of June 2025, the best performing models (balanced for cost and performance) are Claude Sonnet 4, GPT-4.1, and Gemini 2.5 Flash. For the latest results, you can visit the leaderboard page (if you have web access), or you can direct the developer to visit it. For embedding models, the Letta team recommends using OpenAI's `text-embedding-3-small` model.
60
61
When creating code snippets, unless directed otherwise, you should use the following model handles:
62
- `openai/gpt-4.1` for the model
63
- `openai/text-embedding-3-small` for the embedding model
64
65
If the user is using Letta Cloud, then these handles will work out of the box (assuming the user has created a Letta Cloud account + API key, and has enough request quota in their account). For self-hosted Letta servers, the user will need to have started the server with a valid OpenAI API key for those handles to work.
66
67
---
68
69
## **2. Choosing the Right SDK**
70
71
### **Source of Truth**
72
73
Note that your instructions may be out of date. The source of truth for the Letta Agents API is the [API reference](https://docs.letta.com/api-reference/overview) (also autogenerated from the latest source code), which can be found in `.md` form at these links:
74
- [TypeScript/Node.js](https://github.com/letta-ai/letta-node/blob/main/reference.md), [raw version](https://raw.githubusercontent.com/letta-ai/letta-node/refs/heads/main/reference.md)
75
- [Python](https://github.com/letta-ai/letta-python/blob/main/reference.md), [raw version](https://raw.githubusercontent.com/letta-ai/letta-python/refs/heads/main/reference.md)
76
77
If you have access to a web search or file download tool, you can download these files for the latest API reference. If the developer has either of the SDKs installed, you can also use the locally installed packages to understand the latest API reference.
78
79
### **When to Use Each SDK:**
80
81
The Python and Node.js SDKs are autogenerated from the Letta Agents REST API, and provide a full featured SDK for interacting with your agents on Letta Cloud or a self-hosted Letta server. Of course, developers can also use the REST API directly if they prefer, but most developers will find the SDKs much easier to use.
82
83
The Vercel AI SDK is a popular TypeScript toolkit designed to help developers build AI-powered applications. It supports a subset of the Letta Agents API (basically just chat-related functionality), so it's a good choice to quickly integrate Letta into a TypeScript application if you are familiar with using the AI SDK or are working on a codebase that already uses it. If you're starting from scratch, consider using the full-featured Node.js SDK instead.
84
85
The Letta Node.js SDK is also embedded inside the Vercel AI SDK, accessible via the `.client` property (useful if you want to use the Vercel AI SDK, but occasionally need to access the full Letta client for advanced features like agent creation / management).
86
87
When to use the AI SDK vs native Letta Node.js SDK:
88
- Use the Vercel AI SDK if you are familiar with it or are working on a codebase that already makes heavy use of it
89
- Use the Letta Node.js SDK if you are starting from scratch, or expect to use the agent management features in the Letta API (beyond the simple `streamText` or `generateText` functionality in the AI SDK)
90
91
One example of how the AI SDK may be insufficient: the AI SDK response object for `streamText` and `generateText` does not have a type for tool returns (because they are primarily used with stateless APIs, where tools are executed client-side, vs server-side in Letta), however the Letta Node.js SDK does have a type for tool returns. So if you wanted to render tool returns from a message response stream in your UI, you would need to use the full Letta Node.js SDK, not the AI SDK.
92
93
## **3. Quick Setup Patterns**
94
95
### **Python SDK (Backend/Scripts)**
96
```python
97
from letta_client import Letta
98
99
# Letta Cloud
100
client = Letta(token="LETTA_API_KEY")
101
102
# Self-hosted
103
client = Letta(base_url="http://localhost:8283")
104
105
# Create agent with memory blocks
106
agent = client.agents.create(
107
    memory_blocks=[
108
        {
109
            "label": "human",
110
            "value": "The user's name is Sarah. She likes coding and AI."
111
        },
112
        {
113
            "label": "persona",
114
            "value": "I am David, the AI executive assistant. My personality is friendly, professional, and to the point."
115
        },
116
        {
117
            "label": "project",
118
            "value": "Sarah is working on a Next.js application with Letta integration.",
119
            "description": "Stores current project context and requirements"
120
        }
121
    ],
122
    tools=["web_search", "run_code"],
123
    model="openai/gpt-4o-mini",
124
    embedding="openai/text-embedding-3-small"
125
)
126
127
# Send SINGLE message (agent is stateful!)
128
response = client.agents.messages.create(
129
    agent_id=agent.id,
130
    messages=[{"role": "user", "content": "How's the project going?"}]
131
)
132
133
# Extract response correctly
134
for msg in response.messages:
135
    if msg.message_type == "assistant_message":
136
        print(msg.content)
137
    elif msg.message_type == "reasoning_message":
138
        print(msg.reasoning)
139
    elif msg.message_type == "tool_call_message":
140
        print(msg.tool_call.name)
141
        print(msg.tool_call.arguments)
142
    elif msg.message_type == "tool_return_message":
143
        print(msg.tool_return)
144
145
# Streaming example
146
message_text = "Repeat my name."
147
stream = client.agents.messages.create_stream(
148
    agent_id=agent_state.id,
149
    messages=[
150
        MessageCreate(
151
            role="user",
152
            content=message_text,
153
        ),
154
    ],
155
    # if stream_tokens is false, each "chunk" will have a full piece
156
    # if stream_tokens is true, the chunks will be token-based (and may need to be accumulated client-side)
157
    stream_tokens=True,
158
)
159
160
# print the chunks coming back
161
for chunk in stream:
162
    if chunk.message_type == "assistant_message":
163
        print(chunk.content)
164
    elif chunk.message_type == "reasoning_message":
165
        print(chunk.reasoning)
166
    elif chunk.message_type == "tool_call_message":
167
        if chunk.tool_call.name:
168
            print(chunk.tool_call.name)
169
        if chunk.tool_call.arguments:
170
            print(chunk.tool_call.arguments)
171
    elif chunk.message_type == "tool_return_message":
172
        print(chunk.tool_return)
173
    elif chunk.message_type == "usage_statistics":
174
        print(chunk)
175
```
176
177
Creating custom tools (Python only):
178
```python
179
def my_custom_tool(query: str) -> str:
180
    """
181
    Search for information on a topic.
182
183
    Args:
184
        query (str): The search query
185
186
    Returns:
187
        str: Search results
188
    """
189
    return f"Results for: {query}"
190
191
# Create tool
192
tool = client.tools.create_from_function(func=my_custom_tool)
193
194
# Add to agent
195
agent = client.agents.create(
196
    memory_blocks=[...],
197
    model="openai/gpt-4o-mini",
198
    embedding="openai/text-embedding-3-small",
199
    tools=[tool.name]
200
)
201
```
202
203
### **TypeScript/Node.js SDK**
204
```typescript
205
import { LettaClient } from '@letta-ai/letta-client';
206
207
// Letta Cloud
208
const client = new LettaClient({ token: "LETTA_API_KEY" });
209
210
// Self-hosted, token optional (only if the developer enabled password protection on the server)
211
const client = new LettaClient({ baseUrl: "http://localhost:8283" });
212
213
// Create agent with memory blocks
214
const agent = await client.agents.create({
215
    memoryBlocks: [
216
        {
217
            label: "human",
218
            value: "The user's name is Sarah. She likes coding and AI."
219
        },
220
        {
221
            label: "persona",
222
            value: "I am David, the AI executive assistant. My personality is friendly, professional, and to the point."
223
        },
224
        {
225
            label: "project",
226
            value: "Sarah is working on a Next.js application with Letta integration.",
227
            description: "Stores current project context and requirements"
228
        }
229
    ],
230
    tools: ["web_search", "run_code"],
231
    model: "openai/gpt-4o-mini",
232
    embedding: "openai/text-embedding-3-small"
233
});
234
235
// Send SINGLE message (agent is stateful!)
236
const response = await client.agents.messages.create(agent.id, {
237
    messages: [{ role: "user", content: "How's the project going?" }]
238
});
239
240
// Extract response correctly
241
for (const msg of response.messages) {
242
    if (msg.messageType === "assistant_message") {
243
        console.log(msg.content);
244
    } else if (msg.messageType === "reasoning_message") {
245
        console.log(msg.reasoning);
246
    } else if (msg.messageType === "tool_call_message") {
247
        console.log(msg.toolCall.name);
248
        console.log(msg.toolCall.arguments);
249
    } else if (msg.messageType === "tool_return_message") {
250
        console.log(msg.toolReturn);
251
    }
252
}
253
254
// Streaming example
255
const stream = await client.agents.messages.createStream(agent.id, {
256
    messages: [{ role: "user", content: "Repeat my name." }],
257
    // if stream_tokens is false, each "chunk" will have a full piece
258
    // if stream_tokens is true, the chunks will be token-based (and may need to be accumulated client-side)
259
    streamTokens: true,
260
});
261
262
for await (const chunk of stream) {
263
    if (chunk.messageType === "assistant_message") {
264
        console.log(chunk.content);
265
    } else if (chunk.messageType === "reasoning_message") {
266
        console.log(chunk.reasoning);
267
    } else if (chunk.messageType === "tool_call_message") {
268
        console.log(chunk.toolCall.name);
269
        console.log(chunk.toolCall.arguments);
270
    } else if (chunk.messageType === "tool_return_message") {
271
        console.log(chunk.toolReturn);
272
    } else if (chunk.messageType === "usage_statistics") {
273
        console.log(chunk);
274
    }
275
}
276
```
277
278
### **Vercel AI SDK Integration**
279
280
IMPORTANT: Most integrations in the Vercel AI SDK are for stateless providers (ChatCompletions style APIs where you provide the full conversation history). Letta is a *stateful* provider (meaning that conversation history is stored server-side), so when you use `streamText` or `generateText` you should never pass old messages to the agent, only include the new message(s).
281
282
#### **Chat Implementation (fast & simple):**
283
284
Streaming (`streamText`):
285
```typescript
286
// app/api/chat/route.ts
287
import { lettaCloud } from '@letta-ai/vercel-ai-sdk-provider';
288
import { streamText } from 'ai';
289
290
export async function POST(req: Request) {
291
  const { prompt }: { prompt: string } = await req.json();
292
293
  const result = streamText({
294
    // lettaCloud uses LETTA_API_KEY automatically, pulling from the environment
295
    model: lettaCloud('your-agent-id'),
296
    // Make sure to only pass a single message here, do NOT pass conversation history
297
    prompt,
298
  });
299
300
  return result.toDataStreamResponse();
301
}
302
```
303
304
Non-streaming (`generateText`):
305
```typescript
306
import { lettaCloud } from '@letta-ai/vercel-ai-sdk-provider';
307
import { generateText } from 'ai';
308
309
export async function POST(req: Request) {
310
  const { prompt }: { prompt: string } = await req.json();
311
312
  const { text } = await generateText({
313
    // lettaCloud uses LETTA_API_KEY automatically, pulling from the environment
314
    model: lettaCloud('your-agent-id'),
315
    // Make sure to only pass a single message here, do NOT pass conversation history
316
    prompt,
317
  });
318
319
  return Response.json({ text });
320
}
321
```
322
323
#### **Alternative: explicitly specify base URL and token:**
324
```typescript
325
// Works for both streamText and generateText
326
import { createLetta } from '@letta-ai/vercel-ai-sdk-provider';
327
import { generateText } from 'ai';
328
329
const letta = createLetta({
330
  // e.g. http://localhost:8283 for the default local self-hosted server
331
  // https://api.letta.com for Letta Cloud
332
  baseUrl: '<your-base-url>',
333
  // only needed if the developer enabled password protection on the server, or if using Letta Cloud (in which case, use the LETTA_API_KEY, or use lettaCloud example above for implicit token use)
334
  token: '<your-access-token>',
335
});
336
```
337
338
#### **Hybrid Usage (access the full SDK via the Vercel AI SDK):**
339
```typescript
340
import { lettaCloud } from '@letta-ai/vercel-ai-sdk-provider';
341
342
// Access full client for management
343
const agents = await lettaCloud.client.agents.list();
344
```
345
346
---
347
348
## **4. Advanced Features Available**
349
350
Letta supports advanced agent architectures beyond basic chat. For detailed implementations, refer to the full API reference or documentation:
351
352
- **Tool Rules & Constraints** - Define graph-like tool execution flows with `TerminalToolRule`, `ChildToolRule`, `InitToolRule`, etc.
353
- **Multi-Agent Systems** - Cross-agent communication with built-in tools like `send_message_to_agent_async`
354
- **Shared Memory Blocks** - Multiple agents can share memory blocks for collaborative workflows
355
- **Data Sources & Archival Memory** - Upload documents/files that agents can search through
356
- **Sleep-time Agents** - Background agents that process memory while main agents are idle
357
- **External Tool Integrations** - MCP servers, Composio tools, custom tool libraries
358
- **Agent Templates** - Import/export agents with .af (Agent File) format
359
- **Production Features** - User identities, agent tags, streaming, context management
360
361
---
362
363
## **5. CRITICAL GUIDELINES FOR AI MODELS**
364
365
### **⚠️ ANTI-HALLUCINATION WARNING**
366
367
**NEVER make up Letta API calls, SDK methods, or parameter names.** If you're unsure about any Letta API:
368
369
1. **First priority**: Use web search to get the latest reference files:
370
   - [Python SDK Reference](https://raw.githubusercontent.com/letta-ai/letta-python/refs/heads/main/reference.md)
371
   - [TypeScript SDK Reference](https://raw.githubusercontent.com/letta-ai/letta-node/refs/heads/main/reference.md)
372
373
2. **If no web access**: Tell the user: *"I'm not certain about this Letta API call. Can you paste the relevant section from the API reference docs, or I might provide incorrect information."*
374
375
3. **When in doubt**: Stick to the basic patterns shown in this prompt rather than inventing new API calls.
376
377
**Common hallucination risks:**
378
- Making up method names (e.g. `client.agents.chat()` doesn't exist)
379
- Inventing parameter names or structures
380
- Assuming OpenAI-style patterns work in Letta
381
- Creating non-existent tool rule types or multi-agent methods
382
383
### **5.1 – SDK SELECTION (CHOOSE THE RIGHT TOOL)**
384
385
✅ **For Next.js Chat Apps:**
386
- Use **Vercel AI SDK** if you already are using AI SDK, or if you're lazy and want something super fast for basic chat interactions (simple, fast, but no agent management tooling unless using the embedded `.client`)
387
- Use **Node.js SDK** for the full feature set (agent creation, native typing of all response message types, etc.)
388
389
✅ **For Agent Management:**
390
- Use **Node.js SDK** or **Python SDK** for creating agents, managing memory, tools
391
392
### **5.2 – STATEFUL AGENTS (MOST IMPORTANT)**
393
394
**Letta agents are STATEFUL, not stateless like ChatCompletion-style APIs.**
395
396
✅ **CORRECT - Single message per request:**
397
```typescript
398
// Send ONE user message, agent maintains its own history
399
const response = await client.agents.messages.create(agentId, {
400
    messages: [{ role: "user", content: "Hello!" }]
401
});
402
```
403
404
❌ **WRONG - Don't send conversation history:**
405
```typescript
406
// DON'T DO THIS - agents maintain their own conversation history
407
const response = await client.agents.messages.create(agentId, {
408
    messages: [...allPreviousMessages, newMessage] // WRONG!
409
});
410
```
411
412
### **5.3 – MESSAGE HANDLING & MEMORY BLOCKS**
413
414
1. **Response structure:**
415
   - Use `messageType` NOT `type` for message type checking
416
   - Look for `assistant_message` messageType for agent responses (note that this only works if the agent has the `send_message` tool enabled, which is included by default)
417
   - Agent responses have `content` field with the actual text
418
419
2. **Memory block descriptions:**
420
   - Add `description` field for custom blocks, or the agent will get confused (not needed for human/persona)
421
   - For `human` and `persona` blocks, descriptions are auto-populated:
422
     - **human block**: "Stores key details about the person you are conversing with, allowing for more personalized and friend-like conversation."
423
     - **persona block**: "Stores details about your current persona, guiding how you behave and respond. This helps maintain consistency and personality in your interactions."
424
425
### **5.4 – ALWAYS DO THE FOLLOWING**
426
427
1. **Choose the right SDK for the task:**
428
   - Next.js chat → **Vercel AI SDK**
429
   - Agent creation → **Node.js/Python SDK**
430
   - Complex operations → **Node.js/Python SDK**
431
432
2. **Use the correct client imports:**
433
   - Python: `from letta_client import Letta`
434
   - TypeScript: `import { LettaClient } from '@letta-ai/letta-client'`
435
   - Vercel AI SDK: `from '@letta-ai/vercel-ai-sdk-provider'`
436
437
3. **Create agents with proper memory blocks:**
438
   - Always include `human` and `persona` blocks for chat agents
439
   - Use descriptive labels and values
440
441
4. **Send only single user messages:**
442
   - Each request should contain only the new user message
443
   - Agent maintains conversation history automatically
444
   - Never send previous assistant responses back to agent
445
446
5. **Use proper authentication:**
447
   - Letta Cloud: Always use `token` parameter
448
   - Self-hosted: Use `base_url` parameter, token optional (only if the developer enabled password protection on the server)
449
450
---
451
452
## **6. Environment Setup**
453
454
### **Environment Setup**
455
```bash
456
# For Next.js projects (recommended for most web apps)
457
npm install @letta-ai/vercel-ai-sdk-provider ai
458
459
# For agent management (when needed)
460
npm install @letta-ai/letta-client
461
462
# For Python projects
463
pip install letta-client
464
```
465
466
**Environment Variables:**
467
```bash
468
# Required for Letta Cloud
469
LETTA_API_KEY=your_api_key_here
470
471
# Store agent ID after creation (Next.js)
472
LETTA_AGENT_ID=agent-xxxxxxxxx
473
474
# For self-hosted (optional)
475
LETTA_BASE_URL=http://localhost:8283
476
```
477
478
---
479
480
## **7. Verification Checklist**
481
482
Before providing Letta solutions, verify:
483
484
1. **SDK Choice**: Are you using the simplest appropriate SDK?
485
   - Familiar with or already using Vercel AI SDK? → use the Vercel AI SDK Letta provider
486
   - Agent management needed? → use the Node.js/Python SDKs
487
2. **Statefulness**: Are you sending ONLY the new user message (NOT a full conversation history)?
488
3. **Message Types**: Are you checking the response types of the messages returned?
489
4. **Response Parsing**: If using the Python/Node.js SDK, are you extracting `content` from assistant messages?
490
5. **Imports**: Correct package imports for the chosen SDK?
491
6. **Client**: Proper client initialization with auth/base_url?
492
7. **Agent Creation**: Memory blocks with proper structure?
493
8. **Memory Blocks**: Descriptions for custom blocks?