Simple RAG with Letta

In the Simple RAG approach, your application manages the retrieval process. You query your vector database, retrieve the relevant documents, and include them directly in the message you send to your Letta agent.

By the end of this tutorial, you’ll have a research assistant that uses your vector database to answer questions about scientific papers.

Prerequisites

To follow along, you need free accounts for:

• Letta - To access the agent development platform
• Hugging Face - For generating embeddings (MongoDB and Qdrant users only)
• One of the following vector databases:
  • ChromaDB Cloud for a hosted vector database
  • MongoDB Atlas for vector search with MongoDB
  • Qdrant Cloud for a high-performance vector database

You will also need Python 3.8+ or Node.js v18+ and a code editor.

Getting Your API Keys

We’ll need API keys for Letta and your chosen vector database.

mongodb+srv://<username>:<password>@cluster0.xxxxx.mongodb.net/?retryWrites=true&w=majority&appName=Cluster0

Once you have these credentials, create a .env file in your project directory. Add the credentials for your chosen database:

$
LETTA_API_KEY="..."
>
CHROMA_API_KEY="..."
>
CHROMA_TENANT="..."
>
CHROMA_DATABASE="..."

Step 1: Set Up the Vector Database

First, we need to populate your chosen vector database with the content of the research papers. We’ll use two papers for this demo: “Attention Is All You Need” and “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding”.

Before we begin, let’s create a virtual environment to keep our dependencies isolated:

$
python -m venv venv
>
source venv/bin/activate  # On Windows, use: venv\Scripts\activate

Download the research papers using curl with the -L flag to follow redirects:

curl -L -o 1706.03762.pdf https://arxiv.org/pdf/1706.03762.pdf
curl -L -o 1810.04805.pdf https://arxiv.org/pdf/1810.04805.pdf

Verify the PDFs downloaded correctly:

file 1706.03762.pdf 1810.04805.pdf

You should see output indicating these are PDF documents, not HTML files.

Install the necessary packages for your chosen database:

# requirements.txt
letta-client
chromadb
pypdf
python-dotenv

$
pip install -r requirements.txt

Now create a setup.py or setup.ts file to load the PDFs, split them into chunks, and ingest them into your database:

1
import os
2
import chromadb
3
import pypdf
4
from dotenv import load_dotenv
5
6
load_dotenv()
7
8
def main():
9
    # Connect to ChromaDB Cloud
10
    client = chromadb.CloudClient(
11
        tenant=os.getenv("CHROMA_TENANT"),
12
        database=os.getenv("CHROMA_DATABASE"),
13
        api_key=os.getenv("CHROMA_API_KEY")
14
    )
15
16
    # Create or get the collection
17
    collection = client.get_or_create_collection("rag_collection")
18
19
    # Ingest PDFs
20
    pdf_files = ["1706.03762.pdf", "1810.04805.pdf"]
21
    for pdf_file in pdf_files:
22
        print(f"Ingesting {pdf_file}...")
23
        reader = pypdf.PdfReader(pdf_file)
24
        for i, page in enumerate(reader.pages):
25
            text = page.extract_text()
26
            if text:
27
                collection.add(
28
                    ids=[f"{pdf_file}-{i}"],
29
                    documents=[text]
30
                )
31
32
    print("\nIngestion complete!")
33
    print(f"Total documents in collection: {collection.count()}")
34
35
if __name__ == "__main__":
36
    main()

Run the script from your terminal:

$
python setup.py

If you are using MongoDB Atlas, you must manually create a vector search index by following the steps below.

1
{
2
  "fields": [
3
    {
4
      "type": "vector",
5
      "path": "embedding",
6
      "numDimensions": 384,
7
      "similarity": "cosine"
8
    }
9
  ]
10
}

Your vector database is now populated with research paper content and ready to query.

Step 2: Create a Simple Letta Agent

For the Simple RAG approach, the Letta agent doesn’t need any special tools or complex instructions. Its only job is to answer a question based on the context we provide. We can create this agent programmatically using the Letta SDK.

Create a file named create_agent.py or create_agent.ts:

1
import os
2
from letta_client import Letta
3
from dotenv import load_dotenv
4
5
load_dotenv()
6
7
# Initialize the Letta client
8
client = Letta(token=os.getenv("LETTA_API_KEY"))
9
10
# Create the agent
11
agent = client.agents.create(
12
    name="Simple RAG Agent",
13
    description="This agent answers questions based on provided context. It has no tools or special memory.",
14
    memory_blocks=[
15
        {
16
            "label": "persona",
17
            "value": "You are a helpful research assistant. Answer the user's question based *only* on the context provided."
18
        }
19
    ]
20
)
21
22
print(f"Agent '{agent.name}' created with ID: {agent.id}")

Run this script once to create the agent in your Letta project.

$
python create_agent.py```
>
>
```bash title="TypeScript"
>
npx tsx create_agent.ts

Step 3: Query, Format, and Ask

Now we’ll write the main script, simple_rag.py or simple_rag.ts, that ties everything together. This script will:

1. Take a user’s question.
2. Query your vector database to find the most relevant document chunks.
3. Construct a detailed prompt that includes both the user’s question and the retrieved context.
4. Send this combined prompt to our Simple Letta agent and print the response.

1
import os
2
import chromadb
3
from letta_client import Letta
4
from dotenv import load_dotenv
5
6
load_dotenv()
7
8
# Initialize clients
9
letta_client = Letta(token=os.getenv("LETTA_API_KEY"))
10
chroma_client = chromadb.CloudClient(
11
    tenant=os.getenv("CHROMA_TENANT"),
12
    database=os.getenv("CHROMA_DATABASE"),
13
    api_key=os.getenv("CHROMA_API_KEY")
14
)
15
16
AGENT_ID = "your-agent-id"  # Replace with your agent ID
17
18
def main():
19
    while True:
20
        question = input("\nAsk a question about the research papers: ")
21
        if question.lower() in ['exit', 'quit']:
22
            break
23
24
        # 1. Query ChromaDB
25
        collection = chroma_client.get_collection("rag_collection")
26
        results = collection.query(query_texts=[question], n_results=3)
27
        context = "\n".join(results["documents"][0])
28
29
        # 2. Construct the prompt
30
        prompt = f'''Context from research paper:
31
{context}
32
33
Question: {question}
34
35
Answer:'''
36
37
        # 3. Send to Letta Agent
38
        response = letta_client.agents.messages.create(
39
            agent_id=AGENT_ID,
40
            messages=[{"role": "user", "content": prompt}]
41
        )
42
43
        for message in response.messages:
44
            if message.message_type == 'assistant_message':
45
                print(f"\nAgent: {message.content}")
46
47
if __name__ == "__main__":
48
    main()

When you run this script, your application performs the retrieval, and the Letta agent provides the answer based on the context it receives. This gives you full control over the data pipeline.

Next Steps

Now that you’ve integrated Simple RAG with Letta, you can explore more advanced integration patterns: