Skip to content
My Docs My Docs letta-sdk
  • Auto
  • Light
  • Dark
DiscordForumGitHubSign up
RAG
Simple RAG with Letta
View as Markdown
Copy Markdown
Open in Claude
Open in ChatGPT

Simple RAG with Letta

In the Simple RAG approach, your application manages the retrieval process. You query your vector database, retrieve the relevant documents, and include them directly in the message you send to your Letta agent.

By the end of this tutorial, you’ll have a research assistant that uses your vector database to answer questions about scientific papers.

Prerequisites

Section titled “Prerequisites”

To follow along, you need free accounts for:

  • Letta - To access the agent development platform
  • Hugging Face - For generating embeddings (MongoDB and Qdrant users only)
  • One of the following vector databases:

You will also need Python 3.8+ or Node.js v18+ and a code editor.

Getting Your API Keys

Section titled “Getting Your API Keys”

We’ll need API keys for Letta and your chosen vector database.

Get your Letta API Key

  1. Create a Letta Account

    If you don’t have one, sign up for a free account at letta.com.

  2. Navigate to API Keys

    Once logged in, click on API keys in the sidebar.

    Letta API Key Navigation

  3. Create and Copy Your Key

    Click + Create API key, give it a descriptive name, and click Confirm. Copy the key and save it somewhere safe.

Get your ChromaDB Cloud credentials

  1. Create a ChromaDB Cloud Account

    Sign up for a free account on the ChromaDB Cloud website.

  2. Create a New Database

    From your dashboard, create a new database.

    ChromaDB New Project

  3. Get Your API Key and Host

    In your project settings, you’ll find your API Key, Tenant, Database, and Host URL. We’ll need all of these for our scripts.

    ChromaDB Keys

Get your MongoDB Atlas credentials

  1. Create a MongoDB Atlas Account

    Sign up for a free account at mongodb.com/cloud/atlas/register.

  2. Create a Free Cluster

    Click Build a Cluster and select the free tier (M0). Choose your preferred cloud provider and region and click Create deployment.

    Create MongoDB Cluster

  3. Set Up Database Access

    Next, set up connection security.

    1. Create a database user, then click Choose a connection method

    2. Choose Drivers to connect to your application, choose Python as the driver.

    3. Copy the entire connection string, including the query parameters at the end. It will look like this:

      mongodb+srv://<username>:<password>@cluster0.xxxxx.mongodb.net/?retryWrites=true&w=majority&appName=Cluster0

      MongoDB Connection String

  4. Configure Network Access (IP Whitelist)

    By default, MongoDB Atlas blocks all outside connections. You must grant access to the services that need to connect.

    1. Navigate to Database and Network Access in the left sidebar.

    2. Click Add IP Address.

    3. For local development and testing, select Allow Access From Anywhere. This will add the IP address 0.0.0.0/0.

    4. Click Confirm.

      MongoDB IP Configuration

Get your Qdrant Cloud credentials

  1. Create a Qdrant Cloud Account

    Sign up for a free account at cloud.qdrant.io.

  2. Create a New Cluster

    From your dashboard, click Clusters and then + Create. Select the free tier and choose your preferred region.

    Create Qdrant Cluster

  3. Get Your API Key and URL

    Once your cluster is created, click on it to view details.

    Copy the following:

    1. API Key
    2. Cluster URL

    Qdrant Connection Details

Get your Hugging Face API Token (MongoDB & Qdrant users)

  1. Create a Hugging Face Account

    Sign up for a free account at huggingface.co.

  2. Create Access Token

    Click the profile icon in the top right. Navigate to Settings > Access Tokens (or go directly to huggingface.co/settings/tokens).

  3. Generate New Token

    Click New token, give it a name (e.g., “Letta RAG Demo”), select Read role, and click Create token. Copy the token and save it securely.

    Hugging Face Token

Once you have these credentials, create a .env file in your project directory. Add the credentials for your chosen database:

Terminal window
CHROMA_DATABASE="..." ```
</TabItem>
<TabItem label="MongoDB Atlas">
```bash LETTA_API_KEY="..."
MONGODB_URI="mongodb+srv://username:[email protected]/?retryWrites=true&w=majority&appName=Cluster0"
MONGODB_DB_NAME="rag_demo" HF_API_KEY="..." ```
</TabItem>
<TabItem label="Qdrant">
```bash LETTA_API_KEY="..." QDRANT_URL="https://xxxxx.cloud.qdrant.io"
QDRANT_API_KEY="..." HF_API_KEY="..." ```
</TabItem>
</Tabs>


## Step 1: Set Up the Vector Database


First, we need to populate your chosen vector database with the content of the research papers. We'll use two papers for this demo: ["Attention Is All You Need"](https://arxiv.org/abs/1706.03762) and ["BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding"](https://arxiv.org/abs/1810.04805).


Before we begin, let's create a virtual environment to keep our dependencies isolated:


<Tabs>
<TabItem label="Python">
Before we begin, let's create a Python virtual environment to keep our dependencies isolated:


```bash
python -m venv venv
source venv/bin/activate  # On Windows, use: venv\Scripts\activate

Download the research papers using curl with the -L flag to follow redirects:

curl -L -o 1706.03762.pdf https://arxiv.org/pdf/1706.03762.pdf
curl -L -o 1810.04805.pdf https://arxiv.org/pdf/1810.04805.pdf

Verify the PDFs downloaded correctly:

file 1706.03762.pdf 1810.04805.pdf

You should see output indicating these are PDF documents, not HTML files.

Install the necessary packages for your chosen database:

Python
# requirements.txt
letta-client
chromadb
pypdf
python-dotenv
TypeScript
npm install @letta-ai/letta-client chromadb @chroma-core/default-embed dotenv pdf-ts
npm install --save-dev typescript @types/node ts-node tsx

For Python, install with:

Terminal window
pip install -r requirements.txt

Now create a setup.py or setup.ts file to load the PDFs, split them into chunks, and ingest them into your database:

Python
import os
import chromadb
import pypdf
from dotenv import load_dotenv


load_dotenv()


def main(): # Connect to ChromaDB Cloud
client = chromadb.CloudClient(
tenant=os.getenv("CHROMA_TENANT"),
database=os.getenv("CHROMA_DATABASE"),
api_key=os.getenv("CHROMA_API_KEY")
)


    # Create or get the collection
    collection = client.get_or_create_collection("rag_collection")


    # Ingest PDFs
    pdf_files = ["1706.03762.pdf", "1810.04805.pdf"]
    for pdf_file in pdf_files:
        print(f"Ingesting {pdf_file}...")
        reader = pypdf.PdfReader(pdf_file)
        for i, page in enumerate(reader.pages):
            text = page.extract_text()
            if text:
                collection.add(
                    ids=[f"{pdf_file}-{i}"],
                    documents=[text]
                )


    print("\nIngestion complete!")
    print(f"Total documents in collection: {collection.count()}")


if **name** == "**main**":
main()
TypeScript
import { CloudClient } from 'chromadb';
import { DefaultEmbeddingFunction } from '@chroma-core/default-embed';
import * as dotenv from 'dotenv';
import * as path from 'path';
import * as fs from 'fs';
import { pdfToPages } from 'pdf-ts';


dotenv.config();


async function main() {
    // Connect to ChromaDB Cloud
    const client = new CloudClient({
        apiKey: process.env.CHROMA_API_KEY || '',
        tenant: process.env.CHROMA_TENANT || '',
        database: process.env.CHROMA_DATABASE || ''
    });


    // Create embedding function
    const embedder = new DefaultEmbeddingFunction();


    // Create or get the collection
    const collection = await client.getOrCreateCollection({
        name: 'rag_collection',
        embeddingFunction: embedder
    });


    // Ingest PDFs
    const pdfFiles = ['1706.03762.pdf', '1810.04805.pdf'];


    for (const pdfFile of pdfFiles) {
        console.log(`Ingesting ${pdfFile}...`);
        const pdfPath = path.join(__dirname, pdfFile);
        const dataBuffer = fs.readFileSync(pdfPath);


        const pages = await pdfToPages(dataBuffer);


        for (let i = 0; i < pages.length; i++) {
            const text = pages[i].text.trim();
            if (text) {
                await collection.add({
                    ids: [`${pdfFile}-${i}`],
                    documents: [text]
                });
            }
        }
    }


    console.log('\nIngestion complete!');
    const count = await collection.count();
    console.log(`Total documents in collection: ${count}`);
}


main().catch(console.error);

Run the script from your terminal:

Terminal window
python setup.py

If you are using MongoDB Atlas, you must manually create a vector search index by following the steps below.

Create the Vector Search Index (MongoDB Atlas Only)

  1. Navigate to Atlas Search

    Log in to your MongoDB Atlas dashboard, and click on “Search & Vector Search” in the sidebar.

  2. Create Search Index

    Click “Create Search Index”, choose Vector Search.

  3. Select Database and Collection

    • Database: Select rag_demo (or whatever you set as MONGODB_DB_NAME)
    • Collection: Select rag_collection

  4. Name and Configure Index

    • Index Name: Enter vector_index (this exact name is required by the code)
    • Choose “JSON Editor” (not “Visual Editor”). Click Next
    • Paste this JSON definition:
    {
      "fields": [
        {
          "type": "vector",
          "path": "embedding",
          "numDimensions": 384,
          "similarity": "cosine"
        }
      ]
    }

    Note: 384 dimensions is for Hugging Face’s BAAI/bge-small-en-v1.5 model.

  5. Create and Wait

    Click Next, then click “Create Search Index”. The index will take a few minutes to build. Wait until the status shows as “Active” before proceeding.

Your vector database is now populated with research paper content and ready to query.

Step 2: Create a Simple Letta Agent

Section titled “Step 2: Create a Simple Letta Agent”

For the Simple RAG approach, the Letta agent doesn’t need any special tools or complex instructions. Its only job is to answer a question based on the context we provide. We can create this agent programmatically using the Letta SDK.

Create a file named create_agent.py or create_agent.ts:

import os
from letta_client import Letta
from dotenv import load_dotenv


load_dotenv()


# Initialize the Letta client


client = Letta(token=os.getenv("LETTA_API_KEY"))


# Create the agent


agent = client.agents.create(
name="Simple RAG Agent",
description="This agent answers questions based on provided context. It has no tools or special memory.",
memory_blocks=[
{
"label": "persona",
"value": "You are a helpful research assistant. Answer the user's question based *only* on the context provided."
}
]
)


print(f"Agent '{agent.name}' created with ID: {agent.id}")

Run this script once to create the agent in your Letta project.

Python
python create_agent.py```


```bash title="TypeScript"
npx tsx create_agent.ts

Stateless Agent in Letta UI

Step 3: Query, Format, and Ask

Section titled “Step 3: Query, Format, and Ask”

Now we’ll write the main script, simple_rag.py or simple_rag.ts, that ties everything together. This script will:

  1. Take a user’s question.
  2. Query your vector database to find the most relevant document chunks.
  3. Construct a detailed prompt that includes both the user’s question and the retrieved context.
  4. Send this combined prompt to our Simple Letta agent and print the response.
Python
import os
import chromadb
from letta_client import Letta
from dotenv import load_dotenv


load_dotenv()


# Initialize clients


letta_client = Letta(token=os.getenv("LETTA_API_KEY"))
chroma_client = chromadb.CloudClient(
tenant=os.getenv("CHROMA_TENANT"),
database=os.getenv("CHROMA_DATABASE"),
api_key=os.getenv("CHROMA_API_KEY")
)


AGENT_ID = "your-agent-id" # Replace with your agent ID


def main():
while True:
question = input("\nAsk a question about the research papers: ")
if question.lower() in ['exit', 'quit']:
break


        # 1. Query ChromaDB
        collection = chroma_client.get_collection("rag_collection")
        results = collection.query(query_texts=[question], n_results=3)
        context = "\n".join(results["documents"][0])


        # 2. Construct the prompt
        prompt = f'''Context from research paper:


{context}


Question: {question}


Answer:'''


        # 3. Send to Letta Agent
        response = letta_client.agents.messages.create(
            agent_id=AGENT_ID,
            messages=[{"role": "user", "content": prompt}]
        )


        for message in response.messages:
            if message.message_type == 'assistant_message':
                print(f"\nAgent: {message.content}")


if **name** == "**main**":
main()
TypeScript
import { LettaClient } from '@letta-ai/letta-client';
import { CloudClient } from 'chromadb';
import { DefaultEmbeddingFunction } from '@chroma-core/default-embed';
import * as dotenv from 'dotenv';
import * as readline from 'readline';


dotenv.config();


const AGENT_ID = 'your-agent-id';  // Replace with your agent ID


// Initialize clients
const lettaClient = new LettaClient({
    token: process.env.LETTA_API_KEY || ''
});


const chromaClient = new CloudClient({
    apiKey: process.env.CHROMA_API_KEY || '',
    tenant: process.env.CHROMA_TENANT || '',
    database: process.env.CHROMA_DATABASE || ''
});


async function main() {
    const embedder = new DefaultEmbeddingFunction();
    const collection = await chromaClient.getCollection({
        name: 'rag_collection',
        embeddingFunction: embedder
    });


    const rl = readline.createInterface({
        input: process.stdin,
        output: process.stdout
    });


    const askQuestion = () => {
        rl.question('\nAsk a question about the research papers (or type "exit" to quit): ', async (question) => {
            if (question.toLowerCase() === 'exit' || question.toLowerCase() === 'quit') {
                rl.close();
                return;
            }


            // 1. Query ChromaDB
            const results = await collection.query({
                queryTexts: [question],
                nResults: 3
            });


            const context = results.documents[0].join('\n');


            // 2. Construct the prompt
            const prompt = `Context from research paper:
${context}


Question: ${question}


Answer:`;


            // 3. Send to Letta Agent
            const response = await lettaClient.agents.messages.create(AGENT_ID, {
                messages: [{ role: 'user', content: prompt }]
            });


            for (const message of response.messages) {
                if (message.messageType === 'assistant_message') {
                    console.log(`\nAgent: ${(message as any).content}`);
                }
            }


            askQuestion();
        });
    };


    askQuestion();
}


main().catch(console.error);

When you run this script, your application performs the retrieval, and the Letta agent provides the answer based on the context it receives. This gives you full control over the data pipeline.

Next Steps

Section titled “Next Steps”

Now that you’ve integrated Simple RAG with Letta, you can explore more advanced integration patterns:

Agentic RAG

Learn how to empower your agent with custom search tools for autonomous retrieval.

Custom Tools

Explore creating more advanced custom tools for your agents.