Talk to Your PDF

Upload PDFs and query them with an AI agent

Overview

This tutorial demonstrates how to build a PDF chat application using Letta. You’ll learn how to upload PDF documents to the Letta Filesystem, attach them to an agent, and query the agent about the content. Letta automatically extracts text from PDFs using OCR, making the content accessible to your agents.

By the end of this guide, you’ll understand how to create document analysis workflows where agents can read, understand, and answer questions about PDF files.

This tutorial uses Letta Cloud. Generate an API key at app.letta.com/api-keys and set it as LETTA_API_KEY in your environment. Self-hosted servers only need an API key if authentication is enabled. You can learn more about self-hosting here.

What You’ll Learn

  • Creating folders to organize documents
  • Uploading PDF files to Letta
  • Creating agents configured for document analysis
  • Attaching folders to give agents access to files
  • Querying agents about PDF content
  • Understanding how Letta processes PDFs

Prerequisites

Install the required dependencies:

1npm install @letta-ai/letta-client

Steps

Step 1: Initialize Client

1import { LettaClient } from '@letta-ai/letta-client';
2
3// Initialize the Letta client using LETTA_API_KEY environment variable
4const client = new LettaClient({ token: process.env.LETTA_API_KEY });
5
6// If self-hosting, specify the base URL:
7// const client = new LettaClient({ baseUrl: "http://localhost:8283" });

Step 2: Create a Folder for PDFs

Folders in the Letta Filesystem organize files and make them accessible to agents. Create a folder specifically for storing PDF documents:

1// Create a folder to store PDF documents (or use existing one)
2// API Reference: https://docs.letta.com/api-reference/folders/create
3let folderId: string;
4try {
5 // Try to retrieve existing folder by name
6 folderId = await client.folders.retrieveByName("PDF Documents");
7 console.log(`Using existing folder: ${folderId}\n`);
8} catch (error: any) {
9 // If folder doesn't exist (404), create it
10 if (error.statusCode === 404) {
11 const folder = await client.folders.create({
12 name: "PDF Documents",
13 description: "A folder containing PDF files for the agent to read",
14 });
15 folderId = folder.id;
16 console.log(`Created folder: ${folderId}\n`);
17 } else {
18 throw error;
19 }
20}
Created folder: folder-a1b2c3d4-e5f6-7890-abcd-ef1234567890

If the folder already exists, you’ll see:

Using existing folder: folder-a1b2c3d4-e5f6-7890-abcd-ef1234567890

Step 3: Download and Upload a PDF

Let’s download a sample PDF (the MemGPT research paper) and upload it to the folder. Letta will automatically extract the text content using OCR.

1import * as fs from 'fs';
2import * as https from 'https';
3
4// Download the PDF if it doesn't exist locally
5const pdfFilename = "memgpt.pdf";
6
7if (!fs.existsSync(pdfFilename)) {
8 console.log(`Downloading ${pdfFilename}...`);
9
10 await new Promise<void>((resolve, reject) => {
11 const file = fs.createWriteStream(pdfFilename);
12 https.get("https://arxiv.org/pdf/2310.08560", (response) => {
13 response.pipe(file);
14 file.on('finish', () => {
15 file.close();
16 console.log("Download complete\n");
17 resolve();
18 });
19 file.on('error', reject);
20 }).on('error', reject);
21 });
22}
23
24// Upload the PDF to the folder
25// API Reference: https://docs.letta.com/api-reference/folders/files/upload
26const uploadedFile = await client.folders.files.upload(
27 fs.createReadStream(pdfFilename),
28 folderId,
29 { duplicateHandling: "skip" }
30);
31
32console.log(`Uploaded PDF: ${uploadedFile.id}\n`);
Downloading memgpt.pdf...
Download complete
Uploaded PDF: file-a1b2c3d4-e5f6-7890-abcd-ef1234567890

PDF Processing: Letta extracts text from PDFs using OCR automatically during upload. The extracted text becomes searchable and accessible to agents attached to the folder.

Step 4: Create an Agent for Document Analysis

Create an agent with a persona configured for analyzing documents. The agent’s memory blocks define its purpose and capabilities:

1// Create an agent configured to analyze documents
2// API Reference: https://docs.letta.com/api-reference/agents/create
3const agent = await client.agents.create({
4 name: "pdf_assistant",
5 model: "openai/gpt-4o-mini",
6 memoryBlocks: [
7 {
8 label: "persona",
9 value: "I am a helpful research assistant that analyzes PDF documents and answers questions about their content."
10 },
11 {
12 label: "human",
13 value: "Name: User\nTask: Analyzing PDF documents"
14 }
15 ],
16});
17
18console.log(`Created agent: ${agent.id}\n`);
Created agent: agent-a1b2c3d4-e5f6-7890-abcd-ef1234567890

Step 5: Attach the Folder to the Agent

Attach the folder containing the PDF to the agent. This gives the agent the ability to search through all files in the folder:

1// Attach the folder to the agent
2// API Reference: https://docs.letta.com/api-reference/agents/folders/attach
3await client.agents.folders.attach(agent.id, folderId);
4
5console.log(`Attached folder to agent\n`);
Attached folder to agent

Once a folder is attached, the agent can use search tools to retrieve relevant content from files in the folder. Learn more in the Letta Filesystem guide.

Step 6: Query the PDF Content

Now ask the agent questions about the PDF. The agent will search through the document content to find relevant information:

1// Ask the agent to summarize the PDF
2// API Reference: https://docs.letta.com/api-reference/agents/messages/create
3const response = await client.agents.messages.create(agent.id, {
4 messages: [{
5 role: "user",
6 content: "Can you summarize the main ideas from the MemGPT paper?"
7 }]
8});
9
10for (const msg of response.messages) {
11 if (msg.messageType === "assistant_message") {
12 console.log(`Assistant: ${msg.content}\n`);
13 }
14}
Assistant: The MemGPT paper introduces a system that enables LLMs to manage their own
memory hierarchy, similar to how operating systems manage memory. It addresses the limited
context window problem in large language models by introducing a memory management system
inspired by traditional operating systems. The key innovation is allowing LLMs to explicitly
move information between main context (limited) and external storage (unlimited), enabling
extended conversations and document analysis that exceed typical context limits.

Step 7: Ask Specific Questions

You can continue the conversation to ask more specific questions about the document:

1// Ask a specific question about the PDF content
2const response2 = await client.agents.messages.create(agent.id, {
3 messages: [{
4 role: "user",
5 content: "What problem does MemGPT solve?"
6 }]
7});
8
9for (const msg of response2.messages) {
10 if (msg.messageType === "assistant_message") {
11 console.log(`Assistant: ${msg.content}\n`);
12 }
13}
Assistant: MemGPT addresses the limited context window problem in large language models.
Traditional LLMs can only process a fixed amount of text at once (their context window),
which makes it difficult to maintain long conversations or analyze large documents. MemGPT
solves this by introducing a memory management system that allows the model to intelligently
move information between its limited context and unlimited external storage, enabling
extended conversations and document analysis beyond typical context limits.

Complete Example

Here’s the full code in one place that you can run:

1import { LettaClient } from '@letta-ai/letta-client';
2import * as fs from 'fs';
3import * as https from 'https';
4
5async function main() {
6 // Initialize client
7 const client = new LettaClient({ token: process.env.LETTA_API_KEY });
8
9 // Create folder (or use existing one)
10 let folderId: string;
11 try {
12 folderId = await client.folders.retrieveByName("PDF Documents");
13 console.log(`Using existing folder: ${folderId}\n`);
14 } catch (error: any) {
15 if (error.statusCode === 404) {
16 const folder = await client.folders.create({
17 name: "PDF Documents",
18 description: "A folder containing PDF files for the agent to read",
19 });
20 folderId = folder.id;
21 console.log(`Created folder: ${folderId}\n`);
22 } else {
23 throw error;
24 }
25 }
26
27 // Download and upload PDF
28 const pdfFilename = "memgpt.pdf";
29
30 if (!fs.existsSync(pdfFilename)) {
31 console.log(`Downloading ${pdfFilename}...`);
32 await new Promise<void>((resolve, reject) => {
33 const file = fs.createWriteStream(pdfFilename);
34 https.get("https://arxiv.org/pdf/2310.08560", (response) => {
35 response.pipe(file);
36 file.on('finish', () => {
37 file.close();
38 console.log("Download complete\n");
39 resolve();
40 });
41 file.on('error', reject);
42 }).on('error', reject);
43 });
44 }
45
46 const uploadedFile = await client.folders.files.upload(
47 fs.createReadStream(pdfFilename),
48 folderId,
49 { duplicateHandling: "skip" }
50 );
51
52 console.log(`Uploaded PDF: ${uploadedFile.id}\n`);
53
54 // Create agent
55 const agent = await client.agents.create({
56 name: "pdf_assistant",
57 model: "openai/gpt-4o-mini",
58 memoryBlocks: [
59 {
60 label: "persona",
61 value: "I am a helpful research assistant that analyzes PDF documents and answers questions about their content."
62 },
63 {
64 label: "human",
65 value: "Name: User\nTask: Analyzing PDF documents"
66 }
67 ],
68 });
69
70 console.log(`Created agent: ${agent.id}\n`);
71
72 // Attach folder to agent
73 await client.agents.folders.attach(agent.id, folderId);
74
75 console.log(`Attached folder to agent\n`);
76
77 // Query the PDF
78 const response = await client.agents.messages.create(agent.id, {
79 messages: [{
80 role: "user",
81 content: "Can you summarize the main ideas from the MemGPT paper?"
82 }]
83 });
84
85 for (const msg of response.messages) {
86 if (msg.messageType === "assistant_message") {
87 console.log(`Assistant: ${msg.content}\n`);
88 }
89 }
90
91 // Ask specific question
92 const response2 = await client.agents.messages.create(agent.id, {
93 messages: [{
94 role: "user",
95 content: "What problem does MemGPT solve?"
96 }]
97 });
98
99 for (const msg of response2.messages) {
100 if (msg.messageType === "assistant_message") {
101 console.log(`Assistant: ${msg.content}\n`);
102 }
103 }
104}
105
106main();

Key Concepts

Folder Organization

Folders in the Letta Filesystem organize and group files, making them easy to manage and attach to agents

Automatic OCR

PDFs are automatically processed using OCR to extract searchable text content during upload

Document Access

Attaching folders gives agents search capabilities to retrieve relevant content from files

Contextual Search

Agents use search tools to find relevant passages in documents when answering questions

Use Cases

Upload academic papers and have agents summarize findings, extract key concepts, or compare methodologies.

Build customer support systems that answer questions based on product documentation or manuals.

Process multiple PDFs to build a searchable knowledge base that agents can query for information.

Next Steps