OpenAI

You can use Letta with OpenAI if you have an OpenAI account and API key. Once you have set your OPENAI_API_KEY in your environment variables, you can select what model and configure the context window size.

Currently, Letta supports the following OpenAI models:

• gpt-4 (recommended for advanced reasoning)
• gpt-4o-mini (recommended for low latency and cost)
• gpt-4o
• gpt-4-turbo (not recommended, should use gpt-4o-mini instead)
• gpt-3.5-turbo (not recommended, should use gpt-4o-mini instead)

Enabling OpenAI models with Docker

To enable OpenAI models when running the Letta server with Docker, set your OPENAI_API_KEY as an environment variable:

$
# replace `~/.letta/.persist/pgdata` with wherever you want to store your agent data
>
docker run \
>
  -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
>
  -p 8283:8283 \
>
  -e OPENAI_API_KEY="your_openai_api_key" \
>
  letta/letta:latest

See the self-hosting guide for more information on running Letta with Docker.

Specifying agent models

When creating agents on your self-hosted server, you must specify both the LLM and embedding models to use via a handle. You can additionally specify a context window limit (which must be less than or equal to the maximum size).

1
from letta_client import Letta
2
import os
3
4
# Connect to your self-hosted server
5
client = Letta(base_url="http://localhost:8283")
6
7
openai_agent = client.agents.create(
8
    model="openai/gpt-4o-mini",
9
    embedding="openai/text-embedding-3-small",  # An embedding model is required for self-hosted
10
    # optional configuration
11
    context_window_limit=16000
12
)