Model providers

Trying Letta for the first time? Follow our Letta Code or API quickstart to learn the basics first. Docker requires additional setup and debugging work.

This page is for Docker setups only. The Letta API is pre-configured and doesn’t require provider setup to access frontier models - simply select from the dropdown in Letta Code (/model), or in the ADE.

To connect your own API keys to Letta Code, use /connect in the terminal.

This page shows how to configure different LLM providers when running the Letta server with Docker. Set the appropriate environment variables when starting your container.

Cloud providers

OpenAI

docker run \
  -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
  -p 8283:8283 \
  -e OPENAI_API_KEY="your_openai_api_key" \
  letta/letta:latest

Anthropic

docker run \
  -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
  -p 8283:8283 \
  -e ANTHROPIC_API_KEY="your_anthropic_api_key" \
  letta/letta:latest

Anthropic does not provide embedding models. Use OpenAI for embeddings.

Google AI (Gemini)

docker run \
  -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
  -p 8283:8283 \
  -e GEMINI_API_KEY="your_gemini_api_key" \
  letta/letta:latest

OpenRouter

docker run \
  -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
  -p 8283:8283 \
  -e OPENROUTER_API_KEY="your_openrouter_api_key" \
  letta/letta:latest

Together AI

docker run \
  -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
  -p 8283:8283 \
  -e TOGETHER_API_KEY="your_together_api_key" \
  letta/letta:latest

Enterprise cloud

Azure OpenAI

docker run \
  -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
  -p 8283:8283 \
  -e AZURE_API_KEY="your_azure_api_key" \
  -e AZURE_BASE_URL="your_azure_base_url" \
  -e AZURE_API_VERSION="2024-09-01-preview" \
  letta/letta:latest

AWS Bedrock

docker run \
  -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
  -p 8283:8283 \
  -e AWS_ACCESS_KEY_ID="your_aws_access_key_id" \
  -e AWS_SECRET_ACCESS_KEY="your_aws_secret_access_key" \
  -e AWS_DEFAULT_REGION="your_aws_default_region" \
  letta/letta:latest

To use a model with AWS Bedrock, you must ensure it is enabled in your AWS Model Catalog.

Google Vertex AI

First authenticate with Google Cloud:

gcloud auth application-default login

Then run the container:

docker run \
  -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
  -p 8283:8283 \
  -e GOOGLE_CLOUD_PROJECT="your-project-id" \
  -e GOOGLE_CLOUD_LOCATION="us-central1" \
  letta/letta:latest

Local models

Ollama

Always use tags when downloading Ollama models (e.g., ollama pull dolphin2.2-mistral:7b-q6_K). Avoid compression below Q5.

macOS/Windows (use host.docker.internal):

docker run \
  -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
  -p 8283:8283 \
  -e OLLAMA_BASE_URL="http://host.docker.internal:11434/v1" \
  letta/letta:latest

Linux (use --network host):

docker run \
  -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
  --network host \
  -e OLLAMA_BASE_URL="http://localhost:11434/v1" \
  letta/letta:latest

LM Studio

Models marked as “native tool use” on LM Studio work best with Letta.

macOS/Windows (use host.docker.internal):

docker run \
  -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
  -p 8283:8283 \
  -e LMSTUDIO_BASE_URL="http://host.docker.internal:1234" \
  letta/letta:latest

Linux (use --network host):

docker run \
  -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
  --network host \
  -e LMSTUDIO_BASE_URL="http://localhost:1234" \
  letta/letta:latest

Multiple providers

You can enable multiple providers by setting multiple environment variables:

docker run \
  -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
  -p 8283:8283 \
  -e OPENAI_API_KEY="your_openai_api_key" \
  -e ANTHROPIC_API_KEY="your_anthropic_api_key" \
  -e OLLAMA_BASE_URL="http://host.docker.internal:11434/v1" \
  letta/letta:latest