Models
Configure models in the Letta app or CLI
Letta agents are model-agnostic. Agents can use Claude, GPT, Gemini, or other supported models. Users can change an agent’s underlying model at any time, even mid-conversation.
We recommend first-time users use a large frontier model (for example from OpenAI or Anthropic) to learn how Letta agents work, since weaker models can cause the agent to behave in unexpected ways. See leaderboard.letta.com for benchmarks comparing model performance.
Switching models
Section titled “Switching models”Use the /model command to switch models during a session. This opens a model selector that allows you to swap models and choose their reasoning settings (if applicable).
Model toolsets
Section titled “Model toolsets”Different models are trained to work best with different tools. For example, GPT models from OpenAI are trained to use a patch-based editing tool, while Claude models work better with string-based edit tools.
The Letta agent harness handles this automatically with dynamic toolsets - tool configurations optimized for each model family. When you switch models with /model, the toolset switches automatically. If you want to force a specific toolset, use the /toolset command.
Connecting model providers
Section titled “Connecting model providers”The Letta app and CLI support a wide variety of LLM providers, as well as coding plans such as ChatGPT and zAI.
To connect additional providers, use /connect in the CLI.
To see your available models, use /model.
Local providers are configured on your machine. When you sign in with Letta, /connect also lets you configure providers for agents backed up to the cloud.
Cloud providers are managed through Letta’s LLM gateway. The gateway supports a smaller provider set than local setup; direct integrations with local inference providers such as LM Studio, Ollama, and llama.cpp are available only for local agents. You can pay for LLM gateway usage with pay-as-you-go credits from the usage page.
Local models
Section titled “Local models”There are two ways to use your own locally hosted models with Letta agents:
- Run the agent and model locally. Use local setup, open
/connect, and choose the built-in Ollama or LM Studio provider. Both the agent state and model endpoint remain on your machine. - Connect a cloud-hosted agent to your model. Open
/connect, add an OpenAI-compatible API custom provider, and enter the base URL and API key for your inference server. Because the agent runs in the cloud, the endpoint must be reachable from the public internet rather than only throughlocalhost. Use a stable public URL or static IP, secure it with authentication and HTTPS, and restrict access where possible.
The OpenAI-compatible endpoint must support Chat Completions and tool calling. After connecting either kind of provider, use /model to select the model.
| Provider | Local support | Signed in with Letta |
|---|---|---|
| Amazon Bedrock | AWS credentials | AWS credentials |
| Anthropic API | API key | API key |
| Claude Pro/Max | Subscription | — |
| Azure OpenAI Responses | API key | — |
| Cerebras | API key | — |
| ChatGPT Plus/Pro Codex | Subscription | Subscription |
| Cloudflare AI Gateway | API key | — |
| Cloudflare Workers AI | API key | — |
| DeepSeek | API key | — |
| Fireworks | API key | — |
| GitHub Copilot | Subscription | — |
| Google Gemini | API key | API key |
| Google Vertex AI | API key | — |
| Groq | API key | — |
| Hugging Face | API key | — |
| Kimi Code | API key | API key |
| llama.cpp | Local endpoint | — |
| LM Studio | Local endpoint | — |
| MiniMax | API key or coding plan | API key or coding plan |
| MiniMax (China) | API key | — |
| Mistral | API key | — |
| Moonshot AI | API key | API key |
| Moonshot AI (China) | API key | — |
| Ollama | Local endpoint | — |
| Ollama Cloud | API key | — |
| OpenAI API | API key | API key |
| OpenAI-compatible API | API key + base URL | API key + base URL |
| OpenCode Go | API key | — |
| OpenCode Zen | API key | — |
| OpenRouter | API key | API key |
| Together AI | API key | — |
| Vercel AI Gateway | API key | — |
| xAI | API key | — |
| Xiaomi MiMo | API key | — |
| Xiaomi MiMo Token Plan (Amsterdam) | Token plan | — |
| Xiaomi MiMo Token Plan (China) | Token plan | — |
| Xiaomi MiMo Token Plan (Singapore) | Token plan | — |
| zAI API | API key | API key |
| zAI Coding Plan | Coding plan | Coding plan |