Anthropic — Letta

To enable Anthropic models with Letta, set ANTHROPIC_API_KEY in your environment variables.

You can use Letta with Anthropic if you have an Anthropic account and API key. Currently, we support the following models:

claude-3-opus-20240229
claude-3-sonnet-20240229
claude-3-haiku-20240307

Currently, only there are no supported embedding models for Anthropic. You will need to use a seperate provider (e.g. OpenAI) or the Letta embeddings endpoint (letta-free) for embeddings.

Enabling Anthropic models

To enable the Anthropic provider, set your key as an environment variable:

$ export ANTHROPIC_API_KEY="sk-ant-..."

Now, Anthropic models will be enabled with you run letta run or the letta service.

Using the `docker run` server with Anthropic

To enable Anthropic models, simply set your ANTHROPIC_API_KEY as an environment variable:

$ # replace `~/.letta/.persist/pgdata` with wherever you want to store your agent data
> docker run \
>   -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
>   -p 8283:8283 \
>   -e ANTHROPIC_API_KEY="your_anthropic_api_key" \
>   letta/letta:latest

CLI (pypi only)

Using `letta run` and `letta server` with Anthropic

To chat with an agent, run:

$ export ANTHROPIC_API_KEY="sk-ant-..."
> letta run

This will prompt you to select an Anthropic model.

? Select LLM model: (Use arrow keys)
 » letta-free [type=openai] [ip=https://inference.memgpt.ai]
   claude-3-opus-20240229 [type=anthropic] [ip=https://api.anthropic.com/v1]
   claude-3-sonnet-20240229 [type=anthropic] [ip=https://api.anthropic.com/v1]
   claude-3-haiku-20240307 [type=anthropic] [ip=https://api.anthropic.com/v1]

To run the Letta server, run:

$ export ANTHROPIC_API_KEY="sk-ant-..."
> letta server

To select the model used by the server, use the dropdown in the ADE or specify a LLMConfig object in the Python SDK.

Configuring Anthropic models

When creating agents, you can specify the exactly model configurations to use, such as the model name and context window size (which can be less than the maximum size).

To set a default LLMConfig to claude-3-opus-20240229 with a context window of 200000, you can set the default config for your client:

1 from letta import create_client, LLMConfig
2 
3 client = create_client()
4 
5 # provide full config (llm config)
6 client.set_default_llm_config(
7     LLMConfig(
8         model="claude-3-opus-20240229",
9         model_endpoint_type="anthropic",
10         model_endpoint="https://api.anthropic.com/v1",
11         context_window=200000 # NOTE: can be set to <= 200000
12     )
13 )

$	# replace `~/.letta/.persist/pgdata` with wherever you want to store your agent data
>	docker run \
>	-v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
>	-p 8283:8283 \
>	-e ANTHROPIC_API_KEY="your_anthropic_api_key" \
>	letta/letta:latest

1	from letta import create_client, LLMConfig
2
3	client = create_client()
4
5	# provide full config (llm config)
6	client.set_default_llm_config(
7	LLMConfig(
8	model="claude-3-opus-20240229",
9	model_endpoint_type="anthropic",
10	model_endpoint="https://api.anthropic.com/v1",
11	context_window=200000 # NOTE: can be set to <= 200000
12	)
13	)