Models

List LLM Models

List available LLM models using the asynchronous implementation for improved performance

Headers

AuthorizationstringRequired

Header authentication of the form Bearer <token>

Query parameters

provider_categorylist of enumsOptional
Allowed values:
provider_namestringOptional
provider_typeenumOptional

Response

Successful Response
modelstring
LLM model name.
model_endpoint_typeenum
The endpoint type for the model.
context_windowinteger
The context window size for the model.
model_endpointstring or null
The endpoint for the model.
provider_namestring or null
The provider name for the model.
provider_categoryenum or null
The provider category for the model.
Allowed values:
model_wrapperstring or null
The wrapper for the model.
put_inner_thoughts_in_kwargsboolean or nullDefaults to true

Puts ‘inner_thoughts’ as a kwarg in the function call if this is set to True. This helps with function calling performance and also the generation of inner thoughts.

handlestring or null

The handle for this config, in the format provider/model-name.

temperatureinteger or nullDefaults to 0.7
The temperature to use when generating text with the model. A higher temperature will result in more random text.
max_tokensinteger or nullDefaults to 4096
The maximum number of tokens to generate. If not set, the model will use its default value.
enable_reasonerboolean or nullDefaults to false
Whether or not the model should use extended thinking if it is a 'reasoning' style model
reasoning_effortenum or null
The reasoning effort to use when generating text reasoning models
Allowed values:
max_reasoning_tokensinteger or nullDefaults to 0

Configurable thinking budget for extended thinking, only used if enable_reasoner is True. Minimum value is 1024.