models

List LLM Models

List available LLM models using the asynchronous implementation for improved performance

Headers

AuthorizationstringRequired

Header authentication of the form Bearer <token>

Query parameters

provider_categorylist of enums or nullOptional
Allowed values:
provider_namestring or nullOptional
provider_typeenum or nullOptional

Response

Successful Response
modelstring
LLM model name.
model_endpoint_typeenum
The endpoint type for the model.
context_windowinteger
The context window size for the model.
model_endpointstring or null
The endpoint for the model.
provider_namestring or null
The provider name for the model.
provider_categoryenum or null
The provider category for the model.
Allowed values:
model_wrapperstring or null
The wrapper for the model.
put_inner_thoughts_in_kwargsboolean or nullDefaults to true

Puts ‘inner_thoughts’ as a kwarg in the function call if this is set to True. This helps with function calling performance and also the generation of inner thoughts.

handlestring or null

The handle for this config, in the format provider/model-name.

temperaturedouble or nullDefaults to 0.7
The temperature to use when generating text with the model. A higher temperature will result in more random text.
max_tokensinteger or null
The maximum number of tokens to generate. If not set, the model will use its default value.
enable_reasonerboolean or nullDefaults to true
Whether or not the model should use extended thinking if it is a 'reasoning' style model
reasoning_effortenum or null
The reasoning effort to use when generating text reasoning models
Allowed values:
max_reasoning_tokensinteger or nullDefaults to 0

Configurable thinking budget for extended thinking. Used for enable_reasoner and also for Google Vertex models like Gemini 2.5 Flash. Minimum value is 1024 when used with enable_reasoner.

frequency_penaltydouble or null

Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model’s likelihood to repeat the same line verbatim. From OpenAI: Number between -2.0 and 2.0.

compatibility_typeenum or null
The framework compatibility type for the model.
Allowed values:
verbosityenum or null

Soft control for how verbose model output should be, used for GPT-5 models.

Allowed values:
tierstring or null

The cost tier for the model (cloud only).

Errors