Other¶
HolmesGPT supports all AI providers available through LiteLLM, including 100+ different LLM providers. This guide shows how to configure any LiteLLM-supported provider using DeepSeek via Novita as an example.
Example: DeepSeek 3.1 Terminus via Novita¶
Let's walk through setting up DeepSeek 3.1 Terminus using the Novita AI provider.
Step 1: Check LiteLLM Documentation¶
First, visit the LiteLLM Novita documentation to understand the provider requirements:
- Model naming format:
novita/<model-name> - Required environment variable:
NOVITA_API_KEY
Step 2: Get Your API Key¶
Sign up at Novita AI to obtain your API key.
Step 3: Configure HolmesGPT¶
Context window
LiteLLM's model registry lists this model with a 131,072-token context window, but Novita serves it with a smaller one, so the configuration sets it:
- Context window: 98,304 tokens
- Max output: 32,768 tokens, the registry's value, which Holmes uses without configuration
Option 1: API Key Kept Out of the Configuration (Recommended)¶
Set the API key:
Use the model, with its 98k context window:
OVERRIDE_MAX_CONTENT_SIZE applies to every model that sets no max_context_size, so set it only on the command that uses this one.
OVERRIDE_MAX_CONTENT_SIZE=98304 \
holmes ask "what pods are failing?" --model="novita/deepseek/deepseek-v3.1-terminus"
Or pass the API key directly:
Create a Kubernetes secret in the namespace Holmes runs in:
kubectl create secret generic holmes-other \
--from-literal=NOVITA_API_KEY="your-novita-api-key" \
-n <namespace>
When using the standalone Holmes Helm Chart, merge this into your values.yaml, appending list entries (such as additionalEnvVars) to the lists already there:
extraEnvVarsSecrets:
- holmes-other
modelList:
deepseek-terminus:
model: novita/deepseek/deepseek-v3.1-terminus
api_key: "{{ env.NOVITA_API_KEY }}"
custom_args:
max_context_size: 98304 # Override context window to 98k tokens
Apply the configuration:
Create a Kubernetes secret in the namespace Holmes runs in:
kubectl create secret generic holmes-other \
--from-literal=NOVITA_API_KEY="your-novita-api-key" \
-n <namespace>
When using the Robusta Helm Chart (which includes HolmesGPT), merge the keys under holmes: into the holmes: key of your generated_values.yaml (add one if it has none), appending list entries (such as additionalEnvVars) to the lists already there:
holmes:
extraEnvVarsSecrets:
- holmes-other
modelList:
deepseek-terminus:
model: novita/deepseek/deepseek-v3.1-terminus
api_key: "{{ env.NOVITA_API_KEY }}"
custom_args:
max_context_size: 98304 # Override context window to 98k tokens
Apply the configuration:
Option 2: API Key in the Configuration (Less Secure)¶
Warning
This method stores the API key in plain text in your configuration. Use Kubernetes Secrets for production environments.
When using the standalone Holmes Helm Chart, merge this into your values.yaml, appending list entries (such as additionalEnvVars) to the lists already there:
additionalEnvVars:
- name: NOVITA_API_KEY
value: "your-novita-api-key" # API key directly in values file
modelList:
deepseek-terminus:
model: novita/deepseek/deepseek-v3.1-terminus
api_key: "{{ env.NOVITA_API_KEY }}"
custom_args:
max_context_size: 98304 # Override context window to 98k tokens
Apply the configuration:
When using the Robusta Helm Chart (which includes HolmesGPT), merge the keys under holmes: into the holmes: key of your generated_values.yaml (add one if it has none), appending list entries (such as additionalEnvVars) to the lists already there:
holmes:
additionalEnvVars:
- name: NOVITA_API_KEY
value: "your-novita-api-key" # API key directly in values file
modelList:
deepseek-terminus:
model: novita/deepseek/deepseek-v3.1-terminus
api_key: "{{ env.NOVITA_API_KEY }}"
custom_args:
max_context_size: 98304 # Override context window to 98k tokens
Apply the configuration:
General Model Configuration Parameters¶
When configuring models in modelList, you can use these parameters:
Core Parameters:
modelList:
my-model:
# Required: LiteLLM model identifier
model: "provider/model-name"
# Optional: Authentication
api_key: "{{ env.API_KEY }}" # Supports environment variables
# Optional: Custom endpoints
api_base: "https://custom-endpoint.example.com/v1" # Custom API URL
api_version: "2024-01-01" # API version (e.g., for Azure)
Advanced Parameters:
modelList:
advanced-model:
model: "provider/model-name"
api_key: "{{ env.API_KEY }}"
# GPT-5 specific
reasoning_effort: "medium" # Options: minimal, low, medium, high
# Thinking mode (experimental, model-specific)
thinking: true # Enable thinking mode if supported by the model
# Override context window size for any model
custom_args:
max_context_size: 1000000 # Override the model's context window
Context Window Override
The custom_args.max_context_size parameter can be used with any model to override its default context window size. This is particularly useful when the model's actual context window is different from what LiteLLM reports.
This parameter takes precedence over the OVERRIDE_MAX_CONTENT_SIZE environment variable, giving you per-model control when using modelList configuration.
Example: If your model isn't recognized by LiteLLM and defaults to 200k tokens, but it actually supports 1M tokens:
Parameter Passthrough
Any additional parameters not listed here (like temperature, max_tokens, etc.) are passed directly to the LiteLLM completion API. Refer to your provider's documentation for supported parameters.
Finding Other Providers¶
To use any other LiteLLM-supported provider, follow the same pattern:
- Check the provider's page in LiteLLM documentation
- Note the required environment variables and model naming format
- Configure using the same structure shown above
Additional Resources¶
- LiteLLM Providers: Full list at LiteLLM Providers