The Model Providers tab lets you connect your own AI model providers to use alongside the platform's built-in models. Once connected, these models appear in Model Routing configuration and can be assigned to specific routing rules or used in the sandbox for testing.
The Model Providers tab shows connected providers with their status and enabled models. The Beta badge indicates this feature is in preview.
Supported Provider Types
| Provider Type | Credential Fields | Configuration Fields |
|---|---|---|
| OpenAI | API Key | — |
| Anthropic | API Key | — |
| Azure OpenAI | API Key | Base URL, API Version |
| AWS Bedrock | AWS Access Key ID, AWS Secret Access Key | Region |
| GCP Vertex AI | Service Account JSON | Project, Region |
| Custom (LiteLLM) | API Key | Base URL |
Adding a Provider
Open the Model Providers Tab
Click the Model Providers tab at the top of the Settings page. The tab has a Beta badge.
Click Add Provider
Click Add Provider in the top-right corner. The Add Model Provider dialog opens.
The dialog adapts its credential and configuration fields based on the selected provider type.
Enter Provider Details
Fill in the required fields:
- Provider Name — a display label for this provider (for example, "Production OpenAI" or "Internal Azure GPT-4")
- Provider Type — select from the dropdown (OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, GCP Vertex AI, or Custom)
- Credentials — enter the required credentials for the selected provider type. Credentials are encrypted and stored securely — they are never exposed in API responses.
If your provider type requires additional configuration (such as a Base URL for Azure OpenAI or a Region for AWS Bedrock), those fields appear automatically.
Add Models
Click Add Model in the Models section. For each model, enter:
- Model ID — the provider's model identifier (for example,
gpt-4-turboorclaude-3-5-sonnet-20241022) - Display Name — a human-readable label shown in routing configuration
- Supported Stages — select which pipeline stages this model can handle: planning, tool call, summarization, or guardrail
Click Add to save the model. You can add multiple models to a single provider.
Create the Provider
Click Create Provider to save. The provider appears as a card on the Model Providers tab with its status and model count.
Provider Card
Each connected provider displays as a card showing:
| Element | Description |
|---|---|
| Provider icon and name | Visual identifier with the provider type label below |
| Status chip | Active (green), Inactive (grey), or Error (red) |
| Model count | Number of enabled models on this provider |
| Edit button | Opens the edit dialog to update settings or credentials |
| Delete button | Removes the provider and its stored credentials |
| Model list | Each enabled model with its display name, supported stages, and a play button to test |
Editing a Provider
Click the edit icon on a provider card to open the edit dialog. You can update:
- Provider Name — change the display label
- Status — toggle between Active and Inactive using the switch
- Credentials — leave blank to keep current values, or enter new credentials to replace them. Click the eye icon to reveal a stored credential value on demand.
- Configuration — update Base URL, API Version, Region, or Project as applicable
- Models — add new models or remove existing ones
The provider type cannot be changed after creation. To switch provider types, delete the existing provider and create a new one.
Testing a Model
Click the play icon next to any model in a provider card to open the test drawer. Enter a prompt and click Send Test to verify the model responds correctly. The result shows:
- Success — the model's response text and latency in milliseconds
- Failed — the error message from the provider
Credentials are encrypted and stored securely — they are never returned in API responses. When editing a provider, existing credentials appear as masked dots. You can reveal a stored value by clicking the eye icon.
Related Pages
- Model Routing — configure which models handle different query types using your connected providers
- Confidence Thresholds — set the score boundaries that drive routing and escalation
- Sandbox — test AI responses with different model configurations