Connect model providers

A provider is an upstream API endpoint, its credential, and the models it serves. Declare providers in providers and store credentials in Kubernetes Secrets.

For the legacy AI Gateway, see legacy gateway provider setup.

Supported providers

The provider kind decides which client APIs serve its models. The gateway doesn't translate between APIs, so a model is reachable only on the APIs its provider speaks.

Provider	`kind`	Client APIs	Credential
OpenAI	`openai`	Chat Completions, Responses	API key
OpenRouter	`openrouter`	Chat Completions, Responses	API key
Anthropic	`anthropic`	Anthropic Messages	API key
Claude on AWS Bedrock	`bedrock-anthropic`	Anthropic Messages	EKS Pod Identity, or an AWS credentials file
AWS Bedrock OpenAI-compatible API	`bedrock-openai`	Chat Completions, Responses	EKS Pod Identity, or an AWS credentials file
Google AI Studio	`gemini-aistudio`	Gemini `generateContent` and streaming	API key
Google Vertex AI	`vertex`	Gemini `generateContent` and streaming	Service account key

bedrock-anthropic calls the Bedrock InvokeModel API with Anthropic's native request format and requires region. bedrock-openai calls Bedrock's OpenAI-compatible API with the OpenAI request format and requires region. vertex uses the Vertex Gemini API and requires project and region.

Each provider needs kind, baseUrl, and at least one model. The chart refuses to render a provider that is missing one of them or uses an unknown kind.

Audit events identify a provider by its kind.

Add a provider

Create the credential Secret in the platform namespace:

kubectl create secret generic ai-gateway-v2-openai \
  -n <NAMESPACE> \
  --from-literal=api-key='<OPENAI_API_KEY>'

Declare the provider and the models it serves. credentialSecretRef names the Secret and the data key that holds the credential:
values.yaml
```
ai-gateway-v2:
  providers:
    - kind: openai
      baseUrl: https://api.openai.com
      models: ['gpt-5', 'gpt-5-mini']
      credentialSecretRef:
        name: ai-gateway-v2-openai
        key: api-key
```
The models list is the route table. See Route models to providers for how the gateway matches a request to a provider.
Confirm that each model has a price in the pricing catalog. The gateway refuses requests for a model with no catalog entry, even when the provider credential works. Look up the entry by its catalog provider and model:
```
curl -s <ENTERPRISE_MANAGER_URL>/v1/budgets/pricing \
  -H "Authorization: Bearer <ADMIN_TOKEN>" \
  | jq '.spec.entries[] | select(.provider == "openai" and .model == "gpt-5-mini")'
```
Replace <ENTERPRISE_MANAGER_URL> with the Enterprise Manager API base URL and <ADMIN_TOKEN> with a platform administrator's access token. An entry with a rate block, such as openai, means the model is priced. If the command prints nothing, or the entry has "unpriced": true, price the model first. See Update the pricing catalog for the request, and Match provider kinds to catalog entries for the catalog provider name of each kind.
Upgrade the release and wait for the rollout to finish:
```
helm upgrade <RELEASE_NAME> <CHART> -n <NAMESPACE> -f values.yaml
kubectl rollout status deployment/<RELEASE_NAME>-ai-gateway-v2 -n <NAMESPACE>
```
If the rollout stalls, the gateway rejected the configuration and the previous pods keep serving. See How configuration changes apply.
Send a request through the gateway to confirm the credential works end to end. Send it as a caller with an applicable budget and, if model access is on, a grant for the model:
```
curl -s https://<AI_GATEWAY_HOST>/v1/chat/completions \
  -H "Authorization: Bearer <ACCESS_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "ping"}]}'
```
A completion confirms the credential, route, and price. A 402 means the caller has no budget with capacity, and a 403 means the caller has no grant for the model.

Provider specifics

Anthropic

ai-gateway-v2:
  providers:
    - kind: anthropic
      baseUrl: https://api.anthropic.com
      models: ['claude-sonnet-5', 'claude-haiku-4-5']
      credentialSecretRef:
        name: ai-gateway-v2-anthropic
        key: api-key

Claude on AWS Bedrock

Without a credentialSecretRef, the gateway resolves AWS credentials from the default AWS credential chain. Associate an IAM role that can invoke the Bedrock models with the gateway's ServiceAccount through EKS Pod Identity.

ai-gateway-v2:
  providers:
    - kind: bedrock-anthropic
      baseUrl: https://bedrock-runtime.us-east-1.amazonaws.com
      region: us-east-1
      models: ['us.anthropic.claude-sonnet-5']

To use long-lived keys instead, store an AWS shared credentials file in a Secret, reference it with credentialSecretRef, and optionally select a profile with awsProfile. The profile defaults to default.

kubectl create secret generic ai-gateway-v2-bedrock \
  -n <NAMESPACE> \
  --from-file=credentials=<PATH_TO_AWS_CREDENTIALS_FILE>

ai-gateway-v2:
  providers:
    - kind: bedrock-anthropic
      baseUrl: https://bedrock-runtime.us-east-1.amazonaws.com
      region: us-east-1
      models: ['us.anthropic.claude-sonnet-5']
      credentialSecretRef:
        name: ai-gateway-v2-bedrock
        key: credentials
      awsProfile: <AWS_PROFILE>

AWS Bedrock OpenAI-compatible API

bedrock-openai serves Bedrock models to OpenAI clients on Chat Completions and Responses through Bedrock's OpenAI-compatible API. It signs requests with AWS credentials and takes the same credential options as Claude on AWS Bedrock: EKS Pod Identity by default, or a credentials file in credentialSecretRef with an optional awsProfile.

Set baseUrl to the bedrock-runtime endpoint with the /openai prefix. The gateway appends the OpenAI API path, such as /v1/chat/completions. List Bedrock's model IDs. OpenAI models use us. or global. inference profiles. Open-weight models, such as gpt-oss, Nemotron, Qwen, and DeepSeek, have no inference profile, so list the in-region model ID and use a region that offers the model.

Bedrock serves open-weight models on Chat Completions only, so list them in a separate provider block with dialects: ['openai-chat']. Valid values are openai-chat and openai-responses, and the gateway refuses to start if a block names a dialect its kind doesn't serve.

ai-gateway-v2:
  providers:
    # OpenAI models: Chat Completions and Responses
    - kind: bedrock-openai
      baseUrl: https://bedrock-runtime.us-east-1.amazonaws.com/openai
      region: us-east-1
      models: ['us.openai.gpt-6.1-sol', 'us.openai.gpt-6-luna']
    # Open-weight models: Chat Completions only
    - kind: bedrock-openai
      baseUrl: https://bedrock-runtime.us-east-1.amazonaws.com/openai
      region: us-east-1
      dialects: ['openai-chat']
      models: ['nvidia.nemotron-super-3-120b', 'qwen.qwen3-coder-next']

GET /v1/models still lists Chat-only models, because the endpoint can't tell a Chat Completions client from a Responses client. A Responses client, such as Codex, that selects one gets a 404, or a 501 when no provider serves the Responses API.

The gateway's IAM role needs bedrock:InvokeModel and bedrock:InvokeModelWithResponseStream on each model's foundation-model ARN. For a profile ID, also grant the inference-profile ARN, and grant the foundation-model ARN in every destination Region the profile lists, such as us-east-1, us-east-2, and us-west-2. The Responses API also authorizes bedrock:InvokeModel on the account's default project, arn:aws:bedrock:<REGION>:<ACCOUNT_ID>:project/default. A permissions boundary on the role must allow the same actions.

A few Bedrock behaviors affect clients on this kind:

Bedrock retains Responses API calls for 30 days unless the request sets "store": false. The gateway forwards the field as the client sent it, and Codex sends "store": false. To turn off retention for the whole account, set Bedrock's data retention mode to none.
Bedrock rejects the hosted web_search tool on the Responses API. The stream ends with an invalid_request_error, and the gateway records no usage. Codex sends this tool by default, so set web_search = "disabled" in the Codex configuration for Bedrock models.
gpt-oss returns its reasoning in <reasoning> tags inside the message content on Chat Completions, and the gateway relays it unchanged.

The baseline pricing catalog includes bedrock-openai rates, priced separately from openai. To confirm a model is priced, look up its bedrock-openai entry as described in step 3 of Add a provider.

Google Vertex AI

Vertex AI authenticates with a Google service account key. Store the key file in a Secret and set project and region. With the global hostname aiplatform.googleapis.com, set region: global. For a regional endpoint, such as us-central1-aiplatform.googleapis.com, match region to the region prefix in the hostname.

kubectl create secret generic ai-gateway-v2-vertex \
  -n <NAMESPACE> \
  --from-file=service_account.json=<PATH_TO_SERVICE_ACCOUNT_JSON>

ai-gateway-v2:
  providers:
    - kind: vertex
      baseUrl: https://aiplatform.googleapis.com
      project: <GCP_PROJECT_ID>
      region: <GCP_REGION>
      models: ['gemini-2.5-pro']
      credentialSecretRef:
        name: ai-gateway-v2-vertex
        key: service_account.json

Service account keys are the only supported Vertex credential. Workload Identity Federation isn't supported.

Google AI Studio

gemini-aistudio uses the Gemini API with an API key:

ai-gateway-v2:
  providers:
    - kind: gemini-aistudio
      baseUrl: https://generativelanguage.googleapis.com
      models: ['gemini-2.5-flash']
      credentialSecretRef:
        name: ai-gateway-v2-gemini
        key: api-key

OpenRouter

openrouter serves OpenRouter's model slugs on the OpenAI Chat Completions and Responses APIs. Set baseUrl to the host only; the gateway adds OpenRouter's /api/v1 path.

ai-gateway-v2:
  providers:
    - kind: openrouter
      baseUrl: https://openrouter.ai
      models: ['deepseek/deepseek-v4-pro', 'moonshotai/kimi-k3']
      credentialSecretRef:
        name: ai-gateway-v2-openrouter
        key: api-key

Rotate a credential

Update the Secret in place, then restart the gateway so new pods mount the new value:

kubectl create secret generic ai-gateway-v2-openai \
  -n <NAMESPACE> \
  --from-literal=api-key='<NEW_OPENAI_API_KEY>' \
  --dry-run=client -o yaml | kubectl apply -f -
kubectl rollout restart deployment/<RELEASE_NAME>-ai-gateway-v2 -n <NAMESPACE>

A helm upgrade also rolls the pods. See How configuration changes apply. Send a test request after the rollout finishes to confirm the new credential is in use.

Remove a provider

Remove the provider entry from providers, upgrade the release, and wait for the rollout to finish. Delete the provider's Secret only after both commands succeed, because pods from the previous ReplicaSet still mount it until the rollout completes:

helm upgrade <RELEASE_NAME> <CHART> -n <NAMESPACE> -f values.yaml
kubectl rollout status deployment/<RELEASE_NAME>-ai-gateway-v2 -n <NAMESPACE>
kubectl delete secret ai-gateway-v2-openai -n <NAMESPACE>

Next steps

Control model access to decide which users and groups can call the models you added.
Route models to providers to understand how requests reach a provider.

Supported providers​

Add a provider​

Provider specifics​

Anthropic​

Claude on AWS Bedrock​

AWS Bedrock OpenAI-compatible API​

Google Vertex AI​

Google AI Studio​

OpenRouter​

Rotate a credential​

Remove a provider​

Next steps​