Configure an Upstream

Tasks

Add an upstream provider so AIControls can route developer requests to it. For how upstream routing works conceptually, see Upstreams.

Anthropic

1. Go to Settings → Upstreams → Add upstream and select Anthropic.

2. Fill in the upstream fields:

   Field	Value
   Name	Any label, e.g. anthropic-prod
   API Key	Your sk-ant-... key from console.anthropic.com
   URL	Leave blank for Anthropic's own API. Point it at a third-party gateway instead to use a service that speaks the same Anthropic Messages wire format (e.g. z.ai's GLM models) — the API key can then be in that gateway's own format instead of sk-ant-...
   Models	Leave blank to allow all, or restrict to specific model IDs

note

AIControls supports all Claude models including claude-opus-4-8, claude-sonnet-4-6, and claude-haiku-4-5. New models appear automatically as Anthropic releases them.

note

The sk-ant-... key format is only enforced when the URL is left blank or points at Anthropic's own API (api.anthropic.com). A custom URL accepts whatever key format that gateway issues.

OpenAI

1. Select OpenAI as the provider.

2. Fill in the upstream fields:

   Field	Value
   Name	e.g. openai-prod
   API Key	Your sk-... key from platform.openai.com
   Organization ID	Optional — your OpenAI org ID if using multiple orgs

Azure OpenAI

1. Select Azure OpenAI as the provider.

2. Fill in the upstream fields:

   Field	Value
   Name	Any label, e.g. azure-foundry-prod
   URL	Your resource endpoint plus /openai, e.g. https://YOUR_RESOURCE.cognitiveservices.azure.com/openai
   Model Patterns	Glob(s) matching the deployment name(s), e.g. gpt-4o — controls routing
   Static Model List	The deployment name(s) you created in Azure AI Foundry, e.g. gpt-4o — controls what's advertised via model discovery

note

This covers Azure OpenAI-family models (e.g. GPT-4o) reached via chat completions — the endpoint format above is specific to that path. If your organization instead routes Claude models through Azure AI Foundry (the Claude Code cloud provider gateway), a different endpoint format applies — check with your AIControls administrator for that setup.

note

Azure upstreams added here always authenticate via Azure AD managed identity (DefaultAzureCredential) — there's no API key field, and any value you'd enter here is never sent. Credentials (a service principal's AZURE_CLIENT_ID/AZURE_TENANT_ID/AZURE_CLIENT_SECRET, or workload identity federation) are configured on the AIControls proxy's environment by your platform team, not in this form. (Self-hosted deployments configuring upstreams directly in config.yaml can instead set an explicit API key per upstream — that path isn't covered by this page.)

Google Vertex

1. Select Google Vertex as the provider.

2. Fill in the upstream fields:

   Field	Value
   Name	Any label, e.g. vertex-prod
   URL	https://aiplatform.googleapis.com, or a regional variant, e.g. https://us-east5-aiplatform.googleapis.com
   Model Patterns	Glob(s) matching Gemini model IDs, e.g. gemini-* — controls routing
   Static Model List	Optional — explicit Gemini model IDs to advertise via model discovery, e.g. gemini-2.5-pro

note

Vertex upstreams authenticate via Google Application Default Credentials (ADC) — a service account key file, or GKE/GCE Workload Identity — configured on the AIControls proxy's pod environment, not in this form. There is no API key field.

note

Send requests through the native /vertex/* route (used by CLAUDE_CODE_USE_VERTEX=1 and the Vertex/Gemini SDKs' own request format) — this is the only path that speaks Vertex's native generateContent/streamGenerateContent protocol today. The OpenAI-compatible /v1/chat/completions route accepts a google-vertex upstream as a routing target but does not yet translate between the two protocols, so it isn't usable for Vertex traffic yet.

GET /v1/models for a Vertex upstream is served from the Gemini entries in AIControls' cost table, not a live probe against Google — Vertex has no OpenAI-shaped /v1/models endpoint to probe. This means the models listed there track what AIControls has pricing for, not necessarily every Gemini model your GCP project has access to; set Static Model List to advertise a specific model that isn't priced yet — it's always included alongside the cost-table entries, regardless of the Live Model Discovery setting.

Amazon Bedrock

1. Select Amazon Bedrock as the provider.

2. Fill in the upstream fields:

   Field	Value
   AWS Region	e.g. us-east-1 — the region where your Bedrock models are enabled
   IAM Role ARN	Role AIControls assumes to sign Bedrock requests, e.g. arn:aws:iam::123456789012:role/MyBedrockRole
   Inference Profile	Which cross-Region profile to invoke: US (default), Europe, Asia Pacific, GovCloud, or Global

   AWS Region and IAM Role ARN are required; Inference Profile defaults to US. Bedrock upstreams authenticate with IAM (SigV4) — there is no API key field, and the endpoint URL is derived from the region for you.

note

Grant the role bedrock:InvokeModel scoped to the model and inference-profile ARNs you want to allow. Claude 3.7 and newer can only be invoked through a cross-Region inference profile, so the policy needs inference-profile/ resources as well as foundation-model/ ones — and Global profiles need a third ARN on top. See Prerequisites for policies you can copy.

tip

Model names are translated to the matching Bedrock inference-profile ID for you, so send the short name (claude-sonnet-4-5) from your client. Inference Profile decides which one — set it to match your geography, since the default is US regardless of the region you configured. Requests still go to your configured region; Bedrock does the cross-Region routing itself.

Cost is attributed against the short name, so Bedrock traffic rolls up alongside direct-provider traffic for the same model whichever profile you pick. Use Model Aliases to pin an exact profile ID for a single model — it takes precedence over this setting.

warning

Upstream names must be unique — adding an upstream with a name that already exists is rejected so an existing upstream is never overwritten by mistake.

When you configure more than one upstream for the same provider, give each a distinguishing scope — a different Priority, or a Model Patterns / Identity Source match — so AIControls knows which one to route each request to. Two enabled same-provider upstreams with the same priority and no distinguishing scope are ambiguous, and requests that either could serve are denied with an ambiguous-routing error. The Add/Edit form warns you before you save such a pair.

Configure a pool for failover

Preview

Upstream pools are a preview feature — see Upstream pools for how membership and failover are decided.

1. Configure the upstreams you want as pool members first (see the provider steps above) — a pool references existing upstreams by name, it doesn't hold its own connection details.
2. Go to Settings → Upstreams, switch to the Pools tab, and select Add Pool.

3. Fill in the pool fields:

   Field	Value
   Name	Any label, e.g. prod-anthropic
   Model Patterns	Glob(s) controlling which requests route to this pool, e.g. claude-* — leave empty to match any model
   OIDC groups	Leave empty to match every caller
   Priority	Tie-break order against other pools/upstreams with equal specificity — leave at 0 unless you need explicit ordering

4. Add members in the order you want them tried. For each member:

   • Pick the upstream from the dropdown.
   • Watch the compatibility indicator — a member whose wire format conflicts with (or is ambiguous relative to) the members above it is flagged, and an Azure OpenAI / Azure AI Foundry member additionally needs its wire format set explicitly before it's included.
   • Check Fallback only if this member should never be tried first, regardless of its position in the list.

   Use the arrow buttons to reorder members, or the ✕ button to remove one — a pool needs at least one member.

5. Save. If any member was excluded for a wire-format conflict, the save still succeeds but names the excluded member so you can fix it.

note

An upstream referenced by a pool is routed through the pool's Model Patterns / OIDC groups / Priority from then on — its own routing settings (if any) are ignored while it's a member. Removing it from the pool restores its own routing.

See also

For how AIControls picks between multiple configured upstreams, see Upstreams.