Skip to main content

Add a Provider

Tasks

Add a provider so AIControls can send developer requests to it. For what a provider is, see Providers. To decide which requests it serves, see Add a route.

In Settings → Providers, the Providers tab lists your providers. Add Upstream opens a page with four steps:

StepWhat you set
ProviderThe provider or vendor. Type in the search box to filter the cards, grouped under Providers and OpenAI-compatible vendors.
ConnectionName, URL, the API key or the provider's own credentials, Test Connection, and Model Aliases.
RoutingModel Patterns, Static Model List, Live Model Discovery, and Enabled. Model Patterns are the models the provider serves through its own route while it is a member of no route. On a route member, the field shows Set by route with an Edit route button for each route instead.
ReviewA summary of the upstream. Each row's Edit opens its step. Select Save.

Picking another provider clears API Key, so a key is never sent to a provider it wasn't entered for, and sets Live Model Discovery to the new provider's default: on for providers that list their models live, such as OpenRouter and Fireworks AI, off otherwise. When you edit an upstream, picking its saved provider again restores its saved setting. Editing a value that Test Connection checked, such as the URL, key, or AWS region, clears the test result.

Next checks the current step before moving on. Editing an upstream opens the same page on Connection: every step can be opened from the step bar, and Save is available on every step. Cancel and Back to LLM Upstreams return to the upstreams table.

  1. Go to Settings → Providers → Add Upstream and select Anthropic.

  2. Fill in the upstream fields on the Connection and Routing steps:

    FieldValue
    NameAny label, e.g. anthropic-prod
    API KeyYour sk-ant-... key from console.anthropic.com
    URLLeave blank for Anthropic's own API. Point it at a third-party gateway instead to use a service that speaks the same Anthropic Messages wire format (e.g. z.ai's GLM models) — the API key can then be in that gateway's own format instead of sk-ant-...
    Model PatternsLeave blank to serve the Claude family defaults (claude-* and related patterns), or list specific model IDs
note

AIControls supports all Claude models including claude-opus-4-8, claude-sonnet-4-6, and claude-haiku-4-5. New models appear automatically as Anthropic releases them.

note

The sk-ant-... key format is only enforced when the URL is left blank or points at Anthropic's own API (api.anthropic.com). A custom URL accepts whatever key format that gateway issues.

warning

Provider names must be unique, and share one name space with routes — adding a provider with a name that already exists is rejected so an existing provider or route is never overwritten by mistake. Editing an existing upstream or pool, such as rotating its API key, keeps working when the route of the same name has the upstream, or one of the pool's upstreams, as a member, as the route converted from it does. Deleting a provider that a route uses is refused with 409; remove it from the route first. Deleting or disabling a provider whose own implicit route an Access Binding grant names is refused with 409 too; change the binding's Via route first. On a deployment whose providers come from the configuration file, saved providers replace the file's upstreams: once a stored route uses a config-file provider, saving the first enabled provider is refused with 409, naming the route, until you remove that provider from the route or save a provider of the same name.

When you add more than one provider for the same models, such as two Anthropic keys or Anthropic and Amazon Bedrock, put them in one route, in the order you want them tried. See Add a route. Two providers of the same family that each serve the same models through their own route tie, and requests either could serve fail with 503 and the code UPSTREAM_AMBIGUOUS.

A provider with no API key and no managed credential has the passthrough credential mode: it forwards the caller's own credential and serves only callers that bring one. Every other provider has the org key mode. See Credential mode.

Add Claude on Google Vertex AI​

The google-vertex-anthropic provider serves Claude models from Vertex AI Model Garden. Add it in Settings → Providers, in the config file, or with the admin API. The setup wizard offers it too, under Other — see Other providers. Once the upstream exists, the wizard's Model step offers it as configured, with Google workload identity and its project and region in place of a key — see An upstream that's already set up.

  1. Enable the Claude models you want in Vertex AI Model Garden for your Google Cloud project, and note the region they're enabled in.
  2. Give the AIControls proxy Google Application Default Credentials: a service account key file, or GKE/GCE Workload Identity. The service account needs permission to call Vertex AI predictions, such as the Vertex AI User role (roles/aiplatform.user).

  3. Add the upstream.

    Go to Settings → Providers → Add Upstream and select Google Vertex (Anthropic Claude), listed under Providers after Google Vertex. On the Connection step:

    FieldValue
    NameAny label, e.g. vertex-claude
    Project IDThe Google Cloud project where Claude is enabled in Vertex AI Model Garden, e.g. my-project
    RegionWhere Vertex AI runs the model: global (any region with capacity), us-east5, europe-west1, or asia-southeast1
    URLRead-only. Built from the project ID and region
    Model AliasesOptional — map the name your client sends to a Vertex model name, e.g. claude-sonnet-4-5=claude-sonnet-4-5@20250929

    There's no API Key field and no Test Connection: the upstream uses the proxy's Google credentials. Until Project ID and Region are set, Review shows the URL as From the project ID and region.

    The region also sets the upstream's jurisdiction for data-handling policies: a us- region is US, a europe- region is EU, and global meets neither a US nor an EU requirement. See Jurisdiction of a Google Vertex upstream.

    When you edit the upstream, Project ID and Region are filled in from its saved URL. A saved URL in any other form, such as a proxy in front of Vertex AI, is kept as it is, with the two fields empty. Filling them in replaces the URL.

  4. Send a test request to /v1/messages or /v1/chat/completions with a model the upstream matches. Test Connection isn't available for this provider, because it uses the proxy's Google credentials.

AIControls sends each request to {url}/{model}:rawPredict, or {url}/{model}:streamRawPredict for streaming, with the model removed from the body and "anthropic_version": "vertex-2023-10-16" added. Use Model Aliases (modelAliases) to map the model name your client sends to a Vertex model name with a version suffix. Vertex has no model-list endpoint, so list the models to advertise on GET /v1/models in Static Model List (staticModels) and leave Live Model Discovery off. count_tokens returns a 501 for this provider. For details, see Claude on Google Vertex AI.

See also​

Testing private upstreams​

Connection tests connect directly to the configured endpoint and do not follow HTTP redirects or use environment HTTP proxies. For a private upstream, ask your self-hosted operator to add its IP or CIDR to proxy.ssrfAllowlist in the proxy configuration. Use the final endpoint URL rather than a redirect URL. The same restriction applies to OIDC connectivity probes, OAuth metadata fetches, Okta discovery, and cross-app token exchange. Permit only the internal ranges these integrations require; metadata and loopback addresses are blocked by default.

Cross-app access token endpoints must use HTTPS, including endpoints on allowlisted private networks. An HTTP endpoint is rejected before any client secret or signed assertion is sent.

Last updated on