Providers

Concepts

A provider is an LLM endpoint that AIControls sends traffic to on behalf of your developers: the vendor, its URL, and its credential. You configure API keys once — developers never touch them directly. Which requests a provider receives is decided by model routes.

In Settings → Providers, a provider is listed on the Providers tab and added with Add Upstream; the admin API and the audit log call a provider an upstream.

How providers work

When a developer's AI tool makes a request, AIControls:

1. Authenticates the developer using their personal access token
2. Evaluates policies against the request
3. Picks a provider through the matching model route and forwards the request with the provider's credential
4. Returns the response and logs the full audit event

Developers connect with workspace credentials — they don't need Anthropic or OpenAI accounts.

Proxy and Passthrough modes

AIControls can run in two modes at the same time. Settings → Mode shows both on its Deployment Mode card.

Mode	What it is	Where you change it
Proxy	AIControls calls the provider with the organization's own credential. Developers and agents use workspace tokens or virtual keys and never see the real key.	Derived from your upstreams. Add, edit, or remove upstreams on Settings → Providers (under Models).
Passthrough	The caller's own credential (a claude.ai sign-in, a ChatGPT plan, GitHub Copilot's OAuth token, or a personal API key) is forwarded upstream.	The Passthrough switch on Settings → Mode.

Proxy shows Active when at least one enabled upstream has org credentials: an API key, or managed credentials such as an AWS IAM role, GCP workload identity, or Azure managed identity. The card then lists those upstreams with their provider, credential type, and source, sorted by name. Select the Upstream or Provider column header to change the sort. Lists longer than 10 upstreams are split into pages. config-managed marks an upstream that comes from the deployment's config file (values.yaml or config.yaml); change it there and redeploy. UI marks one stored in the AIControls database, added on Settings → Providers or through the admin API. Select Manage in Models → Providers to change them.

Proxy shows Not configured when no enabled upstream has org credentials. Select Add Upstream in Models → Providers to add one. An upstream with no API key and no managed credentials is a Passthrough upstream and doesn't count. Either way, View upstream status in Health opens Setup → Health on its Upstreams tab, which shows whether each upstream is reachable.

Passthrough is an org-wide setting, tagged Org-wide on the card. Turning it on takes effect immediately. Turning it off asks you to confirm, because callers that forward their own credential (GitHub Copilot, Anthropic team mode) lose access unless Proxy is active or they have their own passthrough grant. It differs from a single user's Allow Passthrough Mode grant on Settings → Users: the grant lets only that user forward their own credential, and matters only while the org-wide switch is off.

With Passthrough on and Proxy not configured, the card warns that passthrough-only mode has no org credential to govern. For PAT-backed callers, model restrictions follow the PAT owner's team and user model scopes and the Global Model Allowlist. Provider-key and OAuth passthrough callers must send a valid AIControls personal access token in X-AIControls-Token unless they already present a supported AIControls credential; otherwise the request is rejected with HTTP 401. GitHub Copilot and GitHub Models can use a GitHub token without the extra PAT header. That GitHub-token-only path resolves a GitHub identity without the PAT/team model-scope fields, so it does not apply team or user model access limits or the Global Model Allowlist. Callers that already present an AIControls credential (a PAT as the bearer token, a session, a virtual key, a token verified by your identity provider, or a GitHub token whose email GitHub resolves) need no extra header. A GitHub token resolves to an email but isn't cryptographically verified, so identity.requireVerifiedIdentity still blocks it. Viewers see the card read-only.

Supported providers

AIControls can route to six classes of upstream provider, each with its own identity and authentication model:

• Anthropic — authenticated with a single sk-ant-... API key when using Anthropic's own API (the default, or an explicit api.anthropic.com URL). AIControls supports all Claude models, including claude-opus-4-8, claude-sonnet-4-6, and claude-haiku-4-5. New models appear automatically as Anthropic releases them, so you don't need to update an upstream definition when a new model ships. Pointing this provider at a custom URL — a third-party gateway that speaks the same Anthropic Messages wire format (e.g. z.ai's GLM models) — is also supported, and the API key can be in that gateway's own format rather than sk-ant-...; the sk-ant-... requirement is only enforced for the canonical Anthropic host.

• OpenAI — authenticated with a single sk-... API key. An OpenAI account can have multiple organizations, so an upstream optionally carries an organization ID to disambiguate which org's models and billing a request should use.

• Azure OpenAI — identity is resource-based rather than account-based: a request targets a specific Azure resource endpoint and a named deployment within it (rather than a model ID directly), and every request is versioned against a specific Azure API version. An Azure upstream authenticates with the resource's API key, sent in Azure's api-key header, or, when it has no key, with Azure AD managed identity.

• Google Vertex — authenticated via Google Application Default Credentials (a service account key file, or GKE/GCE Workload Identity) rather than a vendor-issued API key. Vertex traffic is only supported through the native /vertex/* route today (Gemini's own generateContent/streamGenerateContent protocol) — the OpenAI-compatible /v1/chat/completions route can't translate to it. A request that routes to a Vertex upstream from that route fails with a 502 and an audit deny (#1578). Model discovery (GET /v1/models) for this provider is served from AIControls' cost table rather than a live probe, since Vertex has no OpenAI-shaped discovery endpoint to call — any explicit Static Model List is included alongside those entries.

• Amazon Bedrock — authenticated with AWS credentials (an access key/secret pair, or an assumed role) rather than a vendor-issued API key, and scoped by AWS region. Because Bedrock authentication is just AWS IAM, you can scope a credential tightly — for example, a dedicated IAM user with bedrock:InvokeModel permission limited to specific model ARNs — instead of using a single all-or-nothing key.

• OpenAI-compatible vendors — DeepSeek, Moonshot (Kimi), Z.ai (GLM), OpenRouter, Fireworks AI, Groq, Cerebras, Together AI, Mistral AI, xAI, Perplexity, and Google Gemini, community vendors such as Hugging Face, NVIDIA NIM, DeepInfra, Cohere, MiniMax, and Sarvam AI (see the vendor catalog for the full list), plus local Ollama and vLLM servers and any other OpenAI-compatible URL. Each hosted vendor is authenticated with the vendor's own API key. A local Ollama server, or a vLLM server without a key, gets no credential at all, never the caller's. Pick the vendor and AIControls fills in its endpoint and host from the built-in vendor catalog; the vendor id is the provider name in the audit log and in cost reports. In policies, object.llm.upstream.provider is the vendor id too, so a check for "openai" matches only OpenAI itself, never a vendor reached through an OpenAI-compatible endpoint; see Policies.

The API also accepts a short alias for three of these providers: bedrock for aws-bedrock, vertex for google-vertex, and azure-openai for azure. An upstream saved under an alias is served by the same route and authenticates the same way as one saved under the full name — AWS credentials, Google Application Default Credentials, or, for Azure, an API key or managed identity.

Google Gemini

Google Gemini is a vendor in the vendor catalog, separate from Google Vertex. It uses a Gemini API key and Google's OpenAI-compatible endpoint, https://generativelanguage.googleapis.com/v1beta/openai, instead of Google Cloud credentials.

• Key: create one in Google AI Studio at aistudio.google.com/apikey. AIControls sends it as Authorization: Bearer <key>, never in the URL.
• Endpoint: pick Google Gemini under OpenAI-compatible vendors, or set vendor: gemini in the config file. AIControls fills in the URL and sends requests to .../v1beta/openai/chat/completions. The key is only sent to generativelanguage.googleapis.com.
• Claude Code: Gemini has no Anthropic Messages endpoint, so Claude Code requests are translated to OpenAI Chat Completions and the answers back to Anthropic Messages. Ask for a Gemini model by name, such as gemini-3.8-flash or gemini/gemini-3.8-flash, or map Claude Code's model names with Model Aliases. See Cross-format routing for what is translated.
• Data handling: on Google's free tier, Google may use prompts and responses to improve its products. The catalog asserts no zero-retention or no-training guarantee for Gemini; set the upstream's capabilities if your agreement with Google provides them.

A Provider Key stored with provider Google Gemini (google) holds a Gemini API key and is listed as Google Gemini.

Claude on Google Vertex AI

The google-vertex-anthropic provider serves Claude models from Vertex AI Model Garden. It speaks the Anthropic Messages format and signs requests with the same Google Application Default Credentials as Google Vertex, so it has no API key. Add it in Settings → Providers as Google Vertex (Anthropic Claude), under Other in the setup wizard, in the config file, or through the admin API; see Add Claude on Google Vertex AI.

• URL: the upstream URL names your Google Cloud project and region and ends at the Anthropic models collection: https://{region}-aiplatform.googleapis.com/v1/projects/{project}/locations/{region}/publishers/anthropic/models. For the global region, the host is aiplatform.googleapis.com. In Settings and the setup wizard, the URL is read-only and built from the Project ID and Region fields.
• Requests: /v1/messages and /v1/chat/completions both reach the upstream. AIControls sends each request to {url}/{model}:rawPredict, or to {url}/{model}:streamRawPredict for streaming. The model moves from the request body into the URL, and the body carries "anthropic_version": "vertex-2023-10-16" instead of the anthropic-version header. A model ID may contain only letters, digits, ., _, @, and -; any other model is rejected with a 400 before it reaches Google.
• Model names: Vertex names some Claude models with a version suffix, such as claude-sonnet-4-5@20250929. Map the name your client sends to the Vertex name with Model Aliases. Policies, budgets, cost, routing, and the audit log use the model your client requested.
• Model discovery: Vertex has no model-list endpoint. Set Static Model List to the models GET /v1/models should advertise for the upstream, and leave Live Model Discovery off.
• count_tokens: POST /v1/messages/count_tokens returns a 501 with the error type not_supported_error.
• Claude Code: with CLAUDE_CODE_USE_VERTEX=1, Claude Code builds the Vertex URL itself and sends requests through the /vertex/* route instead.

Jurisdiction of a Google Vertex upstream

On Vertex AI, the region in the upstream URL decides where Google processes a request. For a google-vertex or google-vertex-anthropic upstream, AIControls takes the jurisdiction from that region instead of the provider's default. A data-handling policy that requires a jurisdiction (the proxy.nirmata.io/provider-jurisdiction annotation) checks this value, and policies see it as object.llm.upstream.jurisdiction.

Region in the upstream URL	Jurisdiction
A us- region, such as us-east5, or the us multi-region	US
A europe- region, such as europe-west1, or the eu multi-region	EU
global	global
Any other region, such as asia-southeast1	The region, such as asia-southeast1

• Where the region comes from: the /locations/{region}/ segment of the URL path. When the path has no such segment, the host names it: {region}-aiplatform.googleapis.com, or aiplatform.googleapis.com for global.
• global: Google may serve a global request from any region, so a global upstream never satisfies a policy that requires US or EU. To meet such a policy, use a regional URL.
• No region in the URL: when the URL names no region, such as a proxy in front of Vertex AI, the upstream keeps the provider's default jurisdiction, US.
• Override: an upstream's capabilities.jurisdiction replaces the value taken from the region. Set it only when your agreement with Google guarantees where requests are processed.

For every provider and vendor with its tier, wire formats, authentication, and data-handling defaults, see Supported providers. For the exact fields, console steps, and config values needed to add each provider, see Add a provider.

Routing to providers

A provider doesn't decide which requests it receives. Model routes do: each route sends requests for its model patterns to an ordered list of providers, the most specific pattern wins, and an Access Binding grant can steer its callers to a named route. A provider in no route serves its own Model Patterns, or its family's defaults, through an implicit route. See Implicit routes.

Provider count and licensing

Up to two providers can be enabled on any tier — enough for one provider holding an organization-managed API key alongside a second that forwards each caller's own credential. Three or more enabled providers require an Enterprise license.

Saving a provider in Settings (or PUT /api/v1/llm-upstreams/{name}) that would enable more providers than the active tier permits is refused with 403 and an error naming the limit; editing or disabling existing providers is always allowed. At startup the limit is only a warning: if more providers are enabled than the tier permits, AIControls starts, routes to all of them, and logs a warning naming the configured count and the limit.

Overriding routing with a policy

The provider routing selects is exposed to every policy as object.defaults.routing.upstreamName (with object.defaults.routing.matchedOn explaining why it was picked). A MutatingPolicy with patchType: Defaults can override this per request — for example, routing a call to a data-retention-compliant upstream once content scanning flags PII in the prompt. See Defaults mutations for the worked example and the override's fail-open behavior (an unknown provider name is ignored, never a denial).

This override is evaluated on every request path, so it also applies to policies that only intend to affect LLM traffic — routing has no meaning for a plain MCP tool call and the override namespace is simply absent there.

note

The routing dry-run APIs (POST /api/v1/routing/resolve and POST /api/v1/llm-routing/test) preview only route selection — they do not simulate a patchType: Defaults override a policy might apply at request time.

Routing preview APIs

These admin APIs answer routing questions from the router itself, so they report what live requests get. They send nothing upstream:

• GET /api/v1/llm-routing/analysis returns every route the router uses, including implicit and family routes, with its model patterns, members in try order with weight and credential mode, and strategy. Viewers can read it.
• POST /api/v1/llm-routing/test runs routing for a model and a caller. Test routing in Settings → Providers uses it. Admin only.
• POST /api/v1/routing/resolve returns the selected route, its strategy and why it chose its first member (strategyReason), and the members in the order the request tries them. Admin only.

See Model Routes reference for their fields.

To send a real request instead, POST /api/v1/llm-gateway/test-call with {"model": "claude-haiku-4-5"} sends one short chat completion through the gateway as you, with no provider key of your own, the way an agent's call goes. It returns ok, the HTTP status, the model, the start of the reply, the upstream (provider) the gateway routed the call to, the gateway's message on failure, and licenseLimit: true when the gateway itself refused the call with HTTP 402 because the license or free tier is exhausted, and the call appears in the audit log. The setup wizard's Save and test uses it. Add "blockDemo": true to send a server-chosen prompt that carries a fake AWS access key instead, which the content-safety-credentials starter policy denies. When a policy denies the call, the result includes blocked, with the policy that denied it and its message. A result includes intercepted: true when the gateway's governance answered instead of the model; for the plain test call that is a failure, and message starts with "A policy answered the test call instead of the model". The setup wizard's Try a blocked call uses it. Admin only. Each administrator can send 6 test calls a minute to each gateway replica; another call returns 429 Too Many Requests with a Retry-After header. The failure message is at most 300 characters.

A request can also name a vendor provider with a <vendor>/<model> model name, such as groq/llama-3.3-70b-versatile. An OpenAI-only vendor provider, such as Groq, Cerebras, or Mistral AI, is created with Model Patterns set to its <vendor>/* prefix and the vendor's catalog models, so its own route doesn't catch other models. The vendor receives the model name without the prefix, except OpenRouter, which receives it unchanged because its own model IDs, such as openrouter/auto, start with openrouter/.

GET /api/v1/llm-upstreams marks each provider with hasCredential: true when it holds its own API key or an organization-managed credential, or is a keyless local vendor such as Ollama, so an agent that sends no key can use it. With no enabled saved provider, the list also includes the config file's keyless llmProxy.upstream when the gateway routes to it, with source: "config".

Credential mode

Every provider has a credential mode, derived from its credential:

• org key: the provider has an API key or an organization-managed credential (AWS IAM, Google workload identity, Azure managed identity), or is a keyless local vendor such as Ollama or vLLM, which sends no credential at all. It serves callers that bring no provider credential of their own: PAT, session, OIDC, Kubernetes service-account, and virtual-key callers.
• passthrough: the provider has no API key and no managed credential, so it forwards the caller's own credential. It serves callers that bring one, such as Claude Code signed in to a Claude plan, a ChatGPT sign-in, or GitHub Copilot.

The route editor tags each member org key or passthrough. Inside a route, a caller is served only by members of its kind; see Credential mode in Model Routes.

When the selected provider has no API key

A caller that authenticates with an AIControls token — a PAT, a governed session, or an OIDC sign-in — carries no provider credential. When routing selects a provider that has no API key or organization-managed credential and isn't a keyless local vendor, and the caller sent no provider key of its own, AIControls refuses the request with HTTP 503 and the audit policy no-upstream-key. The message names the provider, the route, the model, and the caller's identity source.

The status is 503, not 401, because the problem is the gateway's configuration, not the caller's credential, so clients such as Claude Desktop don't ask the user to sign in again. To fix it, add an organization key to that provider, give the callers a passthrough credential, or add a provider with a key to the route. In a route, the first member the caller can use must have a key, because AIControls checks the member it tries first.

A local Ollama or vLLM provider with no key is the exception: it sends no credential at all, never the caller's, and counts as having a credential of its own, so agents can use it.

When the provider can't serve the model

When routing selects a single provider whose vendor can't serve the requested model, such as a Claude model on an OpenAI upstream, AIControls refuses the request instead of forwarding it. The client gets HTTP 502, with the code MODEL_PROVIDER_MISMATCH on /v1/messages, and the audit log records a deny under the policy model-provider-mismatch. The message names the model, the upstream, and its provider:

model-provider mismatch: model "claude-sonnet-4-6" is in the anthropic family but upstream "chatgpt" is provider "openai" (family openai)

AIControls checks only providers whose family it knows: Anthropic, OpenAI, Google, and the OpenAI-only vendors in the vendor catalog, such as Groq. Upstreams such as litellm, azure, dual-endpoint vendors such as DeepSeek, and the bare custom provider are never flagged. Google Gemini is checked against the models it serves: gemini-* models, its catalog models, and gemini/<model> names are accepted, and another family's model, such as claude-sonnet-4-5, is flagged. When a Gemini upstream and a Google Vertex upstream match a gemini-* request equally, the Gemini upstream is chosen. A model with a Model Aliases entry on the upstream is a deliberate route and isn't flagged either; see Map the model name with Model Aliases. AIControls checks the upstream the request is sent to, after any patchType: Defaults policy reroutes it. Routes with more than one member aren't checked, because failover and session pinning choose a member for each request, and neither is /v1/responses, which always forwards to an OpenAI upstream.

To fix it, add the model to a route whose members can serve it, or add a Model Aliases entry that maps the model to one the provider serves.

Cross-format routing

Each upstream speaks one wire format: Anthropic upstreams speak the Anthropic Messages API, and OpenAI upstreams speak OpenAI Chat Completions. When routing sends a request to an upstream that speaks a different format than your client, AIControls either translates the request or rejects it. It never forwards a body the upstream can't understand.

This applies to providers whose wire format AIControls knows (Anthropic, OpenAI, and Google families). If the proxy can't translate the request for that pair, it returns a 502 with the code UPSTREAM_FORMAT_MISMATCH. Providers of an unknown family, such as litellm or the bare custom provider, receive the request natively without translation.

Vendors from the vendor catalog are routed by the endpoints they serve:

• A vendor with an Anthropic-compatible endpoint (DeepSeek, Moonshot, Z.ai, OpenRouter, Fireworks AI, MiniMax, Ollama, vLLM) receives Claude Code traffic on that endpoint, natively, and OpenAI clients on its OpenAI endpoint. One upstream serves both; AIControls derives the second endpoint from the URL you configure. In a route it joins whatever provider family the route or session is in, with no Wire format setting.
• A vendor with only an OpenAI endpoint (such as Groq, Cerebras, Together AI, Mistral AI, xAI, Perplexity, and every community vendor except MiniMax) and a vendor: custom endpoint are in the OpenAI family: an Anthropic client's request is translated, as for an OpenAI upstream. Before sending, AIControls applies the vendor's request quirks.

An Anthropic Messages client (POST /v1/messages) can route to an OpenAI upstream, for both streaming and non-streaming requests. AIControls translates the request to Chat Completions, sends it to the upstream, and translates the response back to an Anthropic Messages response. A streamed response arrives as the usual Anthropic event stream, with text streamed as it's generated. Each tool call's input arrives in one piece when the upstream finishes that call. Policies, content scanning, PII tokenization, budgets, and the audit log apply exactly as they do for a same-format request.

Streaming works, so the route suits clients such as Claude Code that always stream. Responses render as they stream, tool calls run, and the session's cost is recorded against the Claude model name the client requested.

To keep Claude Code working against an OpenAI-family upstream, AIControls adapts a few features that Chat Completions can't honor instead of rejecting the request. See What is adapted. Server tools such as web_search, and a top-level request key that Request fields by direction doesn't list as carried or dropped, still fail closed (see What is rejected).

Map the model name with Model Aliases

Your client still requests a Claude model name, such as claude-sonnet-4-6. Add a Model Aliases entry on the OpenAI upstream that maps that name to the upstream's own model ID, such as gpt-4o. Without the alias, AIControls refuses the request as a model-provider mismatch. Alias-aware model matching compares exact keys only, so a glob such as claude-* in an alias map doesn't match any model name.

Budgets, cost, and the audit log record the model your client requested. The audit event also shows the upstream model that served the request. The response's model field reports the model the upstream says it used, or the aliased model ID when the upstream doesn't say.

What is adapted

When an Anthropic Messages request routes to an OpenAI-family upstream, AIControls adapts these features. The audit event lists each dropped feature in its mutated_fields:

• Extended thinking: the top-level thinking setting is dropped (xlate:thinking-dropped), because Chat Completions has no equivalent. thinking and redacted_thinking blocks from earlier turns are dropped too (xlate:thinking-history-dropped). The OpenAI upstream doesn't need them. A session moves back to an Anthropic member only if the OpenAI family's credit or quota runs out too (switching family when quota runs out), and that later request carries no thinking history.
• Images: an image block becomes an image_url content part, as a data: URL for base64 data or as the image's URL. The upstream model must accept images.
• count_tokens: POST /v1/messages/count_tokens returns an estimate, about four characters per token of the request's strings, because OpenAI has no token-counting API. It runs slightly high. The response carries the header X-AIControls-Token-Count: estimated, and the audit event records xlate:token-count-estimated. A ChatGPT OAuth upstream, which serves only the Responses API, still returns a 501.

What is rejected

Translation fails closed: when a request contains something Chat Completions can't represent faithfully, AIControls rejects the request instead of silently dropping it. Your client gets a 502 error with the code UPSTREAM_FORMAT_MISMATCH and a message that names the reason, and the audit log records a deny with the policy name upstream-format-mismatch. AIControls rejects these requests:

• Document content blocks, and image blocks nested inside a tool result.
• Server tools, such as web search, bash, text editor, and computer use.
• MCP servers (mcp_servers), a top_k value, tool_choice.disable_parallel_tool_use, and any other top-level request field not listed in Request fields by direction. The non-semantic hints service_tier and anthropic_version are dropped instead, and thinking is adapted as described in What is adapted.

In a cross-provider route, a new session's first request that an OpenAI member can't carry skips that member instead of failing: AIControls records the skip in the audit log, sends the request to the next member that speaks the client's format natively, and pins the session to that member's family. The request fails with the 502 only when no member can carry it, or when the session is already pinned to a family whose members can't.

The count_tokens endpoint returns a 501 when the request routes to a Google Vertex (Gemini) upstream.

Requests routed to a Google Vertex (Gemini) upstream from an Anthropic Messages client are always rejected the same way. So are requests routed to a ChatGPT OAuth upstream: its backend serves only the OpenAI Responses API, not Chat Completions, so it isn't supported for cross-format routing.

Usage and known limits

• Cache reads: OpenAI counts cached tokens inside its prompt token total. AIControls reports them the Anthropic way: input_tokens excludes cached tokens, and cache_read_input_tokens reports them separately. Cost and budget accounting use the same split.
• Refusals and reasoning: when an OpenAI model refuses, your client gets the refusal text as a text block, with stop_reason set to refusal. Response policies and DLP evaluate the refusal text like any other response, and full capture records it when it's on. Reasoning content isn't sent to your client, because an Anthropic thinking block needs a signature that only Anthropic can produce. When full capture with Capture reasoning is on, the audit log stores it.
• Invalid tool-call arguments: when the upstream model returns tool-call arguments that aren't a JSON object, such as truncated JSON or plain text, AIControls still delivers the tool_use block, with the raw text as the only input field, _invalid_tool_arguments. A tool that validates its input can reject it, so the model sees its mistake and can retry. A tool that doesn't validate its input may run without the arguments it expects. Empty arguments become an empty input, {}.
• Streaming usage: AIControls asks the upstream to report token usage at the end of the stream, so the final message_delta event carries the input, cache-read, and output token counts. The opening message_start event reports zero tokens, because OpenAI reports usage only at the end.
• Streaming errors: an upstream error before the stream starts reaches your client as an Anthropic error response with the upstream's status code. An error after the stream starts ends it with an Anthropic error event. If the upstream interleaves the arguments of two tool calls, which the Anthropic event format can't represent, AIControls ends the stream with an error instead of delivering corrupted tool input.

OpenAI clients on Amazon Bedrock

An OpenAI Chat Completions client can also route to an aws-bedrock-anthropic upstream, for both streaming and non-streaming requests. AIControls translates the request to Bedrock's InvokeModel format, signs it with the upstream's AWS credentials, and translates the response back to Chat Completions. Budgets, cost, and the audit log record the model your client requested, the same as any other cross-format route.

Request fields by direction

These tables list every top-level request field each translation reads, and what happens to it:

• Carried: the field reaches the upstream, mapped to its equivalent.
• Dropped: the field is removed and the request continues. Where the Reason column names an xlate: marker, the audit event lists that marker in mutated_fields; other drops aren't recorded.
• Refused: the translation fails. The client gets a 502 with the code UPSTREAM_FORMAT_MISMATCH, unless a cross-provider route sends the request to a member that speaks the client's format, as described in What is rejected.
• Refused (non-null): a null value is dropped, and any other value is refused.

Anthropic Messages to OpenAI Chat Completions

Any top-level field not listed here is refused. cache_control markers inside the request are dropped, because they're caching hints with no Chat Completions equivalent.

Field	Behavior	Reason
model	Carried	Sent as the upstream's model ID from its Model Aliases, or unchanged when no alias matches.
max_tokens	Carried	Sent as max_completion_tokens to an openai upstream, and as max_tokens to other OpenAI-compatible upstreams.
messages	Carried	Converted to Chat Completions messages. Image blocks in user messages become image_url parts, and tool_result blocks become tool messages without their is_error flag. Thinking blocks are dropped (xlate:thinking-history-dropped); document blocks and server tool blocks are refused.
system	Carried	Sent as a leading system message. Text blocks are joined with newlines.
tools	Carried	Sent as function tools. Server tools, such as web search, and a tool with no name are refused.
tool_choice	Carried	Mapped to the Chat Completions tool_choice. Refused when it's set but tools is empty, when it forces a tool that isn't in tools, when its type is unknown, or when it sets disable_parallel_tool_use: true.
temperature	Carried	Same meaning in both formats.
top_p	Carried	Same meaning in both formats.
stop_sequences	Carried	Sent as stop.
stream	Carried	AIControls also asks the upstream to report token usage at the end of the stream.
metadata	Carried	metadata.user_id is sent as user.
thinking	Dropped	Chat Completions has no extended thinking. A non-null value is recorded as xlate:thinking-dropped.
service_tier	Dropped	A scheduling hint with no Chat Completions equivalent. It doesn't change the response.
anthropic_version	Dropped	Applies only to the Anthropic API.
top_k	Refused (non-null)	Chat Completions has no top_k.
mcp_servers	Refused	Chat Completions can't call MCP servers on the client's behalf.

OpenAI Chat Completions to Anthropic Messages

This direction applies when a Chat Completions client routes to an Anthropic upstream. Any other field isn't read, so it doesn't reach the upstream.

Field	Behavior	Reason
model	Carried	Sent as the upstream's model ID from its Model Aliases, or unchanged when no alias matches.
messages	Carried	system messages become the Anthropic system prompt, and assistant messages keep their text and tool calls. Adjacent user and tool messages merge into one user turn, and tool messages become tool_result blocks. Only text is kept from content parts, so image_url parts are dropped. Messages with any other role, such as developer, are dropped.
max_tokens	Carried	Anthropic requires max_tokens, so AIControls sends 16384 when the request has none or the value isn't positive.
temperature	Carried	Same meaning in both formats.
top_p	Carried	Same meaning in both formats.
stop	Carried	Sent as stop_sequences.
stream	Carried	The Anthropic event stream is translated back to Chat Completions chunks.
tools	Carried	function tools are sent as Anthropic tools. A tool with no name is dropped.
tool_choice	Carried	auto, required, none, and a forced function map to their Anthropic equivalents. It's dropped when no tools are carried, when it forces a function that isn't in tools, or when it has any other value.
max_completion_tokens	Dropped	Not read. Set max_tokens instead, or AIControls sends the 16384 default.
response_format	Dropped	Not read. Structured output isn't enforced; ask for the format in the prompt.
parallel_tool_calls	Dropped	Not read. The model may still call tools in parallel.
logit_bias	Dropped	Not read. Anthropic has no token-bias option.
presence_penalty	Dropped	Not read. Anthropic has no presence penalty.
frequency_penalty	Dropped	Not read. Anthropic has no frequency penalty.
seed	Dropped	Not read. Anthropic has no sampling seed.
reasoning_effort	Dropped	Not read. Extended thinking isn't requested.
functions	Dropped	Not read. Use tools; legacy function definitions don't reach the model.
function_call	Dropped	Not read. Use tool_choice.
logprobs	Dropped	Not read. Anthropic returns no log probabilities.
top_logprobs	Dropped	Not read. Anthropic returns no log probabilities.
store	Dropped	Not read. Applies only to OpenAI's stored completions.
service_tier	Dropped	Not read. A scheduling hint for OpenAI only.
n	Dropped	Anthropic returns one completion per request.
stream_options	Dropped	Anthropic reports token usage on every response, so the option isn't needed.
user	Dropped	Not mapped to Anthropic's metadata.user_id. The audit log still records the caller's identity.

OpenAI Chat Completions to Amazon Bedrock

This direction uses the same conversion as OpenAI Chat Completions to Anthropic Messages, then adapts the body to Bedrock's InvokeModel format. Any other field isn't read, so it doesn't reach the upstream.

Field	Behavior	Reason
model	Carried	Sent in the request URL as the Bedrock model or inference profile ID, not in the body.
messages	Carried	Converted as in the OpenAI to Anthropic direction.
max_tokens	Carried	Bedrock requires max_tokens, so AIControls sends 16384 when the request has none or the value isn't positive.
temperature	Carried	Same meaning in both formats.
top_p	Carried	Same meaning in both formats.
stop	Carried	Sent as stop_sequences.
stream	Carried	Selects InvokeModelWithResponseStream instead of InvokeModel.
tools	Carried	Converted as in the OpenAI to Anthropic direction.
tool_choice	Carried	Converted as in the OpenAI to Anthropic direction.
max_completion_tokens	Dropped	Not read. Set max_tokens instead, or AIControls sends the 16384 default.
response_format	Dropped	Not read. Structured output isn't enforced; ask for the format in the prompt.
parallel_tool_calls	Dropped	Not read. The model may still call tools in parallel.
logit_bias	Dropped	Not read. Bedrock has no token-bias option.
presence_penalty	Dropped	Not read. Bedrock has no presence penalty.
frequency_penalty	Dropped	Not read. Bedrock has no frequency penalty.
seed	Dropped	Not read. Bedrock has no sampling seed.
reasoning_effort	Dropped	Not read. Extended thinking isn't requested.
functions	Dropped	Not read. Use tools; legacy function definitions don't reach the model.
function_call	Dropped	Not read. Use tool_choice.
logprobs	Dropped	Not read. Bedrock returns no log probabilities.
top_logprobs	Dropped	Not read. Bedrock returns no log probabilities.
store	Dropped	Not read. Applies only to OpenAI's stored completions.
service_tier	Dropped	Not read. A scheduling hint for OpenAI only.
n	Dropped	Bedrock returns one completion per request.
stream_options	Dropped	Bedrock reports token usage on every response, so the option isn't needed.
user	Dropped	Not mapped to metadata.user_id. The audit log still records the caller's identity.

Health criticality

Every upstream counts toward AIControls' health status. By default an upstream is critical: when it's unreachable or degraded, the Health page shows AIControls as degraded, and as down when every critical upstream is unreachable. An admin can mark an upstream non-critical, so its problems show only as warnings, or ignored, so it raises no issues at all. Use non-critical for a fallback or experimental upstream whose outage doesn't stop your users, and ignored for one you're retiring. Criticality only changes how health is reported; it never changes routing or failover. See Mark an upstream non-critical or ignored for the steps and Upstream criticality for the exact effect of each level.

See also

• Add a provider — setup steps for each provider
• Model Routes — how requests reach providers
• Model Routes reference — configuration and API fields