Model Routes Reference

Reference

Lookup tables for providers, model routes, and route grants. For the model behind them, see Model Routes and Providers.

Route fields

A route as the routes API reads and writes it:

Field	Type	Description
name	string	Unique across routes and providers. Required.
enabled	boolean	Default true. A disabled route doesn't route and isn't part of the overlap check.
models	string[]	Model patterns: exact names, or globs with a * at the start, the end, or both. At least one.
members	object[]	Provider members in member order. At least one. See Member fields.
strategy	object	type: failover (default), round-robin, lowest-cost, lowest-latency, or weighted. options: the strategy's options, see Strategy options.
allowTranslation	boolean	Allow cross-provider members. Default false.
switchFamilyOnQuota	boolean	Switch family when credit runs out. Default true. Applies only with allowTranslation.
orgKeyFallbackForCallerCredentials	boolean	Fall back to org key members for callers' own credentials. Default false. When true, a caller who brings its own credential fails over to the route's org-key members after every passthrough member fails with a retryable or quota error. Applies only when the route has both a passthrough member and an org-key member. See Org key fallback.
modelAliases	object[]	Each alias has name and wireModels, a map from member provider name to the model that member receives.
source	string	Read-only. config for a route from the deployment's config file, which can't be changed through the API.

Member fields

Field	Type	Description
provider	string	The provider's name (an entry on the Providers tab, GET /api/v1/llm-upstreams). Required. The provider must exist and be enabled.
fallbackOnly	boolean	Tried only after every other member, in member order.
wireFormat	string	anthropic or openai-chat. Required for an Azure OpenAI or Azure AI Foundry member; ignored for providers whose format is known.
weight	integer	Weighted strategy only: 0 to 1,000,000, default 1. 0 never serves first. Kept and ignored by other strategies.

Strategy options

Strategy	Option	Default	Range
lowest-latency	windowMinutes	15	1–1440
lowest-latency	minRequests	20	1–1,000,000
lowest-latency	explorationShare	0.05	0–0.5

Other strategies take no options. An unknown option, or a value out of range, returns 400. windowMinutes and minRequests must be whole numbers. explorationShare is the share of new sessions that start on a member with fewer than minRequests requests in the window, so it gets measured; 0 turns this off.

Validation

A route save returns 400 when:

• the route has no model pattern, or a pattern has a * anywhere but its start or end, is empty, or starts or ends with a space;
• a pattern overlaps another enabled route's pattern at the same specificity (the error names both routes and patterns);
• a member names an unknown or disabled provider;
• the strategy or an option is unknown or out of range, a weight is negative or above 1,000,000, or a weighted route has no member that isn't Fallback only with a weight above 0;
• an alias contains a wildcard, repeats within the route, maps a provider that isn't a member, or equals another route's exact model pattern.

The response to a successful save carries wire-format warnings, such as a member excluded for a format conflict.

Routes API

Method	Path	Access	Notes
GET	/api/v1/llm-routes	Read	{"routes": [...], "routingSource": true}: stored and config-file routes. routingSource is false until the upstreams and pools are converted to stored routes; the first route write converts them, so it never blocks a write. Implicit routes are listed by the routing analysis.
GET	/api/v1/llm-routes/{name}	Read	404 if missing.
POST	/api/v1/llm-routes	Admin	409 when the name is taken by a route, a provider, or a family route, or when a member is a provider whose implicit route an Access Binding grant names. 400 when a member isn't an enabled provider the router builds.
PUT	/api/v1/llm-routes/{name}	Admin	Replaces the route. 404 if missing; 409 for a config-file route, when it disables a route an Access Binding grant names, when it changes a granted route's model patterns or aliases so it serves none of the grant's models, or when it adds a provider whose implicit route a grant names.
DELETE	/api/v1/llm-routes/{name}	Admin	409 for a config-file route, or while an Access Binding grant names the route.

Every route write is recorded in the configuration audit as upstream.configuration_updated, and the router uses the change right away on every replica. The first route write on a deployment whose upstreams and pools aren't converted yet converts them first (see Routes from an upgrade).

Providers API

The Providers tab reads and writes providers through the upstreams API:

Method	Path	Access	Notes
GET	/api/v1/llm-upstreams	Read	Each provider has hasCredential; see Routing preview APIs.
PUT	/api/v1/llm-upstreams/{name}	Admin	Add ?create=true to create. 409 when the name is a route's, when it disables a provider a stored route uses, when disabling it removes a route an Access Binding grant names, or when it is the first enabled saved provider and a stored route uses a config-file provider of another name (saved providers replace the configuration file's). 403 when the license's provider limit would be exceeded. 400 when the body sets groups (other than ["*"]), matchLabels, matchSources, matchNamespaces, matchServiceAccounts, or priority to a value other than the provider's saved one: routes match on the model only. A provider saved before the upgrade still lists those fields, and a body that sends them back unchanged is accepted.
DELETE	/api/v1/llm-upstreams/{name}	Admin	409 while a stored route uses the provider, or when deleting it removes a route an Access Binding grant names. 404 for a config-file provider.
GET	/api/v1/llm-upstreams/activity	Read	Per-provider activity, with routes per route.

A provider's Model Patterns (matchModels) decide only its own implicit route, while it is a member of no route.

Access Binding route grants

An Access Binding model selector takes an optional route:

{
  "name": "platform-bedrock",
  "decisionMode": "allow",
  "subjects": [{"type": "team", "value": "platform"}],
  "modelSelectors": [{"models": ["claude-*"], "route": "claude-bedrock"}]
}

Rule	Response
Only an allow-mode binding's selector can name a route.	The editor drops route for other decision modes.
The route must be one the router serves.	400, naming the route
The route's model patterns or model aliases must match some of the selector's models.	400, naming the route and the models
A selector that names a route can't also list providers or upstreams.	400
Two bindings grant the same subject the same model pattern at the same level through different routes.	409 with "code": "route_grant_conflict", naming both bindings

GET /api/v1/access-bindings/model-limits?subject=team:platform (admin) returns the model limits outside bindings that apply to the subjects: global.allowedModels, teams[], personalAccessTokens[], and virtualKeys[], each with allowedModels. Repeat subject as <type>:<value>, or pass subject=everyone. The binding editor's Other model limits for these callers reads it.

Routing API

GET /api/v1/llm-routing/analysis

Read access. Returns {"routes": [...]}, every route the router uses, and unavailable: true when no provider is configured. Each route has name, models, strategy, members[] (provider, fallbackOnly, weight, credentialMode), and implicit or family for implicit routes.

POST /api/v1/llm-routing/test

Admin. Test routing uses it. Request:

Field	Description
model	The model to route. Required.
team, user, groups	The caller: a team ID, a user email, and IdP groups. Optional; they decide which Access Binding grant applies.
source	The caller's identity source, such as pat or passthrough. A source that brings its own credential is served only by passthrough members. Optional.
sessionId	Previews the member a session starts on.
format	anthropic or openai-chat, the client's wire format.

Response:

Field	Description
route, routeModels	The selected route and its model patterns.
matchedOn	Why the route was selected, such as binding-route for a grant that names it, or model-alias.
provider	The member a new request starts on.
members[]	Members in try order: provider, fallbackOnly, credentialMode.
strategy, strategyReason	The route's strategy and why it chose the first member, with the price, P95 latency, or weight it chose by.
grant	The winning grant: binding, level (workload, blueprint, team, everyone), pattern, route, implicit, granted.
withoutGrant	route and provider the request would use without a grant that names a route.
wouldFail, failStatus, failCode, failMessage	Set when the request would be refused. failCode is no-route or no-grant.
closestRoute, closestPattern	For no-route, the route whose pattern most resembles the model.

POST /api/v1/routing/resolve

Admin only. Request: model, and optionally sessionId, format, and the caller's team, user, and groups, which decide the Access Binding grant that applies, as in a live request. Response: upstream, url, and provider of the first member; matched_on; strategy and strategyReason; order, the member names in try order; pool, the selected route's name; alias when a model alias matched (matched_on is model-alias); and members, one entry per member in order:

Field	Description
upstream	The member's provider name.
vendor	The vendor catalog ID, omitted for a built-in provider.
provider	The provider type.
wireModel	The model the member receives: the alias's wire model, else the provider's Model Aliases entry, else the built-in mapping, else the model as requested.
format	The member's wire format, such as anthropic, openai-chat, or openai-responses.
translated	true when a client of the request's format is translated to the member's format.
parked, parkedUntil	Whether the member is parked, and when it is probed again.
skipReason	quota_parked, breaker_open, or, with format, unreachable.

When no route serves the model, matched_on is no-route and ambiguous carries the error. When Access Bindings grant models but none grants this model to the caller, matched_on is no-grant and ambiguous carries the error. These APIs preview route selection only; they don't apply a patchType: Defaults policy override.

Errors

HTTP status	Code	Audit policy	When
503	NO_ROUTE	no-route	No route serves the model; the selected route has no member for the caller's credential mode; or the caller's grant names a route the router doesn't have or one that doesn't serve the model.
400	MODEL_DENIED	no-grant	Access Bindings grant models, but none grants this model to the caller.
503	UPSTREAM_AMBIGUOUS	upstream-ambiguous	Two routes tie for the model and its family doesn't break the tie.
503	—	no-upstream-key	The member tried first has no credential and the caller brought none.
varies	UPSTREAM_POOL_EXHAUSTED	—	Every member of a session's pinned family failed.

The code is in error.code on the Anthropic Messages API, /v1/chat/completions, and /v1/responses. On /v1/chat/completions, a no-grant refusal is a 403 with MODEL_DENIED. On /v1/responses it is a 400 with the code invalid_prompt, as every policy refusal there is.

Audit and policy fields

Field	Description
routeName (audit event)	The route the call went through. poolName carries the same value.
routeStrategy, routeStrategyReason (audit event)	The route's strategy and why it started the request on its member.
credentialFallback, credentialFallbackReason (audit event)	true when a route's org-key member handled a caller who brought their own credential, with the failure category (rate_limit, quota_exhausted, server_error, connection) that moved the request. The org key is billed for these requests. Stored as audit_events.credential_fallback (the failure category, empty when unset); a recorded failover to an org-key member has upstream_failovers.credential_fallback set to 1.
object.llm.upstream.route (CEL)	The route the request was routed through. object.llm.upstream.pool is the same value.
object.llm.upstream.name (CEL)	The provider the request was sent to.

Configuration file

A deployment whose providers come from config.yaml or Helm values declares providers under llmProxy.upstreams and routes under llmProxy.pools. AIControls converts them to providers and routes on every start: only a pool's model patterns route, and match.groups, match.labels, match.sources, match.namespaces, match.serviceAccounts, and priority are ignored, with a startup log line for each. The first route you save in the console or the API stores that conversion next to your route, and the stored routes decide routing from then on.

llmProxy:
  upstreams:
    - name: anthropic-prod
      provider: anthropic
      apiKeyEnv: ANTHROPIC_API_KEY
    - name: bedrock-us
      provider: aws-bedrock-anthropic
      url: https://bedrock-runtime.us-east-1.amazonaws.com
      auth:
        type: aws-sigv4
        region: us-east-1
  pools:
    - name: claude
      enabled: true
      match:
        models: ["claude-*"]
      strategy: weighted
      members:
        - upstream: anthropic-prod
          weight: 3
        - upstream: bedrock-us
          weight: 1

Key	Description
llmProxy.upstreams[]	Providers. name, provider, url, apiKey / apiKeyEnv / apiKeyFile, auth, match.models (the provider's own route), staticModels, modelDiscovery, modelAliases. An entry with no key and no auth is a passthrough provider.
llmProxy.pools[].name	The route name.
llmProxy.pools[].enabled	Set true explicitly; an entry without it is disabled.
llmProxy.pools[].match.models	The route's model patterns.
llmProxy.pools[].members[]	upstream (provider name), fallbackOnly, weight, wireFormat.
llmProxy.pools[].strategy	failover, round-robin, lowest-cost, lowest-latency, or weighted.
llmProxy.pools[].strategyOptions	For lowest-latency: windowMinutes, minRequests, explorationShare.
llmProxy.pools[].allowTranslation, switchFamilyOnQuota	Cross-provider members and family switching.
llmProxy.pools[].modelAliases[]	name and wireModels (provider name → model).
llmProxy.sessionPinTTLSeconds	How long a session's provider-family pin lasts. Default 86400.
llmProxy.circuitBreaker	enabled, failureThreshold, cooldownSeconds. See Circuit breaker.

An invalid route, such as an overlap at the same specificity or an unknown strategy, fails startup with a message naming it.

llmProxy.routes and llmProxy.providers declare routes and providers in the routes format. They don't route: a llmProxy.routes entry is listed read-only on the Routes tab (source: config) and reserves its name and model patterns, so a route you save can't take them (a route converted from upstreams and pools is stored even when its patterns overlap, with a warning naming the entry), but its members name llmProxy.providers entries, which the router doesn't build. AIControls logs a warning at startup when either is set. Route with llmProxy.upstreams and llmProxy.pools.

See also

• Model Routes
• Add a route
• Steer a team through a route