Add a Route

Tasks

Create a model route that sends requests for its model patterns to an ordered list of providers. For how routes are chosen and how failover works, see Model Routes.

A provider in no route already serves its own model patterns, or its family's defaults, through an implicit route. Add a route when a second provider should serve the same models, when you want failover or a strategy, or when a team should reach a different set of providers through an Access Binding grant.

Create a route in the console

1. Add the providers you want as members first. See Add a provider. A route references providers by name; it doesn't hold connection details.

2. Go to Settings → Providers, switch to the Routes tab, and select Add Route. The route editor has five steps: Match, Members, Routing, Aliases, and Review. How requests route beside the form previews, for a model you pick, which member serves a new session, which members come next, and which are excluded.

   • The Request model list starts on a model the route serves and groups models into Served by this route and Other models. For a model the route doesn't match, the panel names the route that serves it, or says that no route matches and requests fail closed with NO_ROUTE.
   • A request-flow graph shows new sessions going from New sessions to the members as the strategy sends them, with a count per member. Under each member it shows the provider, weight, list price per million tokens, or P95 latency over the last hour, by strategy. Members with too little traffic show as cold. The flow plays for about 15 seconds after each change; use Play, Pause, and Reset to replay it. With reduced motion turned on in the operating system, the graph shows how 20 new sessions split instead.
   • To widen or narrow the panel, drag the handle between the form and the panel, or focus it and press ← or →. The browser remembers the width. On a narrow screen the panel is below the form.

3. On the Match step:

   Field	Value
   Name	Any label, e.g. prod-anthropic. Names are unique across routes and providers.
   Model patterns	The models the route serves, such as claude-* or claude-opus-*. Exact names and globs with a * at the start or end are allowed. At least one is required.

   Who uses this route lists, read-only, the Access Bindings whose grants reach the route and the subjects they apply to. To change it, select Open Access Bindings.

   If another route already uses an overlapping pattern at the same specificity, saving is refused: each model pattern belongs to one route. Edit that route instead. To send only some callers to this route, give it a pattern that matches the same models but is less specific, such as claude* next to another route's claude-*, and name this route in their binding grant (see Steer a team through a route).

4. On the Members step, select Add member for each provider, in the order you want them tried. For each member:

   • Pick the provider in Select provider. A disabled provider can't be a member.
   • Check the tag: by default, org key members serve every caller except those who bring their own credential, which only passthrough members serve. A route that should serve callers signed in to a Claude or ChatGPT plan needs a passthrough member. With Fall back to org key members for callers' own credentials on (on the Routing step), those callers start on passthrough members and fall back to org key members after every passthrough member fails with a rate limit, quota, or provider error.
   • Watch the compatibility indicator. A member whose wire format conflicts with the members above it is flagged. An Azure OpenAI or Azure AI Foundry member needs Set wire format… before it's included.
   • Check Fallback only — never selected as the first choice for a member that should serve only after every other member, such as a slower or more expensive region.

   The number beside each member is its position in the actual try order, with Fallback only members last. Use the arrow buttons to reorder members, or the remove button to delete one. A route needs at least one member.

   To spend vendor credit before paid usage, list the vendor providers first and the paid provider last. When a vendor reports its credit or quota is exhausted, the request fails over and the exhausted member is parked until its quota resets.

5. On the Routing step:

   • Strategy: Failover (the default), Round robin, Lowest cost, Lowest latency, or Weighted. For Weighted, set each member's weight under Weights; an empty weight is 1, and 0 never serves first. See Strategies.
   • Allow cross-provider members: mix provider families, such as Anthropic and OpenAI members. Each session stays on the family that serves its first request. See Cross-provider routes.
   • Switch family when credit runs out: applies only while cross-provider members are allowed, and is on by default. See Switching family when quota runs out.
   • Fall back to org key members for callers' own credentials: off by default. When on, a caller who brings their own credential, such as Claude Code signed in to a Claude plan, moves to the route's org-key members after every passthrough member fails with a rate limit, quota, or provider error. The org key is billed for those requests. Applies only when the route has both a passthrough member and an org-key member. See Org key fallback for callers' own credentials.
   • Enabled: a disabled route doesn't route, and its members serve their own model patterns again.
6. Optionally, on the Aliases step, add model aliases: client-facing model names that select this route, with a model for each member. See Route a model alias.
7. On the Review step, check the summary and select Save; each row's Edit opens its step. If a member was excluded for a wire-format conflict, the save succeeds and names the excluded member so you can fix it. The router uses the route right away, on every replica.

Editing a route opens the same page, with every step open and Save on each one. To delete a route, open its row's actions menu on the Routes tab and select Delete. Deleting or disabling a route that an Access Binding grant names is refused with 409, naming the binding: change the binding's Via route first. The same applies to an implicit route a grant names: adding its provider to a route, or deleting or disabling the provider, is refused until the binding changes. Likewise, a provider that is a member of a route can't be deleted or disabled until you remove it from the route. Routes from the deployment's config file show config-managed and can't be edited or deleted in the console. Implicit routes show the implicit tag and have no actions: to change one, edit the provider's Model Patterns, or add a route that takes its patterns. Routes converted from the configuration file's pools show live (not stored) until you save a route: the first route you add, edit, or delete stores the deployment's current providers and pools as routes, and they take effect right away, without a restart.

Create a route through the admin API

curl -X POST "https://YOUR_WORKSPACE_URL/api/v1/llm-routes" \
  -H "Authorization: Bearer YOUR_ADMIN_TOKEN" -H "Content-Type: application/json" \
  -d '{"name":"claude","enabled":true,"models":["claude-*"],
       "strategy":{"type":"lowest-latency","options":{"windowMinutes":15,"minRequests":20}},
       "members":[{"provider":"anthropic-prod"},{"provider":"bedrock-us","fallbackOnly":true}]}'

POST returns 409 when the name is taken. PUT /api/v1/llm-routes/{name} replaces a route, and DELETE removes it. A save returns 400 for an overlapping pattern, an unknown or disabled member provider, or an invalid strategy. See Model Routes reference.

Check how requests are routed

1. In Settings → Providers, select Test routing.
2. Enter a Model, and optionally a Team and a User email for the caller. The dry run uses the saved configuration and sends nothing upstream.
3. Read the result: model → route → provider, then the members tried next after Then:. The line below names the grant: Granted to everyone: no Access Binding grants models yet., or Granted by binding with the binding's name and level, and where the request would go without a grant that names a route. A model no route serves shows denied with the closest route; a caller with no grant shows the no-grant denial.

To confirm a route receives traffic, open Setup → Health → Upstreams. Routes with two or more providers appear under Failover, each with its active member, healthy members, requests, error rate, and Failovers (24h). The Providers table lists every provider with the routes it serves and its traffic in the selected window. See Monitor health.

See also

• Model Routes
• Steer a team through a route
• Model Routes reference