Cost Governance

Concepts

Cap AI spend per model family, per team, or per user, and reduce the token usage behind that spend. Budget rules are enforced at request time against a fixed calendar window (hourly, daily, weekly, or monthly); optimization techniques reduce the tokens each request consumes in the first place.

Overview

The Cost → Budgets page has four tabs — Overview, Rules, Limits, and Alerts. Overview summarizes spend for the selected month, which you change with the previous and next month buttons and the month picker in the page header: total spend against your global limit, projected month-end spend at the current burn rate, rules approaching their limit, and a Recent enforcement feed of the requests a budget rule denied, held, warned on, or audited. Each row names the rule that fired (its scope, model match, and mode) and the share of its limit that was used at the moment of the decision, as recorded with the request: later edits to the rule, its mode, or its spend do not change an earlier row. A row recorded without that detail shows Not recorded instead of a guessed rule. Allowed requests are not listed.

Spend totals — Period Spend on the Budgets Overview and Total Cost on the Cost dashboard — count every priced LLM call through the proxy: calls a developer makes, agent calls a developer delegated, and calls an agent makes on its own with no developer behind it. Each call counts once, so a delegated agent call appears under its developer and is not added again for the agent. On the Budgets Overview, the By team breakdown lists spend that belongs to no team (developers without a team and agent calls with no developer) as ungrouped.

Cost Intelligence

The Cost → Cost Intelligence page summarizes spend, budgets, attribution, and waste:

• The summary cards (spend, budget utilization, and the others) show Couldn’t load with a Retry button when their data fails to load, never a zero. A figure that is missing from the response is treated the same way. If one card or chart cannot display its data, only that card shows a notice with Retry; the rest of the page stays. Budget Utilization shows No budget set with a link to Budgets when no budget exists.
• Budget vs actual lists the team and organization budget rules from Budgets → Rules, most used first, with each rule's spend in its current window: the same figures the Rules tab shows and the enforcer counts. For monthly and weekly rules it projects spend to the end of the window at the rate so far, once a tenth of the window has passed. User, agent, and token rules are on the Budgets page.
• Cost Attribution leaves out anonymous identities (anon: IDs and the anonymous caller). A column with more than 7 nodes shows its 6 largest and combines the rest into Other (N). When the underlying flow has 50 or more nodes, the diagram is replaced by a table of the largest flows, largest first, with the same Other (N) rows. A line such as Top 7 of 56 nodes says how much is hidden. The Teams view counts the 500 highest-spend named members.
• Spend by user shows the 10 largest spenders, then one muted gray Other (N) row that sums the rest, with a line such as Top 10 of 143 users. View all opens the full list.
• Model distribution shows one tile per model up to 10 models. With more, models are grouped by provider first, the groups are capped at 10 with Other (N), and a line such as Top 10 of 37 models states the cap. Select a provider to list its models, ranked by spend and capped the same way.
• Waste breakdown estimates spend in four patterns. For model over-selection, it points to route model aliases, which route a client-facing model name to a cheaper model. Repeated context and large file inclusion are informational. OpenAI caches prompts automatically and Claude Code marks its own prompts for caching. Settings → Prompt Caching → Auto-inject cache_control is only for API clients that call the Anthropic Messages API without cache_control (on Anthropic, Amazon Bedrock, and Google Vertex upstreams); requests that already set it are unchanged.

Default model pricing

Every dollar figure on the Budgets and Cost Intelligence pages is computed from a per-model cost table — prompt and completion price per 1M tokens, plus cache-read (and, where applicable, cache-creation) rates for models that support prompt/context caching. AIControls ships default pricing out of the box for common Anthropic, OpenAI, Google Gemini/Vertex, Fireworks AI, and z.ai (Zhipu) GLM models. Fireworks models are keyed by their full namespaced ID (accounts/fireworks/models/<name>), matching what the provider serves.

A model that isn't in this table still has its real token usage tracked, and is priced with a conservative fallback rate rather than $0 — this shows up most often right after a new model release, before its pricing has been added, and keeps budget rules and spend limits from being silently bypassed by an unpriced model in the meantime. The fallback rate is intentionally on the high side (matching AIControls' priciest known frontier-model tier, including cache rates), so it errs toward stopping spend too early rather than letting it run unchecked — expect it to overstate the true cost until real pricing is added. This applies to budget enforcement and to the Budgets/Cost Intelligence totals, which are both derived from the same per-request cost recorded at request time; the Cost dashboards and the per-developer cost from Observer telemetry use the same lookup and the same fallback rate, so they no longer show an unpriced model at $0. To add or correct pricing for any model, go to Settings → Model Costs and set its prompt/completion price per 1M tokens (and cache pricing fields, if applicable); once added, new requests are priced at the real rate immediately, and you can apply the price to past requests that were estimated at the fallback rate.

Models priced at the fallback rate

The model pricing page (Settings → Model Costs) shows a warning when any model had requests priced at the fallback rate in the last 7 days. The warning lists each model with its request count. Add Model opens the add form with the first listed model filled in; viewers see the list without the action. A model drops off the list as soon as it has a row in the table. Its earlier requests keep the fallback estimate until you apply the new price to them, as described in Applying a price to past requests. When no model was priced at the fallback rate, the page shows no warning.

The same data is available to admin and viewer tokens at GET /api/v1/ui/settings/model-costs/fallback. The optional days parameter sets the window (default 7, maximum 90). The response has windowDays and items, each with model, requests, firstSeen and lastSeen, ordered by request count. The response lists every model priced at the fallback rate in the window, including models that have since been added to the table.

Applying a price to past requests

Setting a price changes how new requests are priced. Requests that were already recorded at the fallback rate keep that estimate until you apply the price to them. This is optional, and nothing changes until you confirm.

Admins open the Apply price to past requests drawer in two ways. After Add Model saves a model that had requests priced at the fallback rate, the drawer opens on its own. At any time, the row menu of a model in the pricing table has Apply price to past requests. Viewers do not see either.

The drawer shows a preview before it writes anything: the number of requests priced at the fallback rate in the chosen look-back window (7, 30, 90, or 365 days), the date of the earliest one, and the estimated spend change from the fallback estimate to the cost at the saved price. Apply to past requests then updates those requests and reports how many were updated. Not now closes the drawer without changing anything.

What changes and what does not:

• Only recorded requests change: the stored cost of each LLM request for that model that was priced at the fallback rate, within the window. Each updated request records cost-table as its price source, so it no longer counts as fallback-priced. Requests priced from the table, the vendor price table, or a local model, and requests of other models, are not touched. A request recorded with a cost of zero is not touched.
• The cost views follow the recorded requests. Cost Intelligence, the Usage and cost dashboards, and exports show the corrected cost, and their totals, daily series and per-session or per-project figures stay equal to the sum of the updated requests.
• Budgets, session spend and virtual-key spend are not changed. Budgets were enforced at the price known at the time, so a team that was blocked at the fallback estimate stays recorded as blocked, and spend counters are not rewritten.

Applying is safe to repeat. It updates requests in small batches, and a request that was already updated is never changed again, so running it twice, running it from two admin sessions, or running it on several gateway replicas at once changes each request once. If the connection drops or a batch fails, the drawer reports how many requests were updated and Try again continues from there. Closing the drawer while it runs does not undo the requests already updated.

Each apply call that changes requests adds an entry to the configuration audit with action model_price.applied_to_past_requests. The entry records who applied it, the model, the window, the fallback and new prices, and how many requests it updated.

The preview and apply endpoints are described in the Model pricing API.

One exception: an OpenRouter model whose ID ends in :free, such as nvidia/nemotron-3.5-lightning:free, is priced at $0 instead of the fallback rate when an OpenRouter upstream serves it, because OpenRouter doesn't charge for it. The served model decides: a Model Aliases entry that sends another model name to a :free model is priced at $0, and one that sends a :free name to a paid model is priced as the paid model. On any other provider, such as a custom upstream, a model ID ending in :free gets its cost-table row or the fallback rate. A cost-table row for that exact ID still takes precedence.

Finding and reading rows in the pricing table

The model pricing table lists every model with its input, output, cache-read, and cache-write price per 1M tokens. Use it to find a row or see where a price came from:

• Search models matches any part of the model ID.
• Provider and Source filters narrow the table to one provider or one source. The count above the table reads "Showing N of M models".
• Click a column heading to sort by it. Click again to reverse. Without a chosen sort, rows are grouped by provider and then ordered by model.
• Each row has a source tag. Default means the row still carries the pricing that ships with AIControls. Custom means an admin added the model or edited its prices, or the row has no recorded source and its prices differ from the bundled table. Refreshed means the price came from the pricing feed. Fallback-priced means no row matches the model, so its recent requests were estimated at the fallback rate.
• Fallback-priced models appear at the top of the table whatever the sort, with a dash in each price column (an unknown price is never shown as $0) and their request count. Add Model on that row opens the add form with the model ID filled in. Viewers see the rows without the action.
• Search, filters, sort, and page are kept in the page address, so a filtered view can be shared or bookmarked.
• The table scrolls sideways inside its card on narrow screens; the Model column stays in view. When a filter matches nothing, Clear filters shows every model again.

How a model name is matched to a price

Clients spell the same model more than one way. GitHub Copilot sends claude-haiku-4.5, the cost table has claude-haiku-4-5, and cloud providers add date suffixes such as @20251001 or -20251001. AIControls finds the row in this order:

1. An exact match. A row you add under a name always wins, so an explicit dotted row such as claude-haiku-4.5 in Settings → Model Costs is used for that spelling.
2. A normalized match. Both the requested name and every row name are normalized the same way before they are compared: lowercased, a leading vendor prefix such as anthropic/ or github-copilot/ removed, a Bedrock -v1:0 and a date suffix (@20251001 or -20251001) removed, and every . changed to -. So claude-haiku-4.5, anthropic/claude-haiku-4.5 and claude-haiku-4-5-20251001 all use the claude-haiku-4-5 row, and gpt-4.1, gemini-2.5-pro and o3.mini still use their own rows.
3. A dated snapshot of a model with a row. A name that is a known row plus a snapshot suffix, such as gpt-4o-2024-08-06, uses that row.

A point release needs its own row. claude-opus-4.8 uses the claude-opus-4-8 row and never the claude-opus-4 row, and a point release with no row is not priced as its base model. It gets the fallback rate until a row is added.

A model that matches none of these is priced at the fallback rate: $30 per 1M prompt tokens, $180 per 1M completion tokens, and the highest cache rates. The Budgets pages, the Cost dashboards and the observed-usage backfill all use that same rate, so an unlisted model is never shown at $0 on one page and priced on another. Two exceptions stay at $0: an empty model name, and a :free model on the Cost dashboards (the per-call price recorded by the proxy is used for OpenRouter :free models).

Scheduled price refresh

Refresh on the model pricing page fetches current prices from the public LiteLLM pricing feed once, on demand. To keep prices current without that step, turn on Refresh prices automatically on the same page and choose how often it runs: every 6 hours, every 12 hours, daily, or weekly. The setting is off by default. The interval is between 1 hour and 30 days.

The scheduled refresh updates models whose prices came from a refresh or still match the bundled defaults, and adds newly listed models from the known providers. It never overwrites a price an admin set. A price counts as set by an admin when you add a model or edit its prices on the page, or when it differs from the bundled default and has no recorded source. Those models are counted as kept in the last-run summary. The Refresh button is different: it replaces the prices of every model that appears in the feed, including edited ones.

The page shows the result of the last automatic run: when it ran, how many models were updated, and how many manual prices were kept. A failed run shows its error and leaves the table unchanged. With several AIControls replicas, only one runs each refresh. The run is also listed as model_price_refresh in the background-job health view.

Models served by an OpenAI-compatible vendor

A call served by a vendor catalog upstream, such as DeepSeek, Groq, or Moonshot, is priced by the vendor and the model the vendor served, not by the model name the client sent. AIControls looks up the price in this order:

1. A cost-table row keyed <vendor>/<model>, such as groq/openai/gpt-oss-120b, that you add in Settings → Model Costs.
2. The bundled vendor price table, which covers every priced chat model of every catalog vendor in the LiteLLM pricing feed and is refreshed weekly with the default pricing.
3. The cost table, by the served model ID, then by the requested model name, then the fallback rate.

A local vendor (Ollama, vLLM) is always priced at $0. A vendor: custom endpoint has no catalog prices: price its models with cost-table rows keyed by its provider id and model, such as custom:llm.example.com/inhouse-2; otherwise step 3 applies.

Budgets stay keyed by the model the client requested. When a Model Aliases entry sends claude-sonnet-4-6 to a vendor model, the call counts against the claude-sonnet-4-6 budget at the vendor model's price.

Each audit event records where its price came from in pricingSource: vendor-table, cost-table, local, or fallback. An OpenRouter :free model priced at $0 records vendor-table.

Models served through a cloud provider

Amazon Bedrock and Google Vertex address the same models under their own IDs — Bedrock uses an inference-profile ID such as us.anthropic.claude-sonnet-4-5-20251001-v1:0. Cost is attributed against the canonical model name (claude-sonnet-4-5), not the provider-specific ID, so:

• You price each model once. There's no need for a separate cost-table entry per region or per inference profile.
• Bedrock and Vertex traffic rolls up together with direct-provider traffic for the same model on the Cost Intelligence pages, rather than appearing as extra rows.
• Prompt caching is applied to Bedrock and Vertex the same way as to the direct provider, so cache-read and cache-creation tokens — and the savings they produce — are attributed there too.

How tokens are counted

Every priced LLM call is stored with the same token split, whatever the provider reports:

Field	Meaning
input_tokens	Uncached input only. It excludes tokens read from or written to the prompt cache.
cache_read_tokens	Input tokens served from the prompt cache.
cache_creation_tokens	Input tokens written to the prompt cache.
output_tokens	Completion tokens.

The estimated cost is input_tokens at the prompt rate, plus cache-read and cache-creation tokens at their own rates, plus output_tokens at the completion rate. Each token is priced once. OpenAI-format providers, including GitHub Copilot, report cached tokens inside their prompt total, so AIControls subtracts them before pricing. Anthropic, Google Gemini, Amazon Bedrock, and the OpenAI Responses API are converted the same way. Add the three input fields to get the full prompt size: context pressure and the OpenTelemetry gen_ai.usage.input_tokens attribute both use that total.

Spend for cached OpenAI-format and Copilot calls was overstated before this fix

Earlier versions priced the cached part of an OpenAI Chat Completions or GitHub Copilot prompt twice: once at the full prompt rate and again at the cache-read rate. Estimated spend for calls with cached tokens was too high, by roughly 4x to 7x for a prompt that is almost entirely cached. On upgrade, AIControls recomputes the stored input tokens and estimated cost of the affected historical calls in the background, and adjusts the Cost, Activity, and identity statistics that were built from them. It does this once, in small batches, and never changes a call recorded after the upgrade. Calls that AIControls translated between API formats are left as recorded. The correction subtracts the cached tokens at the model's current prompt price, so a model whose price changed since the call is corrected approximately. Budget and virtual-key counters that already accumulated the overstated amounts are not rewritten: they reset with their window.

Observed model inventory

Inventory → Models lists every model that has actually been called through the proxy — sourced from LLM request traffic, not a static catalog — with first/last-seen timestamps, call count, and total cost. Each row also shows whether the model is subject to a global model allowlist configured under Settings → Models → Governance:

Allowlist state	Meaning
Unrestricted	No global model allowlist is configured — any model may be called.
Allowed	A global allowlist is active and this model is on it.
Not allowlisted	A global allowlist is active and this model is not on it — calls to it should be reviewed.

Use this page to catch models being called that you didn't expect — a developer routing through a provider directly, or a new model release nobody's added pricing for yet (see Default model pricing).

Rules

Rules define spend caps. Each rule matches a scope (organization, team, or individual) and a model glob, and caps spend within a fixed calendar window:

Field	Description
Scope	Global (all identities), Team (a named group), or User (a specific identity) — plus Agent and Token scopes, configurable via the API
Target	The team name or user email when scope is Team or User; * for Global
Model match	Glob pattern — e.g. claude-opus*, gpt-4o, or * for all models. First-match within each scope.
Limit	Maximum spend in USD for the window
Window	Hourly (1h), Daily (24h), Weekly (7d), or Monthly (30d) — each resets on a fixed calendar boundary (top of the hour, midnight UTC, every Monday, or the 1st of the month), not a trailing period counted back from now
When limit is reached	See enforcement modes below
Warn at	Percentage of the limit at which a budget_warning notification fires — once per rule per window — while spend stays under the hard limit. Requests above the threshold are still allowed and recorded as warnings. Default: 80%.

Enforcement modes

Mode	What happens when the limit is hit
Deny	Requests are blocked until the window resets. The caller receives a clear error with the reset time. Administrators get a budget_exceeded notification.
Audit	Requests are allowed, but would-have-blocked amounts are tracked so you can preview a rule's impact. Promotes to Deny in one click.
Warn	Requests are allowed and recorded as a warning. Useful for awareness without blocking.
Require approval	Requests are held for a manager to approve or deny in the Approvals queue. Administrators get a budget_exceeded notification. On the native cloud-provider endpoints (Amazon Bedrock and Google Vertex AI request formats), a Require-approval model-budget rule is enforced as a denial instead of a hold.

When a Deny or Require-approval rule starts blocking or holding requests (or, on the native cloud-provider endpoints, denying them), the budget_exceeded notification fires once per rule per window — not once per blocked request, and only once across all replicas. It names the scope (organization, team, user, agent, or token), the model match, the window, the spend against the limit, when the window resets, and which identity tripped it, and is delivered to the in-app inbox plus any channel enabled under Settings → Notifications → Budget Exceeded (email, Slack, Teams, webhook). The same notification covers per-agent token budgets on hourly/daily/weekly/monthly windows. The exception is the 1-minute tokens-per-minute (TPM) rate limit: hitting it fires rate_limit_exceeded, not budget_exceeded, since it protects against short bursts rather than tracking cumulative spend.

note

Rules are AND-enforced: a request must pass the first matching rule in every applicable scope (global, team, user, agent, token). If multiple rules match a request, all of them apply.

The Effective budget inspector on the Rules tab answers "what would happen to the next request" — type any identity and model to see which rules bind, the remaining headroom, and the resulting decision.

For step-by-step instructions on creating a rule, see Set a Budget.

Limits

Independent of the per-model budget rules above, a default session cap sets the maximum spend (USD) for any single session. When a session hits the cap, further requests are blocked until the session ends or an admin resets it. A session keeps the cap it was created with, so changing the default affects only sessions created afterwards unless an admin applies it to existing sessions. The cap is enforced for every credential type, including portal aic_ tokens and passthrough, and each session is counted separately; see Session budgets. A changed default can be applied to existing sessions with a preview, for active sessions, ended sessions or both; see Apply a changed default to existing sessions.

Alerts

Alerts notify without blocking:

• Budget warning thresholds — each percentage threshold (e.g. 75, 90) fires a budget_warning notification at most once per window per rule. It's an administrator-facing notification delivered to any configured email, Slack, or Teams channel; it does not create an in-app inbox entry. Per-model rules use their own Warn at percentage (above) for the same notification, which is delivered to the administrators' in-app inbox as well as those channels. Either way it names the developer, team, or agent that tripped it, but does not land in that individual's own inbox or email.
• Session spend alert — a USD threshold that fires an in-app notification the first time a session's spend crosses it. Separate from the session cap — it only notifies.

note

For conditional enforcement near a budget threshold — for example, requiring a ticket ID once spend exceeds 80% — use a CEL policy with object.modelBudget.team.pctUsed.

Context Optimization Preview

AIControls intercepts every tool response before it reaches the LLM and applies techniques that reduce token usage at the source:

Technique	What it does
Noise filtering	Strips ANSI escape codes, progress bars, and terminal noise from shell tool responses before they are forwarded to the LLM.
Response deduplication	Suppresses exact duplicate tool responses within a session using a content-hash cache — repeated results are replaced with a short reference instead of the full text.
Brevity injection	Appends a conciseness directive to requests. Three modes: Off, Auto (applied when context pressure is detected), Always. A custom prompt can override the built-in instruction.

A custom brevity prompt can override the built-in instruction (default: "Be concise. Omit preamble and trailing summaries. Prefer code over prose.").

note

Brevity injection is the highest-leverage technique for most teams — the Context Savings dashboard in the demo shows it accounting for 12M+ tokens eliminated from 29,000 events with a single setting.

For configuration steps, see Optimize Cost. For how compression — the fourth technique — reduces tool-response size and keeps originals retrievable, see Context Optimization.

Compression Preview

Compression reduces the size of tool responses before they enter the LLM context window, independent of the context optimization techniques above. It applies both to MCP tool responses and, optionally, to built-in tool results (Bash, Read, Grep, and similar) embedded in your LLM requests. See Context Optimization for how compression modes, rule precedence, context-pressure escalation, and lossless retrieval work.

Setting	Description
Default mode	Compression level applied to all tool responses. ML (model-based) achieves ~90–95% size reduction by summarizing content semantically. Additional modes may be available depending on your workspace configuration.
Code-aware (AST) compression	When enabled, source code responses are compressed using an AST-aware pass that strips function bodies and retains signatures, imports, and structure. Higher quality than generic ML compression for code. Requires the Compression Service to be reachable from your workspace.
Threshold	Minimum response size in bytes before compression is attempted. Responses smaller than this value are returned unmodified. Default: 4096 bytes.

note

Compression is an experimental feature and isn't enabled by default — contact your Nirmata account team to have it turned on for your workspace. Once enabled, results vary by tool and content type, so monitor the Context Savings dashboard to confirm the savings.

For configuration steps for both features, see Optimize Cost.

Context Savings dashboard

Cost → Context Savings shows what the active optimization techniques are actually saving: the token reduction rate across sessions, estimated cost saved, a breakdown of savings by technique, and the sessions that benefit most. Use it to verify a newly enabled technique is working and to find workflows where further tuning would pay off.

See also

• Cost Overview for the Home → Cost dashboard and its 24-hour, 7-day, and 30-day windows
• Set a Budget
• Optimize Cost
• Context Optimization