Runtime Controls

Concepts

A gateway records what arrived. No field in an audit log can represent a call that never arrived — a clean audit and a workload routing its traffic around AIControls produce the same report. Runtime Controls adds a second, independent vantage point: an eBPF sensor on every node that observes what a workload's pods actually did, so the two views can be compared.

Self-managed installations only

Workload Trust is available only in private, self-managed AIControls installations, on premises or in a cloud cluster that you run. It needs a dedicated cluster that you run, with Kyverno, Kyverno Runtime, and cluster OIDC and RBAC access in place. Nirmata-hosted (SaaS) workspaces don't include it. See Feature availability.

note

Runtime Controls is available on self-hosted Kubernetes only, is on by default there (runtimeControls.enabled defaults to true), and is marked experimental in the admin console. See Availability below and Enable Runtime Controls for setup.

Availability

Runtime Controls needs two things that only exist in a cluster you operate: an API server it can read Report objects from, and a cluster it can author policy in for the sensor to act on.

On a Nirmata-hosted (SaaS) tenant, the AIControls control plane runs in Nirmata's cluster, not yours. There is no customer API server to read from and no customer cluster to author policy in, so the integration cannot function there — and asking for cluster-wide pod and workload read permission inside a multi-tenant control plane would be the wrong thing to request even if it could.

The gate therefore fails closed on the environment, not just the flag: setting runtimeControls.enabled: true on a SaaS tenant still does not start the integration. When it is not running, the Runtime page and its API endpoints do not error or render blank — they state the reason ("unavailable on Nirmata-hosted (SaaS) tenants: Workload Trust requires a self-managed installation", "not running in Kubernetes", "disabled by config").

Attestation, not detection

Most workload-security tooling infers: it guesses what a workload probably is from names, images, and environment variables, scores the guess, and alerts. AIControls' own workload discovery does exactly this, and it is guesswork by construction — a Deployment named payments-svc running an LLM client library is invisible to it, and a static site called ollama-docs is a false positive.

Runtime Controls is built the other way round. It does not ask "does this look like an AI workload?". It asks "what did this workload's pods demonstrably do, at the kernel, and does that match what governance saw?" — a question with evidence behind every answer. The output is not a probability. Each row cites the specific destinations, the specific evidence class that produced them, and the specific reason anything else was excluded.

Static workload discovery and runtime evidence are independent signals. A workload does not have to match the discovery heuristics to produce runtime findings, so absence from Inventory does not imply absence of runtime AI activity. Check the Runtime tab under Health and its observation coverage as well as Inventory; neither an empty inventory nor missing observations establish that a workload is governed.

The trade-off is that evidence has edges. The rest of this page is mostly about where those edges are, because a reader who does not know them will over-read a clean table.

What the kernel plane can and cannot see

The sensor is a per-node DaemonSet with five observable behaviors. Three of them need nothing beyond the sensor; two need the open/exec hooks.

Behavior	Observes	Requires the open/exec hooks
network	Destination addresses a pod connected to	No
protocol	The application protocol a flow speaks (tls, dns, quic, ssh, …)	No
dns	Names a pod resolved — observe-only, never enforceable	No
open	Absolute file paths a pod opened	Yes
exec	Absolute binary paths a pod executed	Yes

The open/exec hooks need bpffs mounted at /sys/fs/bpf on each node. Most distributions mount it by default; kind and some minimal node images don't. The sensor attaches BPF-LSM programs when the node's kernel boots with BPF-LSM, and otherwise attaches a program on security_file_open, which also enforces. It picks once at startup and logs the choice. On nodes without the hooks, open and exec findings cannot be produced at all — not "none were seen", but "this class of evidence is structurally unavailable here". The Runtime tab under Health reports this per node rather than averaging it away, and reports it as three states, with unknown never collapsed into unavailable (see the capability contract).

AIControls authors monitor-mode observation policy for four of these behaviors — dns, network, open, and exec — across every namespace except the excluded namespaces and AIControls' own namespace. The first two need no open/exec hooks and are what the headline number is built from. Both halves are observed: the question a workload asks, and the connection it opens. They are separate facts, and every destination on a row says which one produced it.

Four further limits are structural, not bugs to wait out:

• Nothing reads inside TLS, and there is no SNI, so a connection yields an address rather than a name. AIControls resolves the in-cluster half of that from the Kubernetes API server. For an external address the pod never resolved in the clear, no name can be established, and the destination is rendered honestly as unattributed (ip:203.0.113.9 (unattributed)) rather than guessed. This is now the sharpest limit on the feature — the connection is seen; naming it is what falls short.
• Ingress is out of scope. Every hook is egress or execution, so a pod serving a model, an MCP server, or an agent card is not observed on the serving side.
• Read findings as evidence about IPv4 alone. kyverno-runtime v0.1.3 and earlier are IPv4-only, so with them a workload that reaches a provider over IPv6 contributes nothing to any row, and nothing records that it was out of reach. kyverno-runtime v0.1.4 and later, the version the chart bundles, also observe IPv6 destinations, but AIControls doesn't yet compile IPv6 values or verify how IPv6 findings are attributed (#3452). The product can't tell "no IPv6 traffic occurred" apart from "IPv6 traffic occurred and was never seen", so an empty result is evidence about IPv4 and nothing else.
• Findings carry counts, not ordering. network, protocol, open, and exec observations are drained from kernel counters on a poll interval; only dns questions are streamed as they happen. So a connection can lag the behavior that caused it by up to one interval.

Finding categories

AIControls groups every Runtime Controls finding into one of four categories, and you can turn each one off independently. Each category names what the sensor observed, not what the workload lacks.

Category	What it observes
Ungoverned LLM	A pod accesses an AI SDK, a model weight file, or an agent's credential file — evidence that AI usage exists on the workload at all
Agent Exec	A pod executes an agent CLI (for example claude, codex, gemini) or a local inference server (for example vllm, sglang)
MCP Exec	A pod executes an MCP server package
Ungoverned Egress	A pod resolves a name, or opens a connection, outside the governed path — the dns and network evidence together

Ungoverned LLM, Agent Exec, and MCP Exec all come from the kernel plane's open and exec observations described above; Ungoverned Egress is the dns and network observations that back the bypassed-destinations count in the rest of this page.

A pod opening a broader set of infrastructure credential files — ~/.aws/credentials, .env, ~/.kube/config, an SSH private key, and similar — is a second open observation within Ungoverned LLM, toggled by its own runtimeControls.categories.credentialFiles key (default enabled) rather than by ungovernedLLM. It exists because ungovernedLLM's own open filter only names six agent-CLI credential files (Claude Code, Codex, Gemini CLI, and similar), not general infrastructure credentials. Like every other observation, it is monitor-only and requires the open/exec hooks — see What the kernel plane can and cannot see above. See the reference for the default path list and how to override it.

Renamed in this release

These categories shipped as Shadow LLM, Shadow Agent, Shadow MCP, and Shadow Egress. The names now describe the observation, and "Shadow AI" is reserved for an ungoverned AI workload. The Helm keys and the API category values changed with them.

AIControls renames findings already recorded under the old values when it starts, so no history or counts are lost. Anything that asks for a category by its old name stops matching, though — a saved link, a dashboard, or a script that passes category=shadow-llm returns nothing until you update it. See the reference for the full mapping.

You disable a category with a runtimeControls.categories.* key in the Helm chart's values — see the reference for the exact keys. Turning a category off does two things: AIControls stops authoring that category's observation policy, and it removes the corresponding RuntimePolicy object already on the cluster on the next reconcile tick, rather than leaving it in place unused. Finding history is untouched. Disabling a category stops new observations for it; it does not delete any row already recorded.

Reviewing findings

Findings appear as runtime events in Security → Activity, filtered to the runtime plane, and on a workload's Runtime tab. A finding is one row per (workload, category, subject) combination — a Deployment resolving a bypass name, a StatefulSet executing an agent CLI — with its cumulative count and when it was last seen. A finding lands in Uncategorized when it can't be honestly attributed to one of the four categories, most commonly a RuntimePolicy AIControls did not author itself. See the findings endpoint reference for the exact field semantics.

Bypassed destinations — the headline

The number at the front of each row is Bypassed destinations: the count of distinct unsuppressed provider destinations this workload demonstrably reached outside governance — either by a kernel-observed connection, or by resolving the provider's name itself.

It is a count of a set of destinations, which is what makes it usable:

• Unit-free. It is not a volume, so it does not inflate with retries or deflate with connection reuse.
• Stable under caching. A workload that resolves api.example-provider.com once and then serves 10,000 requests from a cached answer still contributes exactly one destination.
• Honest at zero. A 0 always means the unsuppressed set was empty. It can never mean "a subtraction happened to clamp".

Connection evidence and resolution evidence feed the same set. There is no second headline number for connections, and each destination's Evidence tag records which class produced it.

Whether a connection and a resolution collapse into one destination depends on whether AIControls could name the connection:

• When it could — the sensor observed the pod's own DNS answer for that address — the connection and the resolution key on the same identity and contribute one destination between them.
• When it could not — DNS-over-HTTPS or DNS-over-TLS, a cached answer, or a hardcoded IP, which are precisely the cases connection observation was added for — the connection is unattributed, keys on its raw address, and counts separately from the workload's resolution of the provider name.

Two entries for what may well be one destination is the deliberate cost of not guessing. Nothing in the evidence proves that the name a workload asked for and the address it dialled are the same destination, so AIControls says "I could not name this" rather than manufacturing a join that the kernel evidence does not support. Closing the gap needs identity sources that are not implemented yet — not a correlation heuristic.

One consequence is worth planning around: identity resolution is window-wide. A naming observation made anywhere in the window — including by a different workload — names that address for every row in it, so whether a pair collapses can turn on evidence outside the row you are reading. A count can therefore move between windows without the workload's own behavior changing.

M and N are evidence, not terms in a formula

Reconciliation counts newly reported occurrences within the selected window, not each finding series' lifetime total. Repeated cumulative reports add nothing; an increased counter adds only its increase, and a newer lower counter is treated as a reset and contributes its new count. Pushed findings each contribute one occurrence. The window includes its start and excludes its end.

Cumulative reports do not include a timestamp for every occurrence. AIControls assigns each positive delta, including the first snapshot and a counter reset, to the report's latest observation time. Treat these as reported occurrence counts, not an exact event-time reconstruction. A later report never moves earlier deltas into a new window.

Occurrence history is retained for 90 days, independently of audit-event retention. History begins when this collection format is installed; AIControls does not reconstruct earlier lifetime counters. If your window reaches before retained history, the page shows Runtime history is incomplete for this window. A low count in that window is not evidence of no traffic. Lifetime observation views keep their cumulative counters.

Each row also shows two raw counters, and the temptation to subtract them must be resisted:

Column	Counts	Population
M (kernel)	Kernel events — connections opened plus names resolved — to unsuppressed external provider destinations	What the pods did
N (gateway)	Requests AIControls recorded for identities joined to this workload	What governance handled

These count different things over non-overlapping populations. The kernel counts connections; the gateway counts requests, and HTTP/2 multiplexing means one connection legitimately carries many requests. More fundamentally, in the proxied topology a governed call produces no workload-side kernel evidence at all — the gateway resolves and dials the provider on the workload's behalf, so that traffic leaves the gateway's pod, not the workload's.

M − N is therefore not a quantity. It is two different measurements of two different things with a minus sign between them. M and N are on the row because they are useful corroborating evidence — "this workload is busy on both planes" is worth seeing — and for no other reason. The admin console's column tooltips say so explicitly.

Governed overlap strengthens a finding; it never clears one

Expanding a row shows each destination, and any destination the gateway also carried traffic to is tagged also governed (n).

This is easy to misread as an excuse — "the workload does use the proxy for that host, so this is fine". It is the opposite. Because a governed call leaves no workload-side kernel evidence, direct evidence to a host the workload also reaches through the gateway means the traffic is split: the workload knows and uses the governed path, and something in it — most commonly a library that ignores HTTPS_PROXY — also goes around it. That is a stronger signal than a workload that was never configured at all, not a weaker one.

Governed volume never subtracts from the destination set. A destination with a hundred governed calls and one direct connection still counts as one bypassed destination.

The annotation is only as complete as the gateway-side record it draws on. Destination hosts are recorded by the egress plane, so the tag only appears where network egress filtering is enabled. Its absence means "no gateway record of that host", which on a cluster without the egress plane is uninformative rather than reassuring.

Two things a clean table does not prove

A resolution is not a connection — and the row tells you which one fired

A dns finding proves that a name was looked up. It does not prove that any traffic followed. Code that resolves a hostname during start-up and then fails to connect, a resolver pre-warming a cache, a library enumerating candidate endpoints — all produce a DNS question with no connection behind it.

That is a statement about DNS evidence, not a limit on what AIControls can see. Connections are observed directly. So the table carries both classes and grades them, at both of its levels: each destination carries an Evidence tag of network or dns, and a row whose entire unsuppressed evidence is DNS is badged dns only.

Read the two badges differently:

• A network-sourced destination means the pod opened a socket to it. Traffic happened.
• A dns only row means "this workload asked where a provider lives" — worth investigating, but it does not establish that anything was sent.

The practical consequence is that a row losing its dns only badge is a materially stronger finding than the same row carried before, not a cosmetic change: it is the difference between intent and traffic.

The badge is a statement about a row that has unsuppressed evidence, though, so its absence alone is not the stronger reading. A row with no unsuppressed evidence at all is also not dns only — and since the network observation gives nearly every pod in the cluster a row, most rows are exactly that. Read the badge next to Bypassed destinations: no badge and a non-zero count is the connection-backed finding; no badge and a zero count is simply a quiet row.

A quiet table is still not a clean bill of health

The reverse error is more dangerous, because it makes an empty table look like a result. Connection observation closes most of what used to make this claim broad, so it is worth being precise about how much is left.

Largely closed. These three all used to produce a clean-looking workload that was actively reaching providers. All three are now observed — not as a name, but as a connection, which is the part that matters for the count:

Situation	Was	Is now
DNS-over-HTTPS / DNS-over-TLS	Invisible — the question is inside an encrypted session	The lookup is still unreadable, but the pod still opens a socket to the destination, and that connection is observed
A cached answer	Invisible — the resolution predates the window, and the workload never asks again	The connection recurs every time the cached answer is used, and each one is observed
A hardcoded IP, never resolved	Invisible — there was no question to see	Observed, by address

Still true. Two things survive the change:

• The name may be missing. A connection AIControls cannot attribute to a provider name is reported at its address as unattributed. The destination is counted and shown; only its identity is absent.
• Observations can be lost after they are made. The kernel's per-pod map comes under pressure on a busy workload, and the sensor's report objects are capped at 500 results per namespace and node, after which later findings are dropped permanently. AIControls surfaces this by reconciling the sensor's own counters against what it stored, as an ingest lag figure. That is the operator-visible symptom — loss shows up as a persistent lag figure, not as silence — but a lag figure is a warning, not a correction.

So the claim narrows rather than disappearing: absence of findings is still absence of evidence. Read the Gateway bypass card on Inventory → Workloads alongside the runtime coverage and ingest-lag figure on Health → Runtime, never on its own. The card says so in its own empty state.

Why every direct provider resolution is a finding

The observation policy AIControls authors carries a DNS allow-list, and what goes on that list is the opposite of the intuitive answer.

In a proxy-governed topology, the names a workload is expected to resolve are the in-cluster ones: the AIControls Service and cluster DNS. A correctly governed workload should never resolve api.example-provider.com itself — its gateway does that on its behalf.

Allow-listing providers would therefore whitelist precisely the leak the feature exists to catch. The bypassing call resolves the same provider name the governed path is nominally "expected" to use, matches the allow-list, and disappears from the findings. This is not theoretical: a provider-populated allow-list made a real, actively-leaking workload report clean during development.

With in-cluster names as the expected set, every direct provider resolution is a finding — which is the intended behavior, and why the destination-set headline is meaningful rather than noisy.

The same reasoning applies, more bluntly, to connections: AIControls does not allow-list provider destinations in the observation policy at all. Doing so would hide exactly the traffic the feature exists to surface. The network observation therefore carries no allow list — it reports every connection a governed pod opens, and the destinations that are not interesting are removed afterwards by suppression rules that state their reason, where an operator can see them.

What is deliberately not counted

Left unfiltered, cluster plumbing dominates the evidence list. Suppression is done with deterministic rules, each independently testable, and suppressed destinations stay visible in the expanded row with the reason attached — they are hidden from the count, never from the operator.

Suppressed: CoreDNS and the Kubernetes API server; every in-cluster address, derived from the API server rather than guessed from CIDR configuration; node and link-local addresses; AIControls' own Service, by IP and by name; reverse-DNS (in-addr.arpa) questions and search-domain expansions, which the Kubernetes default ndots:5 resolver configuration generates several-fold for every real lookup; service-mesh and telemetry control planes; and package registries.

Two deliberate exceptions:

• 169.254.169.254 is never suppressed. Access to the cloud instance metadata service gets its own finding kind rather than being swept up with other link-local traffic.
• AIControls' own upstream egress is excluded entirely, not merely suppressed per destination. The proxy is an in-cluster hop, so a governed workload's provider calls appear as egress from the AIControls pod. Counting those would inflate the number in the customer's favour — the worst possible direction for this metric — so rows attributable to AIControls' own workload are dropped before any counting, including rows from a previous AIControls pod.

The full list of suppression reasons is in the reference.

How evidence is attributed to a workload

Two joins have to happen, and they have different strengths.

Kernel evidence to a workload is resolved at ingest, while the pod is alive, by walking owner references to the controller that owns the pod (Pod → ReplicaSet → Deployment, Pod → ReplicaSet → Argo Rollout, Pod → Job → CronJob, Pod → Job → Argo Workflow). The terminal is whatever controller owns the chain, so a workload managed by a custom controller resolves to that controller rather than to the ReplicaSet or Job in between. A listed Job with no controller owner is a standalone workload and terminates at the Job itself. Resolving ownership at ingest rather than at query time is what keeps a pod restart from producing phantom rows. When an intermediate ReplicaSet or Job is absent from the current metadata listing, or a listed ReplicaSet has no controller owner, attribution is unresolved rather than guessed from its name: a ReplicaSet may carry a generated rollout hash and a CronJob-created Job may carry a schedule suffix. When none of the resolution paths succeed, the row falls back to the pod itself and is marked attribution unconfirmed in the Workload column. Such a row keeps its full headline count: the evidence is real and the destinations were genuinely reached — only who reached them is uncertain.

A row can also be stamped pod-level (multi-container). The sensor attributes to a cgroup, which is the pod, so when a known service-mesh or telemetry sidecar is present its egress cannot be separated from the application's.

Gateway identities to a workload is a name-based join: same namespace, and the audit identity matches the workload name, its ServiceAccount name, or the generated pod-name pattern. Every row carries join: name-based so this is never mistaken for a cryptographic binding. One practical consequence is worth planning around: per-workload ServiceAccounts are effectively required for a usable N, because several workloads sharing default cannot be told apart by name.

Note that this join affects N and the governed-overlap annotation only. It cannot change the bypassed-destination count, which is derived from kernel evidence alone.

Registered behavior answers the other half

Runtime Controls supplies the observed half — what a workload's pods demonstrably did. A workload registration supplies expected behavior. The primitives line up with the observations on this page — network, exec, and open are the same behaviors, seen from the other direction.

A declaration is not a grant and is designed so it cannot widen anything: it is intersected with the namespace's platform policy, and its Kubernetes section is a statement of expected use checked against the ServiceAccount's real power rather than conferring any.

AIControls compiles a registration into observing controls, and enforcement begins only after an operator enables it. See Compiling a workload registration for what each tier compiles to.

Known scope limits

• Sidecar-deployed gateways are not yet handled. The reconciliation assumes the in-cluster central-gateway topology, where AIControls runs as its own Deployment and its egress is excluded as described above. With a gateway deployed as a per-pod sidecar (proxy.deploy: sidecar), the gateway's own upstream egress originates from the workload's pod, and the in-pod suppression rule that would recognize it does not exist yet. Expect inflated bypassed-destination counts on that topology, and do not act on them.
• The network-based detector stands itself down in two situations, and states which one. A connection is an address and nothing else, so classifying one depends entirely on knowing which addresses are the cluster's own. When that knowledge is unusable, the detector switches itself off for the window and says so — a stated reason on the response and a banner on the page — rather than going quiet or emitting destination identities it knows to be meaningless. In both cases DNS-based findings continue unaffected, because they are classified by name, which needs none of this. And in both cases the window is thinner, not cleaner: connections are now the dominant source of evidence, so a low count from such a window carries much less weight than the same number from an ordinary one.
  • Egress NAT collapses destination identity. When more than 90% of a cluster's external destinations resolve to a single IP — the signature of an egress NAT gateway — every external destination looks like the same one, and the addresses stop distinguishing anything.
  • Cluster facts are missing or stale. Every in-cluster suppression rule except the ones derived from the Kubernetes API server matches on a name, so on an address-only row that API-server-derived picture of Services, EndpointSlices, Pods, and Nodes is the only thing separating "the pod talked to cluster DNS" from "the pod reached a provider directly". It can be unavailable during the startup window before the first listing completes, after a refresh has gone stale, or when the ServiceAccount is missing a list permission. Reporting the cluster's own plumbing as bypassed provider destinations would be worse than reporting nothing, so the detector stands down instead.
• In-cluster chatter competes for the report cap. The network observation carries no allow list — deliberately, as described above — so on a busy cluster it reports in-cluster traffic alongside external traffic, and all of it counts against the sensor's 500-results-per-report-per-node cap. A genuine provider destination can be crowded out by ordinary service-to-service noise. The dns, open, and exec observations each narrow what they report; network is the one that does not, so it is the one that crowds the others out — most consequentially the namespace's dns findings, a signal the headline depended on before connection evidence existed. The operator-visible symptom is a persistent ingest-lag figure, not a clean result: a lag that does not clear on refresh means findings were dropped, and any of them could have been the interesting one. The cap is the sensor's and AIControls cannot raise it, so the one lever is the operator's: scoping the network observation away from the offending namespace hands its report budget back to the three that narrow, at the cost of no connection evidence there — see Scope the network observation.
• Pods reaching an external address the sensor never saw resolved cannot be attributed to a named provider. The connection is observed and counted; the destination is rendered as unattributed rather than guessed. Naming it would need identity sources that are not implemented yet.
• Observation does not imply enforcement. Runtime Controls collects evidence while Enforcement is Observing. Blocking begins only after an operator enables enforcement and applicable controls become Enforcing.

See also

• Enable Runtime Controls — setup, prerequisites, and verification
• Runtime Controls Reference — endpoints, capability states, and every field on a row
• Compiling a workload registration — the declared counterpart: what a workload says it will do
• Network Egress Filtering — governing a workload's own outbound HTTP(S) calls at the gateway