Skip to main content

Runtime Guard

Concepts

A gateway records what arrived. No field in an audit log can represent a call that never arrived — a clean audit and a workload routing its traffic around AIControls produce the same report. Runtime Guard adds a second, independent vantage point: an eBPF sensor on every node that observes what a workload's pods actually did, so the two views can be compared.

note

Runtime Guard is available on self-hosted Kubernetes only, is off by default, and is marked experimental in the admin console. See Availability below and Enable Runtime Guard for setup.

Availability

Runtime Guard needs two things that only exist in a cluster you operate: an API server it can read Report objects from, and a cluster it can author policy in for the sensor to act on.

On a Nirmata-hosted (SaaS) tenant, the AIControls control plane runs in Nirmata's cluster, not yours. There is no customer API server to read from and no customer cluster to author policy in, so the integration cannot function there — and asking for cluster-wide pod and workload read permission inside a multi-tenant control plane would be the wrong thing to request even if it could.

The gate therefore fails closed on the environment, not just the flag: setting runtimeGuard.enabled: true on a SaaS tenant still does not start the integration. When it is not running, the Runtime page and its API endpoints do not error or render blank — they state the reason ("unavailable on Nirmata-hosted (SaaS) tenants", "not running in Kubernetes", "disabled by config").

Attestation, not detection

Most workload-security tooling infers: it guesses what a workload probably is from names, images, and environment variables, scores the guess, and alerts. AIControls' own workload discovery does exactly this, and it is guesswork by construction — a Deployment named payments-svc running an LLM client library is invisible to it, and a static site called ollama-docs is a false positive.

Runtime Guard is built the other way round. It does not ask "does this look like an AI workload?". It asks "what did this workload's pods demonstrably do, at the kernel, and does that match what governance saw?" — a question with evidence behind every answer. The output is not a probability. Each row cites the specific destinations, the specific evidence class that produced them, and the specific reason anything else was excluded.

The trade-off is that evidence has edges. The rest of this page is mostly about where those edges are, because a reader who does not know them will over-read a clean table.

What the kernel plane can and cannot see

The sensor is a per-node DaemonSet with five observable behaviors. Three of them work on any kernel; two do not.

BehaviorObservesRequires a kernel booted with lsm=bpf
networkDestination addresses a pod connected toNo
protocolThe application protocol a flow speaks (tls, dns, quic, ssh, …)No
dnsNames a pod resolved — observe-only, never enforceableNo
openAbsolute file paths a pod openedYes
execAbsolute binary paths a pod executedYes

Stock managed Kubernetes node images are generally not booted with lsm=bpf. On those nodes open and exec findings cannot be produced at all — not "none were seen", but "this class of evidence is structurally unavailable here". The Coverage section of the Runtime page reports this per node rather than averaging it away, and reports it as three states, with unknown never collapsed into unavailable (see the capability contract).

AIControls authors monitor-mode observation policy for four of these behaviors — dns, network, open, and exec — across every namespace except kube-system and kube-public. The first two need no special kernel and are what the headline number is built from. Both halves are observed: the question a workload asks, and the connection it opens. They are separate facts, and every destination on a row says which one produced it.

Four further limits are structural, not bugs to wait out:

  • Nothing reads inside TLS, and there is no SNI, so a connection yields an address rather than a name. AIControls resolves the in-cluster half of that from the Kubernetes API server. For an external address the pod never resolved in the clear, no name can be established, and the destination is rendered honestly as unattributed (ip:203.0.113.9 (unattributed)) rather than guessed. This is now the sharpest limit on the feature — the connection is seen; naming it is what falls short.
  • Ingress is out of scope. Every hook is egress or execution, so a pod serving a model, an MCP server, or an agent card is not observed on the serving side.
  • The sensor is IPv4-only, so IPv6 egress produces no finding at all. A workload that reaches a provider over IPv6 contributes nothing to any row — and nothing anywhere records that it was out of reach. There is no marker, no flag, and no placeholder destination for it. The honest reading is unobservable rather than absent, but that reading is yours to supply: the product cannot tell "no IPv6 traffic occurred" apart from "IPv6 traffic occurred and was never seen", so an empty result is evidence about IPv4 and nothing else.
  • Findings carry counts, not ordering. network, protocol, open, and exec observations are drained from kernel counters on a poll interval; only dns questions are streamed as they happen. So a connection can lag the behavior that caused it by up to one interval.

Bypassed destinations — the headline

The number at the front of each row is Bypassed destinations: the count of distinct unsuppressed provider destinations this workload demonstrably reached outside governance — either by a kernel-observed connection, or by resolving the provider's name itself.

It is a count of a set of destinations, which is what makes it usable:

  • Unit-free. It is not a volume, so it does not inflate with retries or deflate with connection reuse.
  • Stable under caching. A workload that resolves api.example-provider.com once and then serves 10,000 requests from a cached answer still contributes exactly one destination.
  • Honest at zero. A 0 always means the unsuppressed set was empty. It can never mean "a subtraction happened to clamp".

Connection evidence and resolution evidence feed the same set. There is no second headline number for connections, and each destination's Evidence tag records which class produced it.

Whether a connection and a resolution collapse into one destination depends on whether AIControls could name the connection:

  • When it could — the sensor observed the pod's own DNS answer for that address — the connection and the resolution key on the same identity and contribute one destination between them.
  • When it could not — DNS-over-HTTPS or DNS-over-TLS, a cached answer, or a hardcoded IP, which are precisely the cases connection observation was added for — the connection is unattributed, keys on its raw address, and counts separately from the workload's resolution of the provider name.

Two entries for what may well be one destination is the deliberate cost of not guessing. Nothing in the evidence proves that the name a workload asked for and the address it dialled are the same destination, so AIControls says "I could not name this" rather than manufacturing a join that the kernel evidence does not support. Closing the gap needs identity sources that are not implemented yet — not a correlation heuristic.

One consequence is worth planning around: identity resolution is window-wide. A naming observation made anywhere in the window — including by a different workload — names that address for every row in it, so whether a pair collapses can turn on evidence outside the row you are reading. A count can therefore move between windows without the workload's own behavior changing.

M and N are evidence, not terms in a formula

Each row also shows two raw counters, and the temptation to subtract them must be resisted:

ColumnCountsPopulation
M (kernel)Kernel events — connections opened plus names resolved — to unsuppressed external provider destinationsWhat the pods did
N (gateway)Requests AIControls recorded for identities joined to this workloadWhat governance handled

These count different things over non-overlapping populations. The kernel counts connections; the gateway counts requests, and HTTP/2 multiplexing means one connection legitimately carries many requests. More fundamentally, in the proxied topology a governed call produces no workload-side kernel evidence at all — the gateway resolves and dials the provider on the workload's behalf, so that traffic leaves the gateway's pod, not the workload's.

M − N is therefore not a quantity. It is two different measurements of two different things with a minus sign between them. M and N are on the row because they are useful corroborating evidence — "this workload is busy on both planes" is worth seeing — and for no other reason. The admin console's column tooltips say so explicitly.

Governed overlap strengthens a finding; it never clears one

Expanding a row shows each destination, and any destination the gateway also carried traffic to is tagged also governed (n).

This is easy to misread as an excuse — "the workload does use the proxy for that host, so this is fine". It is the opposite. Because a governed call leaves no workload-side kernel evidence, direct evidence to a host the workload also reaches through the gateway means the traffic is split: the workload knows and uses the governed path, and something in it — most commonly a library that ignores HTTPS_PROXY — also goes around it. That is a stronger signal than a workload that was never configured at all, not a weaker one.

Governed volume never subtracts from the destination set. A destination with a hundred governed calls and one direct connection still counts as one bypassed destination.

The annotation is only as complete as the gateway-side record it draws on. Destination hosts are recorded by the egress plane, so the tag only appears where network egress filtering is enabled. Its absence means "no gateway record of that host", which on a cluster without the egress plane is uninformative rather than reassuring.

Two things a clean table does not prove

A resolution is not a connection — and the row tells you which one fired

A dns finding proves that a name was looked up. It does not prove that any traffic followed. Code that resolves a hostname during start-up and then fails to connect, a resolver pre-warming a cache, a library enumerating candidate endpoints — all produce a DNS question with no connection behind it.

That is a statement about DNS evidence, not a limit on what AIControls can see. Connections are observed directly. So the table carries both classes and grades them, at both of its levels: each destination carries an Evidence tag of network or dns, and a row whose entire unsuppressed evidence is DNS is badged dns only.

Read the two badges differently:

  • A network-sourced destination means the pod opened a socket to it. Traffic happened.
  • A dns only row means "this workload asked where a provider lives" — worth investigating, but it does not establish that anything was sent.

The practical consequence is that a row losing its dns only badge is a materially stronger finding than the same row carried before, not a cosmetic change: it is the difference between intent and traffic.

The badge is a statement about a row that has unsuppressed evidence, though, so its absence alone is not the stronger reading. A row with no unsuppressed evidence at all is also not dns only — and since the network observation gives nearly every pod in the cluster a row, most rows are exactly that. Read the badge next to Bypassed destinations: no badge and a non-zero count is the connection-backed finding; no badge and a zero count is simply a quiet row.

A quiet table is still not a clean bill of health

The reverse error is more dangerous, because it makes an empty table look like a result. Connection observation closes most of what used to make this claim broad, so it is worth being precise about how much is left.

Largely closed. These three all used to produce a clean-looking workload that was actively reaching providers. All three are now observed — not as a name, but as a connection, which is the part that matters for the count:

SituationWasIs now
DNS-over-HTTPS / DNS-over-TLSInvisible — the question is inside an encrypted sessionThe lookup is still unreadable, but the pod still opens a socket to the destination, and that connection is observed
A cached answerInvisible — the resolution predates the window, and the workload never asks againThe connection recurs every time the cached answer is used, and each one is observed
A hardcoded IP, never resolvedInvisible — there was no question to seeObserved, by address

Still true. Two things survive the change:

  • The name may be missing. A connection AIControls cannot attribute to a provider name is reported at its address as unattributed. The destination is counted and shown; only its identity is absent.
  • Observations can be lost after they are made. The kernel's per-pod map comes under pressure on a busy workload, and the sensor's report objects are capped at 500 results per namespace and node, after which later findings are dropped permanently. AIControls surfaces this by reconciling the sensor's own counters against what it stored, as an ingest lag figure. That is the operator-visible symptom — loss shows up as a persistent lag figure, not as silence — but a lag figure is a warning, not a correction.

So the claim narrows rather than disappearing: absence of findings is still absence of evidence. Read the table alongside the Coverage section and the ingest-lag figure, never on its own. The Runtime page says so in its own empty state.

Why every direct provider resolution is a finding

The observation policy AIControls authors carries a DNS allow-list, and what goes on that list is the opposite of the intuitive answer.

In a proxy-governed topology, the names a workload is expected to resolve are the in-cluster ones: the AIControls Service and cluster DNS. A correctly governed workload should never resolve api.example-provider.com itself — its gateway does that on its behalf.

Allow-listing providers would therefore whitelist precisely the leak the feature exists to catch. The bypassing call resolves the same provider name the governed path is nominally "expected" to use, matches the allow-list, and disappears from the findings. This is not theoretical: a provider-populated allow-list made a real, actively-leaking workload report clean during development.

With in-cluster names as the expected set, every direct provider resolution is a finding — which is the intended behavior, and why the destination-set headline is meaningful rather than noisy.

The same reasoning applies, more bluntly, to connections: AIControls does not allow-list provider destinations in the observation policy at all. Doing so would hide exactly the traffic the feature exists to surface. The network observation therefore carries no allow list — it reports every connection a governed pod opens, and the destinations that are not interesting are removed afterwards by suppression rules that state their reason, where an operator can see them.

What is deliberately not counted

Left unfiltered, cluster plumbing dominates the evidence list. Suppression is done with deterministic rules, each independently testable, and suppressed destinations stay visible in the expanded row with the reason attached — they are hidden from the count, never from the operator.

Suppressed: CoreDNS and the Kubernetes API server; every in-cluster address, derived from the API server rather than guessed from CIDR configuration; node and link-local addresses; AIControls' own Service, by IP and by name; reverse-DNS (in-addr.arpa) questions and search-domain expansions, which the Kubernetes default ndots:5 resolver configuration generates several-fold for every real lookup; service-mesh and telemetry control planes; and package registries.

Two deliberate exceptions:

  • 169.254.169.254 is never suppressed. Access to the cloud instance metadata service gets its own finding kind rather than being swept up with other link-local traffic.
  • AIControls' own upstream egress is excluded entirely, not merely suppressed per destination. The proxy is an in-cluster hop, so a governed workload's provider calls appear as egress from the AIControls pod. Counting those would inflate the number in the customer's favour — the worst possible direction for this metric — so rows attributable to AIControls' own workload are dropped before any counting, including rows from a previous AIControls pod.

The full list of suppression reasons is in the reference.

How evidence is attributed to a workload

Two joins have to happen, and they have different strengths.

Kernel evidence to a workload is resolved at ingest, while the pod is alive, by walking owner references (Pod → ReplicaSet → Deployment). Resolving it then rather than at query time is what keeps a pod restart from producing phantom rows. When none of the resolution paths succeed, the row falls back to the pod itself and is marked attribution unconfirmed in the Workload column. Such a row keeps its full headline count: the evidence is real and the destinations were genuinely reached — only who reached them is uncertain.

A row can also be stamped pod-level (multi-container). The sensor attributes to a cgroup, which is the pod, so when a known service-mesh or telemetry sidecar is present its egress cannot be separated from the application's.

Gateway identities to a workload is a name-based join: same namespace, and the audit identity matches the workload name, its ServiceAccount name, or the generated pod-name pattern. Every row carries join: name-based so this is never mistaken for a cryptographic binding. One practical consequence is worth planning around: per-workload ServiceAccounts are effectively required for a usable N, because several workloads sharing default cannot be told apart by name.

Note that this join affects N and the governed-overlap annotation only. It cannot change the bypassed-destination count, which is derived from kernel evidence alone.

The declared side of the same question

Runtime Guard supplies the observed half — what a workload's pods demonstrably did. The team that ships a workload can supply the declared half in its AI BOM: a behavior section naming the models, MCP servers, network destinations, executables, and file paths the image is expected to use, bound to one image digest. The primitives line up with the observations on this page — network, exec, and open are the same behaviors, seen from the other direction.

A declaration is not a grant and is designed so it cannot widen anything: it is to be intersected with the namespace's platform policy, and its Kubernetes section is a statement of expected use checked against the ServiceAccount's real power rather than conferring any. An AI BOM with no behavior section — which is most of them today — is unaffected and still confers inventory.

This release records and validates a declaration; it does not yet compile one into enforcement. Nothing on this page is generated from an AI BOM today. See the AI BOM Behavior Section reference for every field, its validation rules, and the current status.

Known scope limits

  • Sidecar-deployed gateways are not yet handled. The reconciliation assumes the in-cluster central-gateway topology, where AIControls runs as its own Deployment and its egress is excluded as described above. With a gateway deployed as a per-pod sidecar (proxy.deploy: sidecar), the gateway's own upstream egress originates from the workload's pod, and the in-pod suppression rule that would recognize it does not exist yet. Expect inflated bypassed-destination counts on that topology, and do not act on them.
  • The network-based detector stands itself down in two situations, and states which one. A connection is an address and nothing else, so classifying one depends entirely on knowing which addresses are the cluster's own. When that knowledge is unusable, the detector switches itself off for the window and says so — a stated reason on the response and a banner on the page — rather than going quiet or emitting destination identities it knows to be meaningless. In both cases DNS-based findings continue unaffected, because they are classified by name, which needs none of this. And in both cases the window is thinner, not cleaner: connections are now the dominant source of evidence, so a low count from such a window carries much less weight than the same number from an ordinary one.
    • Egress NAT collapses destination identity. When more than 90% of a cluster's external destinations resolve to a single IP — the signature of an egress NAT gateway — every external destination looks like the same one, and the addresses stop distinguishing anything.
    • Cluster facts are missing or stale. Every in-cluster suppression rule except the ones derived from the Kubernetes API server matches on a name, so on an address-only row that API-server-derived picture of Services, EndpointSlices, Pods, and Nodes is the only thing separating "the pod talked to cluster DNS" from "the pod reached a provider directly". It can be unavailable during the startup window before the first listing completes, after a refresh has gone stale, or when the ServiceAccount is missing a list permission. Reporting the cluster's own plumbing as bypassed provider destinations would be worse than reporting nothing, so the detector stands down instead.
  • In-cluster chatter competes for the report cap. The network observation carries no allow list — deliberately, as described above — so on a busy cluster it reports in-cluster traffic alongside external traffic, and all of it counts against the sensor's 500-results-per-report-per-node cap. A genuine provider destination can be crowded out by ordinary service-to-service noise. The dns, open, and exec observations each narrow what they report; network is the one that does not, so it is the one that crowds the others out — most consequentially the namespace's dns findings, a signal the headline depended on before connection evidence existed. The operator-visible symptom is a persistent ingest-lag figure, not a clean result: a lag that does not clear on refresh means findings were dropped, and any of them could have been the interesting one. The cap is the sensor's and AIControls cannot raise it, so the one lever is the operator's: scoping the network observation away from the offending namespace hands its report budget back to the three that narrow, at the cost of no connection evidence there — see Scope the network observation.
  • Pods reaching an external address the sensor never saw resolved cannot be attributed to a named provider. The connection is observed and counted; the destination is rendered as unattributed rather than guessed. Naming it would need identity sources that are not implemented yet.
  • Enforcement is not part of this. Runtime Guard observes. Every policy AIControls authors — including the network one — is monitor mode with nothing blocked; nothing here stops a connection, denies a file handle, or terminates a pod.

See also