Enable Runtime Guard

Tasks

These steps turn on kernel-level observation of AI workloads in a cluster you operate, and show you how to read the result. For what Runtime Guard is, what it can and cannot see, and how to interpret its headline number, read Runtime Guard first — the table is easy to over-read without it.

Self-hosted Kubernetes only

Runtime Guard cannot run on a Nirmata-hosted (SaaS) tenant. Your AIControls control plane must be running inside the cluster you want to observe. See Step 1.

Step 1 — Confirm your deployment is eligible

Runtime Guard reads Report objects from an API server and authors observation policy in a cluster. Both have to be your cluster.

1. Confirm you are self-hosted. AIControls must be installed in your own Kubernetes cluster via the Helm chart. On a Nirmata-hosted tenant the control plane runs in Nirmata's cluster — there is no API server of yours for it to read, and no cluster of yours for it to author policy in — so the feature refuses to start regardless of configuration.

2. Check node kernels if you want file and exec evidence. The network, protocol, and dns behaviors work on any kernel. The open (file) and exec (binary) behaviors require nodes booted with lsm=bpf, which stock managed-Kubernetes node images generally are not. This is not a blocker — the bypassed-destination headline is built entirely from network and dns evidence — but on those nodes file and exec findings cannot be produced at all, and the Coverage section will say so per node.

3. Give AI workloads their own ServiceAccounts if you have not already. Gateway-recorded calls are joined to workloads by name (workload name, ServiceAccount name, or generated pod-name pattern). Several workloads sharing default cannot be told apart, which degrades the N column and the governed-overlap annotation. It does not affect the bypassed-destination count, which comes from kernel evidence alone.

Step 2 — Install the Kyverno runtime sensor

AIControls does not install or bundle the sensor. It authors policy for a sensor you install, and ingests the reports that sensor produces. The sensor is Nirmata Runtime — a per-node eBPF DaemonSet with one cluster-scoped RuntimePolicy CRD.

Version requirement

Runtime Guard depends on the sensor's behaviors/dns schema (runtime.nirmata.io/v1alpha1), which is newer than the currently published chart and images. Until a release carries it, install the sensor from source following the build and install instructions in the nirmata/runtime repository. The admin console shows the current install command inline when it detects the CRD is absent, so you can copy it from there rather than transcribing it.

Whichever install path you use, the sensor must end up with:

• its DaemonSet running on every node you want observed, and
• the runtime.nirmata.io RuntimePolicy CRD served by the API server.

AIControls re-probes for the CRD every five minutes, so installing the sensor after AIControls is running is fine — you do not need to restart anything.

Step 3 — Enable runtimeGuard in the AIControls chart

Add the flag to your values.yaml and upgrade:

# Self-hosted Kubernetes only. Ignored (with a logged reason) when not
# running in Kubernetes or on a Nirmata-hosted tenant.
runtimeGuard:
  enabled: true

helm upgrade aicontrols \
  oci://ghcr.io/nirmata/charts/aicontrols \
  --namespace aicontrols \
  -f values.yaml

The flag does two things. It starts the integration in the process, and it renders a declinable ClusterRole and ClusterRoleBinding for the chart's ServiceAccount. When the flag is false, the chart never even requests those cluster permissions — so a cluster that has not opted in has no AIControls ClusterRole to audit.

The role grants, all cluster-scoped:

API group	Resources	Verbs	Why
runtime.nirmata.io	runtimepolicies	full CRUD	AIControls authors the capability-probe and observation policies
runtime.nirmata.io	runtimepolicies/status	read	Per-node applied state, used for capability detection
openreports.io	reports, clusterreports	read	The sensor's findings
core	pods, services, endpoints, nodes, namespaces	read	Attributing evidence to workloads; deriving in-cluster addresses from the API server rather than guessing them
apps	deployments, statefulsets, daemonsets, replicasets	read	Owner-reference walk from pod to workload
batch	jobs, cronjobs	read	Same walk for Job-backed workloads
discovery.k8s.io	endpointslices	read	Resolving cluster destination identity
apiextensions.k8s.io	customresourcedefinitions	read	Detecting whether the sensor's CRDs are installed at all

Every rule is probed with a SelfSubjectAccessReview at startup. If your cluster's admission or policy stack strips any of them, the missing grant appears as a structured gap on the Runtime page rather than as a log line nobody reads.

What appears in your cluster

Once enabled, AIControls authors its own RuntimePolicy objects — a capability probe, plus one observation policy per signal: dns (names resolved), network (destinations connected to), open (files opened), and exec (binaries executed). They cover every namespace except kube-system and kube-public.

All authored policy is monitor mode

Every one of these observes and reports. None of them denies anything — a network policy from AIControls will not block a connection, and an open or exec policy will not stop a process. Enabling Runtime Guard cannot break a running workload.

The network policy carries no allow list. That is deliberate: allow-listing provider destinations would hide exactly the traffic the feature exists to surface. Uninteresting destinations are removed afterwards by suppression rules that record their reason, so they stay visible to you in the expanded row.

Scope the network observation away from a noisy namespace

Carrying no allow list is what makes the network observation trustworthy, and it is also what makes it expensive. The sensor writes one report object per namespace and node, capped at 500 results, and once that cap is reached later findings for that namespace and node are dropped permanently. All four observations share that one budget — and three of them narrow what they report: dns by its in-cluster allow list, open and exec by their detection filters. network does not, so it reports one series per pod and destination, in-cluster service-to-service traffic included. In a large or chatty namespace that volume can crowd the namespace's own dns findings out of the report, which costs you a signal that was there before connection evidence existed.

The cap belongs to the sensor and AIControls cannot raise it. Your lever is to scope the network observation away from the namespace producing the volume:

runtimeGuard:
  enabled: true
  # Scopes the `network` observation only. kube-system and kube-public are
  # always excluded from every observation and do not need to be listed.
  networkObserveExcludeNamespaces:
    - batch-jobs
    - mesh-system

Only network is scoped. The dns, open, and exec observations continue to cover the listed namespaces unchanged, and excluding a namespace hands its report budget back to those three.

The cost is real, so do not set this preemptively. An excluded namespace produces no connection evidence: its rows go back to being DNS-only, with every weakness that carries — a resolution proves a name was looked up, not that traffic followed, and encrypted DNS, a cached answer, or a hardcoded IP produce nothing there at all. The symptom that justifies the trade is a persistent ingest-lag figure; see Troubleshooting.

Each entry must be a valid namespace name (a DNS-1123 label). A malformed entry fails policy rendering loudly at startup rather than quietly mis-scoping a security control.

Step 4 — Verify coverage

Open Runtime in the admin console sidebar. The nav entry only appears when the integration is actually available, so if it is missing, revisit Steps 1 and 3.

The Coverage card is the first thing to read, and it answers three questions independently:

1. Is the sensor installed? If the CRD is absent you get an explicit alert with the install command, not an empty table.

2. Which nodes have a live sensor? A per-node state of available, unavailable, or unknown. Read unknown as "cannot tell" — a node whose status shard is merely stale reports unknown, never unavailable.

3. Which nodes can produce file and exec findings? The BPF-LSM column, same three states. Nodes not booted with lsm=bpf show unavailable here while still showing available for network, protocol, and DNS.

You can also check it directly:

curl "$YOUR_WORKSPACE_URL/api/v1/runtime/status" \
  -H "Authorization: Bearer $ADMIN_TOKEN"

A response with "enabled": false carries a reason explaining which gate closed. See the reference for the full payload.

Step 5 — Read the reconciliation table

The Reconciliation — kernel vs gateway card lists one row per workload for the selected window (15 minutes to 24 hours; one hour by default), sorted by Bypassed destinations descending.

1. Start with Bypassed destinations. It is the count of distinct provider destinations the workload reached directly, outside governance. It is a set count, not a volume — 0 means the set was empty.

2. Do not subtract M from N. M counts kernel events and N counts gateway requests, over non-overlapping populations. They are on the row as corroborating evidence only; the column tooltips say so. See M and N are evidence, not terms in a formula.

3. Check the Evidence badge. AIControls observes both connections and DNS questions, and every row with evidence says which fired. A destination tagged network means the pod opened a socket to it — traffic happened. A row badged dns only is backed entirely by resolutions: a name was looked up, but nothing establishes that traffic followed.

   The absence of the badge is not by itself the stronger finding. dns only describes a row that has unsuppressed evidence and where all of it is resolutions, so a row with no unsuppressed evidence at all is also unbadged — and since the network observation gives nearly every pod in the cluster a row, most rows are exactly that. Read the badge next to Bypassed destinations: no badge and a non-zero count is the connection-backed finding; no badge with a zero count is just a quiet row.

4. Expand the row. Each destination shows its evidence class, count, identity confidence, and — critically — whether it was suppressed and why. Suppressed destinations stay visible with the reason attached.

5. Read "also governed (n)" as a stronger signal, not an excuse. It means the workload uses the governed path for that host and something in it also went around the path. Split traffic, typically a library ignoring HTTPS_PROXY, is the strongest bypass signal there is. It never clears the destination from the count.

6. Click a row to open the observations drawer: names resolved, destinations reached, and — where BPF-LSM is available — paths opened and binaries executed.

An empty table is not a clean bill of health

Encrypted DNS, a cached answer, and a hardcoded IP all hide the lookup, but the pod still opens a socket to the destination and that connection is observed — so those three no longer produce a clean-looking workload that is actively reaching providers. What survives is narrower: a destination the sensor never saw resolved in the clear is counted but shown at its address rather than a name, and observations can still be lost to kernel map pressure or the sensor's per-report result cap. Always read the table together with the Coverage card and the ingest lag banner — a lag figure that persists across refreshes means findings were dropped, and a lost observation is indistinguishable from a workload that did nothing.

Troubleshooting

The Runtime nav entry is missing. The integration is not available. Check that AIControls is running in Kubernetes, is not a Nirmata-hosted tenant, and that runtimeGuard.enabled: true was actually applied — helm get values aicontrols -n aicontrols will tell you.

"The runtime sensor is not installed on this cluster." The RuntimePolicy CRD is not served. Complete Step 2; the CRD is re-probed every five minutes, so no restart is needed.

"The proxy's ServiceAccount is missing permissions." The startup access review found a gap. The alert lists each missing verb, group, and resource. Confirm the ClusterRoleBinding from Step 3 exists and that nothing in your cluster removed rules from the ClusterRole.

Node state is unknown rather than available or unavailable. This is deliberate and means the evidence to decide is not there — most often no status shard has been reported recently, or the capability probe has not completed yet. Do not treat unknown as clean.

"Network-based bypass detection is disabled for this window." The detector stood down. Read the stated reason — there are two causes, and they need different responses. In both, DNS-based findings continue, and because connections are the dominant source of evidence, treat such a window as thinner than usual, not as clean.

• Egress NAT. More than 90% of the cluster's external destinations collapsed to a single IP, the signature of an egress NAT gateway, so addresses no longer distinguish destinations. Nothing to fix — this is the cluster's network topology, and the detector says so rather than emitting identities it knows to be meaningless.
• Cluster facts unavailable or stale. The Kubernetes listing of Services, EndpointSlices, Pods, and Nodes has not completed, has gone stale, or is being refused. A kernel-observed connection is an address and nothing more, so without that listing a connection to cluster DNS or a peer pod cannot be told apart from one to a provider — and reporting the cluster's own plumbing as bypassed destinations would be worse than reporting nothing. If it clears within a few minutes of a restart or upgrade, it was the startup window and needs no action. If it persists, check the Runtime page for an RBAC gap: a missing list grant on pods, services, nodes, or endpointslices is exactly what produces it, and the startup access review surfaces it as a structured gap listing the missing verb, group, and resource (see the entry above and the role table in Step 3).

The ingest lag figure never clears. Findings the kernel recorded are not becoming stored rows — kernel map pressure, or the sensor's 500-results-per-report-per-node cap. On a busy cluster the usual cause is volume: the network observation deliberately carries no allow list, so ordinary in-cluster service-to-service traffic is reported alongside external traffic and competes for the same cap, and a real provider destination can be crowded out. The cap is the sensor's and cannot be raised from AIControls, so the direct lever is to scope the network observation away from the namespace producing the volume with networkObserveExcludeNamespaces — that returns the namespace's report budget to the dns, open, and exec observations, at the cost of no connection evidence there. Narrowing the observation window or reducing the noise on the affected nodes also helps. A persistent lag figure means the table is incomplete — do not read a low count from that window as a result.

A destination shows as ip:<address> (unattributed). The connection was observed, and it counts — only its name is missing. This happens when the sensor never saw the pod resolve that address in the clear (encrypted DNS, a cached answer, or a hardcoded IP). AIControls resolves in-cluster addresses from the Kubernetes API server; for an external address with no observed answer there is no source to name it from, so it is shown as an address rather than guessed at.

Counts look inflated on a sidecar deployment. The reconciliation currently assumes a central in-cluster gateway. With a gateway deployed as a per-pod sidecar, the gateway's own upstream egress originates from the workload's pod and is not yet suppressed. See Known scope limits.

See also

• Runtime Guard — the model, and what the evidence does and does not prove
• Runtime Guard Reference — endpoints, capability states, and row fields
• Self-Hosted Kubernetes — installing and operating AIControls in your own cluster