Arm a workload kill switch
A kill switch stops a registered workload's activity on the planes you choose, for a time you set. It blocks new activity only: it does not evict, restart, or scale down pods, and it does not kill processes that are already running.
Workload Trust is available only in private, self-managed AIControls installations, on premises or in a cloud cluster that you run. It needs a dedicated cluster that you run, with Kyverno, Kyverno Runtime, and cluster OIDC and RBAC access in place. Nirmata-hosted (SaaS) workspaces don't include it. See Feature availability.
Before you begin
- The kill switch is off by default. An administrator turns it on in Setup → Settings → General → Features → Workload kill switch. While it is off, the Kill switch action is hidden and the arm API returns
404. - The target must be a registered workload (see Register a workload). AIControls resolves the workload to its pods through the same ownerRef walk that generates its controls.
- Arming requires an administrator session. AIControls records who armed the switch and when from the session, never from the request.
- The File system and exec plane needs the open/exec hooks on each node, which need bpffs mounted at
/sys/fs/bpf. The dialog shows the nodes that cannot enforce it before you arm.
What each plane stops
| Plane | What stops | What keeps working |
|---|---|---|
| Network | Every new IPv4 outbound connection from the workload's pods, including DNS. With kyverno-runtime v0.1.4 or later, the version the chart bundles, new IPv6 outbound connections stop too. | Established connections. Inbound traffic. IPv6 egress on a sensor older than v0.1.4. AIControls can't tell which sensor version runs, so the plane reports IPv6 as a coverage gap wherever it is offered, the same gap the enforcement review reports (#3452). |
| File system and exec | New process execs and new file opens in the workload's pods. kubectl exec into the pod is also denied. | Processes that are already running, and files they already hold open. |
| Gateway | Every MCP, LLM, and HTTP request the AIControls gateway sees from an identity bound to the workload's pods, including Cross-App Access agents identified by their client ID. | Anything that does not go through the gateway. Identities that authenticate with a token AIControls cannot tie to one of the workload's pods (for example, a PAT) are not bound and are not denied. |
Choose Full for all three planes, or Partial for any subset.
Arm the kill switch
- Open Inventory → Workloads, select the workload, and open its full detail page.
- Select Kill switch. The dialog opens.
- Choose Full or Partial. Each plane shows whether it is available here. The File system and exec plane lists the nodes that cannot enforce it; the Gateway plane shows how many identities are currently bound to the workload.
- Read the Blast radius and the Pod selector. The selector is exactly what the kernel policy selects: the workload's stable pod labels in its one namespace. The highlighted label is the one AIControls verified as unique to this workload.
- Enter a Reason. It is required and recorded in the configuration audit.
- Choose Expires after. The default is 2 hours and the maximum is 24 hours. The switch disarms itself when the time runs out.
- Type the workload name to confirm, then select Arm kill switch.
Activity on the armed planes stops as the policies apply, usually within seconds. The page shows a red Kill switch armed banner with the planes, who armed it, the reason, and a countdown.
In a deployment with several AIControls replicas, the gateway plane's deny policy takes effect on the replica that armed it at once and on every other replica at its next policy sync (policy.cacheTTLSeconds, 60 seconds by default). The same bound applies when you disarm. The kernel planes are applied by the sensor on every node and are not affected.
Degraded
The banner shows degraded when an armed plane cannot enforce everywhere:
- The File system and exec plane is armed and a node hosting the workload has no open/exec hooks. Pods on that node are not contained on this plane.
- The Gateway plane is armed and no identity is bound to the workload yet. AIControls binds identities as they appear and updates the deny policy on the next reconciliation.
- A kernel policy has not yet been confirmed applied by the sensor.
The banner names each degraded plane and the reason.
Disarm
Select Disarm in the banner and confirm. AIControls removes the kernel policy and the gateway deny policy, then removes the armed state. Activity resumes as soon as the policies are gone. You can disarm even after an administrator turns the feature off: the banner stays visible while a switch is armed.
If AIControls has lost cluster access, it refuses to disarm a switch that has a kernel plane and keeps the armed state, because it cannot remove the RuntimePolicy. Restore cluster access, then disarm again. If the status cannot be loaded, the page shows the error with a Retry action and never reports the workload as unarmed.
Use the API
Check availability and the armed state:
curl https://<your-host>/api/v1/workload-protection/prod/Deployment/rag-indexer/killswitch \
-H "Authorization: Bearer $TOKEN"
The response carries status.availability (one entry per plane, with unavailableNodes for the file system and exec plane), status.podSelector, status.uniqueLabel, status.blastRadius, and, while armed, status.planes, status.expiresAtUnix, status.degraded, and status.planeStates.
Arm:
curl -X POST https://<your-host>/api/v1/workload-protection/killswitch \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"namespace":"prod","kind":"Deployment","name":"rag-indexer","planes":["network","gateway"],"reason":"incident 42","ttlSeconds":3600}'
Use "planes":["full"] or "full":true for all three planes. Omit ttlSeconds for the 2-hour default; values above 86400 are rejected.
Disarm:
curl -X DELETE https://<your-host>/api/v1/workload-protection/killswitch \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"namespace":"prod","kind":"Deployment","name":"rag-indexer"}'
Arming a workload that is already armed returns 409. Disarm first, then arm again.
Contain a workload
Containment is the faster, narrower alternative to the kill switch's Gateway plane. It denies every MCP and LLM request from a workload's bound ServiceAccount, starting with the next call, and it needs no policy reload.
| Kill switch | Containment | |
|---|---|---|
| Stops | Network, file system and exec, and gateway requests, on the planes you choose | MCP and LLM requests at the gateway |
| Keyed on | The workload's Pods | The cluster and the bound ServiceAccount |
| Needs the feature flag | Yes | No |
| Expires | After the time you set | Never. You release it. |
| Works in Audit and Enforce | Yes | Yes. No exception lifts it, and a broken access snapshot doesn't lift it. |
| Changes the access mode | No | No |
Containing a workload in Audit also turns off the audit unmatched override; in Enforce the override has no effect and is left as it is. It doesn't stop outbound HTTP egress or traffic that skips the gateway. Arm a kill switch plane for those.
Contain the workload. AIControls contains the identity of the workload's active identity binding, so bind the workload's identity first. The binding must use the pod_uid rung (a k8s://<namespace>/<serviceaccount> key): gateway requests check containment only for verified Kubernetes ServiceAccount identities, so AIControls refuses to contain a virtual-key binding rather than report a containment that denies nothing. To stop a virtual key, revoke it or arm a kill switch.
curl -X POST "https://YOUR_WORKSPACE_URL/api/v1/workloads/prod/Deployment/rag-indexer/contain?cluster=$CLUSTER_UID" \
-H "Authorization: Bearer $AICONTROLS_TOKEN" \
-H "Content-Type: application/json" \
-d '{"reason": "incident 42"}'
The cluster query parameter is the UID of the cluster's kube-system namespace. Until the controller can read the kube-system namespace, it keys the cluster by its display name instead. AIControls refuses a contain that names that display name as cluster, with 409 cluster_identity_unavailable, because configuration written under the display name wouldn't carry over once the UID resolves. Wait for the UID, then retry with it as cluster. Release stays allowed under the display name, because containment moves to the UID when it resolves. If it has already moved, a release that names the display name returns 409 cluster_identity_changed and releases nothing; retry with the UID as cluster. The body is optional. If you add identityKey, it must be the workload's bound identity. The response reports contained: true.
Release it:
curl -X POST "https://YOUR_WORKSPACE_URL/api/v1/workloads/prod/Deployment/rag-indexer/release?cluster=$CLUSTER_UID" \
-H "Authorization: Bearer $AICONTROLS_TOKEN"
Without a body, release lifts the containment recorded for this workload, even after you retired its identity binding, or the containment on the identity the workload is bound to now. If more than one identity is contained for the workload, release answers 409 with code containment_ambiguous; name the identity to release in the body, for example -d '{"identityKey": "k8s://prod/rag-indexer"}'.
A workload's route acts only on that workload's identity. Contain answers 409 with code identity_binding_required when the workload has no active identity binding, 409 with code containment_ineligible when that binding isn't a pod_uid binding, and 409 with code containment_identity_mismatch when identityKey is not the workload's bound identity. Release answers 409 with code containment_identity_mismatch when the identity belongs to a different workload: when another workload has the identity bound now, or, if no workload has it bound, when it was contained for a different workload. Releasing an identity that isn't contained succeeds and changes nothing.
Contain and release each write a configuration audit event: workload_containment.created when an identity becomes contained, workload_containment.deleted when it's released, and workload_containment.updated when a contain changes an existing record, for example a new reason for an identity that's already contained, or a contain that also turns off the audit unmatched override. The Containment card on the AI access tab of the workload detail page shows the state, with Contain and Release buttons. Contain asks for an optional reason. The card links to the kill switch.
What AIControls refuses
AIControls refuses to arm a workload it doesn't know at all (not discovered and not registered) with 404 and workload not found. It refuses to arm, with a 409 and a reason, when:
- the workload is known but not registered;
- no observed pod currently resolves to the workload;
- no single pod label identifies the workload uniquely in its namespace — the same label and value is on a pod that belongs to another workload; or
- the File system and exec plane is requested and no node can enforce it.
Every kernel kill switch pins exactly one namespace and a non-empty pod selector. AIControls never renders a cluster-wide or namespace-wide kill switch.
While a kill switch is armed, AIControls re-checks the selector against the running pods on every reconciliation. If another workload's pod comes to match it (for example, after a relabel), if none of the target's own pods match it any more (the target was relabelled), or if AIControls cannot observe the cluster's pods completely and so cannot verify the selector, it suspends the kernel planes, removes the RuntimePolicy, and reports the plane as degraded with the reason until the selector verifies again. AIControls never rewrites the recorded selector; if the target's labels changed for good, disarm and arm again. The gateway plane is unaffected.
A workload whose pods AIControls cannot currently resolve is not armable for any plane; the status reports it as unavailable with the reason rather than offering an arm that would be refused.
The kill switch acts on the cluster this AIControls deployment runs in. The cluster-scoped workload detail page (opened with ?cluster=) does not offer the action, and a request that names another cluster is refused with 409.
Audit
Arming writes workload_killswitch.created and disarming or expiry writes workload_killswitch.deleted to the configuration audit (Security → Activity → Configuration). The gateway plane's generated policy appears as policy.created and policy.deleted on the same actor, and in Security → Policies under the Containment category with a workload-killswitch- name.