Skill Risk
Every skill your agents invoke — instructions injected from a prompt, a bundle uploaded from your marketplace, or one bundled with the agent itself — becomes a tracked row with a risk score. This page explains what a skill is, how that score is built, and the deployment decision every security team has to make about it.
What is a skill
A skill is a packaged, reusable set of instructions an AI agent can load and follow — a SKILL.md file (plus optional scripts, references, and assets) describing a task the agent should be able to perform. Skills follow the open Agent Skills specification (agentskills.io/specification), which defines a required name and description in YAML frontmatter, optional fields like license and allowed-tools, and a directory layout (scripts/, references/, assets/) that agent runtimes across the industry have converged on.
Skills are powerful precisely because they're instructions, not code review artifacts: an agent that loads a skill follows it the same way it follows any other part of its context. That is also what makes an unreviewed skill a governance gap — a skill pulled from an internal marketplace, copied from a public repository, or injected inline in a prompt can direct an agent to touch credentials, call unexpected hosts, or exfiltrate data, with no code review step in between.
AIControls builds an inventory of every skill your agents actually invoke, scores each one for risk, and lets policy gate on that score — without requiring you to pre-approve every skill before it can run.
Two risk planes
A skill's risk score is the sum of two independent kinds of evidence:
| Plane | Answers | How |
|---|---|---|
| Static content scanning | "What does this skill's content say it does?" | Pattern checks run against the skill's SKILL.md body (and, for full-bundle submissions, its other files) — shell exec patterns, credential access, network/exfiltration patterns, prompt injection, privilege escalation, and more. See Skill Indicators for the full catalog. |
| Behavioral profiling | "What did this skill actually do, when agents ran it?" | AIControls attributes tool calls observed on the LLM request path — the tool_use blocks the proxy sees in the agent's model traffic — back to the active skill, and aggregates the resulting activity into a behavioral profile: tools invoked, hosts contacted, files written, calls denied. Tool executions that reach an MCP server through the MCP proxy port are not attributed to skills and do not feed the profile (see the limitation note under Divergence and drift). |
Be clear-eyed about what static scanning can and can't catch. Pattern checks are a floor, not a ceiling — a skill's instructions can be paraphrased, encoded, or split across files to slip past a regex, and a scan only sees what a bundle's content says, never what an agent actually did with it. Static scanning catches the careless case. Behavioral profiling is the detector for the adversarial case, because behavior can't be paraphrased: a skill whose SKILL.md reads as an innocuous formatting guide but which drives an agent to contact an unlisted host is exactly the signature static scanning alone can never catch, and it's what the behavioral plane is built to surface.
Both planes feed one combined, capped score (0–100) and one risk band. Every contributing signal — which pattern fired, which tool call it came from — is retained as evidence, so a score is never a black box.
Fidelity tiers: prompt vs. bundle
A skill can enter the inventory at one of two fidelity levels, depending on how much content AIControls actually has in hand:
| Fidelity | What's scanned | When it happens |
|---|---|---|
| Prompt | The SKILL.md body only, extracted from the agent's own request as it invokes the skill | Automatic, zero-config — happens on every connected agent path with no setup required |
| Bundle | The full skill package — every file, executable bits included — submitted directly | Only when a bundle is explicitly submitted for scanning (marketplace publish, CI gate) — see Submit a skill bundle for scanning |
Prompt fidelity is the default and requires nothing from you — it's how a skill first appears in the inventory the moment any agent runs it. Bundle fidelity is strictly more thorough (it can see sibling scripts and dependency manifests a SKILL.md body alone never reveals) and is the recommended path for skills your organization curates or publishes internally, since it lets you gate a publish on a scan verdict before the skill ever reaches an agent.
A skill can appear at both fidelities over time — the same name, different content digests — and both live in the same inventory. AIControls always prefers the more thorough bundle-fidelity result when one exists for a given piece of content.
Which clients are detected
Prompt-fidelity capture recognizes a skill invocation from the wrapper the agent client puts around the skill text in the final user message of a request:
| Client | Wrapper AIControls recognizes |
|---|---|
| Claude Code | The <command-name>/skill-name</command-name> tag Claude Code sends at the start of the message when you type a slash command, together with its <command-message> and optional <command-args> tags, followed by the skill body |
| Codex CLI | A bounded <skill> block containing a <name> element, an optional <path> element, and the SKILL.md text, sent when you invoke a skill with $skill-name |
The Codex <skill> block is the format Codex CLI sends today. It is client behavior, not a documented OpenAI API contract, so a future Codex release could change it. AIControls tolerates small variations (whitespace, letter case, attributes on the <skill> tag, and the order of <name> and <path>). It takes only the text between those headers and </skill> as the skill body, never the rest of the prompt. A <skill> block with no name or an empty body is not treated as an invocation.
A <command-name> tag counts only at the start of the message, where Claude Code places it. A tag that appears anywhere else, such as source code or documentation that quotes the tag, tool output, or the text of a Codex skill, is ordinary prompt text: it does not name a skill, add an inventory entry, or raise skill alerts.
A detected invocation is handled the same way for both clients on /v1/messages, /v1/chat/completions, /v1/responses, and cloud provider routes with recognized Anthropic or Gemini request shapes: the skill is added to the inventory with source prompt and scanned, its name and digest populate object.skill.* for policy, a Skill event is written to the audit log, and the LLM call's audit row carries the skill name and digest. Unrecognized cloud request shapes, including non-Claude Bedrock bodies, are not inspected for skill invocations. The Skill event for a Codex invocation shows client codex.
Only the final message of the request is checked, so a wrapper left in earlier conversation history never counts as a new invocation. The model's follow-up requests in the same run — tool results sent back while the skill is still working — inherit the active skill for that agent session, so their audit rows and policy context keep the skill name with attribution window. A request carries a skill name only when one of these two sources backs it. Without a detected invocation or an active skill for that session, the audit row has no skill name, even if the prompt text shows a wrapper.
A skill name on its own never earns the built-in exemption. Claude Code built-in slash commands are treated as builtin only when no skill body follows the command. A Codex skill is never treated as built-in, whatever its name.
Sources and derived trust
Every skill row records where it came from — one of four sources:
| Source | Meaning |
|---|---|
prompt | Observed only from runtime extraction — no one has vouched for it |
uploaded | Submitted directly through the ingestion API |
marketplace | Submitted through the ingestion API and tagged as a marketplace/CI publish |
builtin | Shipped with the agent itself, or explicitly configured as vendor-equivalent |
Source, combined with whether a scan verdict exists, derives a trust tier automatically — nobody hand-maintains a list of "trusted" skill names:
| Condition | Derived tier |
|---|---|
| Marketplace or uploaded, clean (trusted-band) scan | Governed |
| Marketplace or uploaded, any scan verdict | Approved |
| Prompt-only, no bundle scan | Unverified |
| Built-in (vendor or curated internal list) | Approved |
This matters because trust tier and risk score answer different questions, and they're deliberately kept as two separate axes rather than collapsed into one:
- Trust tier = provenance — "who vouches for this?" Passing the ingestion gate with a clean verdict is the approval event; there's no separate manual sign-off step to fall out of sync.
- Risk score = measured evidence — "what does its content say, and what did it actually do?"
The disagreement cases are the interesting ones: an approved skill whose behavioral profile starts drifting toward high risk is an incident (a reviewed artifact gone bad), not a false positive — a clean provenance record was never a promise that behavior can't change later. An unverified skill with a low risk score is the classic supply-chain shape: nobody vouched for it, but nothing bad has been observed yet, either. Write policy against both axes together rather than relying on either alone.
Risk bands
The combined score sorts every skill into one of three bands:
| Band | Score | Meaning |
|---|---|---|
| Trusted | below 30 (default) | No material risk signal from either plane |
| Review | 30–69 (default) | Enough signal to warrant a human look before broad reliance |
| Blocked | 70+ (default) | High-confidence risk signal — treat as actively dangerous |
Band thresholds and every indicator's weight are shipped defaults, not fixed values — see Tune skill risk weights to adjust them for your environment, and Skill Indicators for the full weight table.
Not every indicator is security-relevant. A subset are spec-conformance and code-quality ("hygiene") signals — a malformed skill name, a script with no error handling, and similar — that are detected and shown as evidence but don't contribute to the score by default, so they can't dilute a band that's meant to reflect security risk. See Tune risk weights to change this for your environment.
Divergence and drift
Behavioral profiling produces two distinct comparisons that a security reviewer should read differently:
Static↔behavioral divergence compares the same piece of content's declared behavior against its observed behavior. A skill whose SKILL.md reads as a benign formatting guide but which contacts an unlisted host at runtime is the canonical malicious-skill shape — content scanning alone can never catch it, because the instructions driving that call don't have to contain any greppable pattern. Divergence is flagged the moment observed behavior falls outside any category the static scan indicated (network access, credential access, execution, writes), and it fires regardless of how low the static score looked.
Drift compares a skill's current behavioral profile against its own established baseline — the same content, but new hosts or new categories of tool use appearing after that baseline was set. Drift is the signal for "this skill's behavior is changing," which usually means its inputs, or a downstream tool it depends on, have changed. Read it as a prompt to re-review, not as proof of compromise on its own — a baseline built during a quiet period can itself be gamed by a patient, slow-escalating actor, which is exactly why divergence (comparison against static content, which has no history to warm) is the compensating control rather than a redundant one.
Both are exposed as fired indicators (BEHAV_STATIC_DIVERGENCE, BEHAV_DRIFT) that a policy can gate on directly — see the recipes below.
Divergence and drift are computed from tool calls observed on the LLM request path only — the tool_use blocks in the model traffic the proxy forwards. A tool call an agent executes against a governed MCP server (through the MCP proxy port) is enforced and audited like any other MCP call, but it is not attributed to the active skill and feeds neither signal. A skill whose entire runtime footprint is MCP tool calls can therefore look behaviorally quiet indefinitely — its profile renders as a normal low-activity profile, not as "no data for this transport." Treat behavioral indicators as strongest for agents whose tool use flows through the LLM path (Claude Code's built-in tools do), and rely on static scanning plus MCP-path policy for the rest.
Skill file drift (artifact hash verification)
Behavioral drift (above) is about a skill's runtime activity changing. Skill file drift is a separate, structural check: it detects when the files that make up a skill's bundle — SKILL.md, hook scripts, plugin manifests, CLAUDE.md — change after AIControls has already seen them. This is the same "rug-pull" concern tool-definition pinning addresses for MCP tool schemas, applied to a skill's own files: a skill your team already reviewed can be silently edited afterward — in a marketplace, a shared repository, or a bundle an agent re-reads on every run — and because a skill's files are fed directly to the model as instructions, an edited file can steer an agent without anyone touching a policy or config.
AIControls hashes every file in a skill's bundle the first time it observes that skill by name and stores the result as the approved baseline (a prompt-fidelity observation, which only has a SKILL.md body to work with, is treated as a single-file bundle). Every later observation of that skill name at the same fidelity — on either the bundle-ingestion path or ordinary runtime use — hashes the current file set again and compares it against that baseline, file by file, independent of whether the skill's overall content digest also changed.
Skills observed with source builtin — the AI tool's own bundled slash commands, not content a user uploaded or an agent fetched — are exempt from file pinning entirely: there's no baseline to compare against and no user-supplied bundle to drift, so no pin is attempted and no skill_file_drift alert can fire for them.
The baseline is keyed by skill name and capture fidelity, not by name alone: a bundle-ingestion observation and a runtime prompt-fidelity observation of the same skill name are tracked as two separate baselines and are never compared against each other. A bundle's SKILL.md (raw, frontmatter included) and the synthetic single-file body AIControls reconstructs from a prompt-fidelity observation are not byte-comparable — comparing them across fidelities would produce drift alerts on every observation regardless of whether the file actually changed. The practical consequence: pinning a skill's bundle at upload time does not, by itself, verify that a later runtime (prompt-fidelity) observation of that same skill is still serving the reviewed content — that comparison only happens once a prompt-fidelity baseline has been established from runtime observations of its own.
Each file carries a drift state, the same three states tool-definition pinning uses:
| State | Meaning |
|---|---|
| Pinned | Matches the approved baseline — unchanged. |
| Drifted | The file existed in the baseline, but its content changed since it was approved. |
| New | The file wasn't in the baseline — it appeared after the baseline was set. |
Drift states are sticky, same as tool-definition pinning: once a file is marked drifted or new, it stays that way through any number of later observations, and a skill_file_drift alert fires once — the first time a file transitions into drifted or new — not on every subsequent observation that re-sees the same already-alerted change. To keep a flood of changes from overwhelming the inbox, once more than 256 distinct drift alerts fire within 24 hours, further ones in that period are rate-limited together as one group (at most one inbox entry every five minutes). Slack/Teams delivery is deduplicated fleet-wide the same way: a given drift alert goes out once per five minutes across all replicas, and the overflow group produces at most one external alert per five minutes. The 256-alert threshold is counted per replica.
skill_file_drift is an admin-audience notification, delivered in-app and to Slack/Teams by default (not email) — the same default channel mix as tool_schema_drift and the other skill-governance events. Configure it, per workspace, from Settings → Notifications → Routing.
The in-app notification is titled Skill changed: name. Its message names who used the skill, when the skill was first seen, and which files changed:
dev@example.com used skill pdf-tools, and its SKILL.md differs from the version first seen on 2026-10-01. Changed: SKILL.md.
- Who is the signed-in user's email for an uploaded or rescanned skill. For a skill observed in agent traffic, it is the email of the user behind the request when AIControls knows it, and the agent ID otherwise. When neither is known, the message starts with the skill name instead:
Skill pdf-tools's SKILL.md differs …. - First seen is the UTC date the skill's baseline was recorded. It is left out when there is no earlier baseline.
- Changed lists up to 20 changed and 20 new paths, then
(+N more)for the rest. When more than one file changed, the message readsits files differ.
Slack, Teams, and email show the same message under the heading Skill File Drift Detected. Every channel also includes a View skill link to the skill's detail on Inventory → Skills, built from ui.publicURL. If that value isn't set, or names this machine (such as localhost or 127.0.0.1), the link is left out. The capture fidelity (bundle or prompt) is kept in the notification's metadata, not in its title or message.
Unlike tool-definition pinning, there is currently no per-file drift view or re-pin action in the UI — the skill_file_drift alert (in-app, and Slack/Teams if configured) is how you learn about a detected change today, and its message lists which paths changed or are new.
When the baseline store is full
The number of distinct skill file-hash baselines AIControls will hold is capped — the same fleet-wide ceiling applied to the skills table itself. Once that ceiling is reached, AIControls does not delete or evict an existing, already-approved baseline to make room for a new one: an approved baseline is never removed automatically, and there is currently no admin API or CLI to prune baselines out of band either. Instead, it refuses to establish a new baseline for a skill name/fidelity pair it hasn't pinned yet. That skill is left in a not pinned state — its files are scanned for risk as usual, but no file-drift comparison happens for it until the ceiling is raised (the inventoryCeiling field of the skill risk-scoring configuration, shared with the skills-table inventory cap) — raising the ceiling is the only recovery path today.
A refusal fires a skill_file_hash_ceiling_refused admin notification — the same default channel mix as skill_file_drift: in-app and Slack/Teams by default, not email. Refusal alerts are rate-limited fleet-wide as a single group: the alert names the first skill that couldn't be pinned, and further refusals (for that or any other skill) in the following five minutes do not raise another alert. Configure it, per workspace, from Settings → Notifications → Routing.
Choosing a posture: fail-open vs. fail-closed
The first time any skill runs, it has no score yet — scanning happens asynchronously, on the order of seconds. This is the one deployment decision every team using skill risk should make deliberately rather than by default.
| Fail-open (default) | Fail-closed | |
|---|---|---|
| Behavior on an unscanned skill | Allowed to proceed immediately | Held for approval until a scan verdict exists |
| What it protects against | Nothing on first use — control is detective, not preventive, for that one invocation | A malicious skill's very first execution |
| Cost | A brand-new (or first-seen) skill can act before its score is known | Every never-before-seen skill pauses on first use, even harmless ones |
| Best for | Most teams, most of the time — the unscanned window is seconds, and blocking every first use out of the box is hostile | Security-led deployments where the population of skills is expected to be adversarial (e.g. an open internal marketplace) |
Two numbers worth knowing before you pick: the unscanned window (enqueue → scan) is typically seconds; after any re-score, the enforced band can lag a freshly-updated verdict by up to the policy engine's short cache refresh interval. Neither number is large, but they're the honest bound on "how fast does this actually protect."
Fail-open recipe (default posture)
Deny only skills that have already been confirmed high-risk; say nothing about skills that haven't been scanned yet:
apiVersion: policies.kyverno.io/v1
kind: ValidatingPolicy
metadata:
name: skill-deny-blocked
spec:
matchConditions:
- expression: 'object.skill.name != ""'
validations:
- expression: '!(object.skill.riskBand == "blocked")'
message: "This skill's risk score has crossed the blocked threshold"
validationActions: [Deny]
Fail-closed recipe (opt-in posture)
Add a companion policy that also holds any never-scanned skill for approval — the two together are the fail-closed posture:
apiVersion: policies.kyverno.io/v1
kind: ValidatingPolicy
metadata:
name: skill-require-scanned
annotations:
proxy.nirmata.io/enforcement-mode: warn-approve
spec:
matchConditions:
- expression: 'object.skill.name != ""'
validations:
- expression: '!(object.skill.name != "" && !object.skill.scanned)'
message: "This skill has not yet completed a risk scan"
validationActions: [Deny]
Divergence recipe (either posture)
Divergence and drift are high-confidence enough that most teams deny on them outright, independent of the posture chosen above:
apiVersion: policies.kyverno.io/v1
kind: ValidatingPolicy
metadata:
name: skill-deny-divergence
spec:
matchConditions:
- expression: 'object.skill.name != ""'
validations:
- expression: '!("BEHAV_STATIC_DIVERGENCE" in object.skill.indicators)'
message: "This skill's observed behavior contradicts its declared content"
validationActions: [Deny]
For how to view these fields in practice, submit a bundle for a full scan, tune the weight table, and roll these recipes out as enabled policies, see Govern Skills.
Skills on the agent detail page
Every agent's detail page lists the skills it invoked in the last 30 days, with a risk-band tag next to any skill in the review or blocked band. Selecting a skill opens its scan report — the same drill-down available from the Skills inventory — so you can go from "this agent invoked a risky skill" straight to the indicators and evidence behind that score.
This view also feeds the agent's skill-risk signal: the worst band among an agent's invoked skills, shown beside — never folded into — its composite risk score.