Validate LLM Responses
A response-phase policy evaluates the text coming back from the model or an MCP tool, not just the request that triggered it. Use it to catch things that only exist in the reply — a missing disclosure, a compliance phrase, PII the model generated rather than received. For policy anatomy and matching basics, see Write a Policy.
Write a response-phase policy
A response-phase policy is an ordinary ValidatingPolicy, scoped to the response pass with one annotation:
metadata:
annotations:
proxy.nirmata.io/direction: response
Without this annotation, a policy only ever sees the request. Combine it with a call-type match condition, exactly as you would for a request-phase policy:
object.llm.model != ""— match LLM calls (Claude Code, chat completions, etc.)object.mcp.tool != ""— match MCP tool calls
spec:
matchConditions:
- expression: 'object.llm.model != ""'
See Write a Policy for the general Audit → Warn → Deny workflow and enforcement-mode annotations — they apply the same way here.
Built-in response fields
The response-phase content scanner populates a handful of fields on every response pass, namespaced by call type (object.llm.response.* for LLM calls, object.mcp.response.* for MCP tool calls — there is no flat object.response.*):
hasPII— true if the reply contains a detected PII entity (email, SSN, credit card, etc.)hasCredential— true if the reply contains a detected credential or secrethasInjection— true if the reply contains a detected prompt-injection patternfindings[]— the individual matches behind the flags above, each with class, entity, severity, score, and byte offsets
These cover the common case — most response-safety policies only need one of the boolean flags. For the full field list, types, and the confidence-score fields, see Policy Context.
The content-safety policy pack ships a ready-made policy built entirely from these fields — flags any LLM reply the scanner detects PII in, with no custom expression needed. Install it with Install a Policy Pack rather than hand-writing the equivalent.
Custom checks against raw response text
The built-in scanner doesn't know your business rules — a required disclosure sentence, an opt-out phrase, a regulatory pattern specific to your industry. For those, write a custom expression against the raw response text:
object.llm.response.text— full LLM response text on the response passobject.mcp.response.text— full MCP tool response text on the response pass
For example, a TCPA/SMS compliance rule can deny an outbound message that lacks a "reply STOP to opt out" style disclosure. Gate on isFinal so the check runs once against the whole reply, not once per streaming chunk:
matchConditions:
- expression: 'object.llm.model != ""'
- expression: object.llm.response.isFinal
validations:
- expression: |
!object.llm.response.text.matches("(?i)(sms|text message)") ||
object.llm.response.text.matches("(?i)(reply|text)\\s+stop\\s+to\\s+opt.?out")
message: "Outbound SMS-style text must include opt-out language (TCPA compliance)"
A rule like this is inherently specific to one compliance regime, so it isn't something a general-purpose policy pack can ship on your behalf — write it against your own required wording and install it as an ordinary policy. See Install a Policy Pack for the built-in checks that are shipped ready-made (PII, credentials, prompt injection).
Streaming responses require buffering for a full-message check
This is the part that trips people up: object.llm.response.text / object.mcp.response.text is populated on every response pass, including per-chunk streaming ones — but on a per-chunk pass it's only that one chunk's text, not the full reply, and the scanner's findings[]/hasPII/etc. stay empty (those are only computed once, against the complete message). Without an isFinal gate, a check like the one above runs on every chunk against whatever fragment just arrived — a required phrase near the end of the message looks "missing" on every chunk until the one that happens to contain it, producing noisy, often-wrong audit results. With an isFinal gate, the check is correct but only evaluates on the pass where isFinal is actually true — and for a streaming call, that pass only happens with buffering enabled (next paragraph); without it, the policy simply never fires.
To get full-text evaluation on streaming responses, set:
llmProxy:
bufferForDLP: true
With bufferForDLP enabled, the proxy accumulates the full response before evaluating response-phase policies and running the content scanner, then flushes it to the client. This is required for two things:
- Correctness — full-message checks (your custom text rule,
findings[]) only have data to work with on this buffered-final pass. - Enforcement — a
denyon a streaming response only actually blocks delivery if evaluation happens before any bytes reach the caller. Without buffering, chunks are already streaming out as they arrive, so a deny decided later can't take them back.
The tradeoff is latency: the client sees no output until the full response is ready, instead of the usual token-by-token stream. Turn it on for upstreams and routes where response-side compliance checks matter enough to accept that delay.
llmProxy.bufferMaxBytes caps how much of the response is held in memory this way. A response that exceeds the cap falls back to per-chunk delivery for the remainder, so a very long reply may not get full-text evaluation — keep the cap high enough for your typical response sizes, or pair it with a policy that treats an over-cap response conservatively.
Traffic your tool sends through the native /bedrock/* or /vertex/* routes (CLAUDE_CODE_USE_BEDROCK=1, CLAUDE_CODE_USE_VERTEX=1, and the Vertex/Gemini SDKs) gets a non-streaming reply evaluated the same as any other route, for Claude-on-Bedrock, Claude-on-Vertex, and native Gemini models. A streaming reply on these two routes, though, is not evaluated at all — not even with llmProxy.bufferForDLP: true — so a response-phase policy or the built-in content scanner never sees it. Non-Claude Bedrock model families (Amazon Nova/Titan, Meta Llama, Mistral, Cohere, and others) also aren't evaluated on the response side yet, regardless of streaming — their reply format isn't recognized. If your workload uses streaming on these routes, or a non-Claude Bedrock model, response-phase checks won't fire until those gaps close; route the traffic through your workspace's Anthropic or OpenAI-compatible endpoint instead if you need response-phase coverage today.
Worked example
A minimal, hand-written policy (not a shipped pack file — see the previous section) that denies any streaming-or-not LLM reply mentioning SMS/text messaging without opt-out language, paired with the config change that makes it actually evaluate on streaming replies:
apiVersion: policies.kyverno.io/v1
kind: ValidatingPolicy
metadata:
name: sms-opt-out-disclosure-required
annotations:
proxy.nirmata.io/direction: response
spec:
matchConditions:
- expression: 'object.llm.model != ""'
- expression: object.llm.response.isFinal
validations:
- expression: |
!object.llm.response.text.matches("(?i)(sms|text message)") ||
object.llm.response.text.matches("(?i)(reply|text)\\s+stop\\s+to\\s+opt.?out")
message: "Outbound SMS-style text must include opt-out language (TCPA compliance)"
validationActions: [Audit]
llmProxy:
bufferForDLP: true
Start in Audit mode to measure matches against real traffic, then graduate to Deny once you've confirmed it doesn't false-positive on your workload.
Where an Audit-mode match shows up
An Audit-mode response-phase match never blocks or alters the reply — the caller gets the response unchanged. To see the match while you're measuring traffic, open Security → Events and find the call: its Response tab shows a Flagged disposition, along with the matched policy, rule, and message. This is the one place a non-blocking response-phase match is recorded — it doesn't appear as a separate audit entry.