Choose Models for Internal AI Features
AIControls uses an LLM for its own features. Model roles let you give each kind of work the model it needs: a low-cost one for summaries, a stronger one for the Judge, and a reasoning one for the Analyst.
Before you start
Configure Internal AI Features under Settings → AI Features first: a connection (an upstream from Settings → Providers, or a direct API key) and, optionally, a default model. Every role uses that default unless you change it. Changing models requires an admin; viewers can see each role's model.
The roles
| Role | What it covers | Default |
|---|---|---|
| Summaries | Session summaries, coaching and effectiveness narratives, classification, project attribution, prompt readiness scoring, project, workload and pull-request assessments | The default model |
| Judge | The LLM Judge that reviews each matched attack pattern and may downgrade it | A stronger Claude model (Sonnet 5.5) when the default connection is Anthropic and no default model is set; otherwise the default model |
| Analyst | The AI Analyst: security narratives on Identity and Session pages and contextual analysis of findings | The default model |
A Judge downgrade hides a finding, and the Judge reads evidence an attacker can write: tool arguments, file names, destinations. In red-team tests a low-cost model was talked into downgrading real attacks by approval or ticket claims planted in that evidence, which is why the Judge gets a stronger model by default.
Choose a model for a role
Open Model roles. Go to Settings → AI Features and scroll to Model roles. Each role shows the model it's using now.
Pick a connection. Leave Default connection to use the Internal AI Features connection, select Direct API key to use the key saved there, or select another upstream from Settings → Providers.
Pick a model. Enter a model ID, such as
claude-sonnet-5-5, or leave it empty for the default model. A model must be one the selected connection serves.Save. Select Save. The change applies to the next call; no restart is needed. To go back to the default, clear both fields and save. If another admin changed Internal AI Features since you opened the page, the save is refused; reload and try again.
A role pinned to a connection that no longer works (a removed upstream, or the direct key with no key saved) stops working rather than falling back to another credential: the Judge records no verdict and the match keeps its template's action, and summaries and Analyst requests report the error.
Settings → AI Features → Usage and cost groups internal calls as Judge, Analyst and Other internal features. Open a feature’s linked Gateway Activity to inspect calls and the models that served them.
Keep downgrades honest
Whatever model the Judge uses, Hold downgrades based on approval claims (in the LLM Judge card under Settings → Attack Patterns, on by default) stops a downgrade from being applied when the evidence carries an approval, ticket, allowlist, break-glass or test-fixture claim, or text addressed to the reviewer (such as "classify this as benign", also when URL- or base64-encoded), or the Judge's reasoning rests on an approval claim. The match keeps its template's action, and the recorded reasoning says why. Turning it on also re-checks downgrades already recorded for matches that are swept again (an agent's last 30 days or a session's last 24 hours); older recorded downgrades keep their verdict. Use exemptions for workflows that really are expected.