Skip to main content

Productivity

Concepts

AIControls sees every AI tool call, its cost, and — for Claude Code — the exact commit SHA it produced. The Productivity dashboard closes the loop on the other side: did that commit ship in a merged pull request, and how much of your team's shipped code went through AI versus manual work? We call this model AI delivery attribution.

Ingestion is unconditional, attribution is a join​

Once the GitHub integration is installed on a repository, every merged pull request is recorded — from registered developers, unregistered developers, contractors, and bots alike. Attribution to an AI session never gates ingestion.

This matters for the rollup: you can't credibly report "60% of shipped code is AI-managed" if the other 40% was never counted in the first place. A pull request with no attributable session is still counted — bucketed honestly as manual (or, if the author has no AIControls session at all, as unattributed) rather than silently dropped.

Bots (recognized via GitHub's account-type field) are excluded from every rollup denominator, so automated PRs (dependency bumps, generated changelogs) don't distort the AI-vs-manual split. Rows keyed by a raw, unmatched GitHub login (no AIControls identity behind them) are still shown — tagged unresolved rather than dropped — because hiding them would quietly corrupt the honest-denominator story. An unresolved login has no known developer identity (no commit-email match, no SSO match, no admin override), so it can never be assigned to a Team — its PRs always land in "Unassigned" on the Breakdown and Adoption tabs until someone resolves it. The Identities tab lists every unresolved login and lets an admin map it to a developer or mark it as a bot.

Attribution tiers​

A merged pull request can be linked back to the AI session(s) that produced it with varying confidence.

TierSignalConfidenceStatus
1 — SHA exact ("witnessed")The PR's head commit or any commit in its history matches a git commit SHA captured from a Claude Code sessionDeterministicShipped
Marker ("self-declared")The PR body, a commit trailer, or the PR's own authorship carries a self-declared AI-tool markerSelf-reportedShipped
2 — Branch/file matchThe PR's branch or changed files match a session's branch/file activity in the right time windowHighPlanned
3 — Content similarityCode the session produced line-matches the PR diffMediumPlanned
4 — Time + ticketSession was active around the same time and references the same ticket, with no stronger signal availableLowPlanned

A single PR can carry commits from more than one session — and even from more than one day, if it went through revisions. Attribution is stored as a many-to-many link between PRs and sessions, not a single column, so "what did this PR cost?" has one correct answer even when several sessions contributed to it.

Witnessed vs. self-declared​

Both tiers count as AI-managed, but they come from very different places and the dashboard never blends them silently:

  • Witnessed means AIControls itself saw the commit come out of a Claude Code session it proxied — the SHA-exact match above. This is the only tier with cost data attached, since cost is tracked per session.
  • Self-declared means no proxy-witnessed session was found, but the PR carries a marker one of these tools leaves behind on its own:
    • PR body — a footer like "🤖 Generated with [Claude Code]" or "Generated by OpenAI Codex"
    • Commit trailer — e.g. Co-Authored-By: Claude <noreply@anthropic.com>, or Cursor's/Aider's equivalent
    • PR authorship — the pull request itself was opened by a coding agent's bot identity (GitHub Copilot's coding agent, Devin, Jules)

Precedence: when a PR has both a witnessed session and a marker, witnessed wins for display — a PR is never double-counted, and the headline reads e.g. "41% AI-managed (28% witnessed · 13% self-declared)".

warning

Self-declared markers are exactly that — self-declared. They're a footer or trailer text, not a cryptographic proof: a developer who strips it disappears from this tier, and anyone can paste one into a commit message by hand. That's why marker-based attribution is always reported as a distinct, lower-confidence tier and never merged into "witnessed." Marker-only PRs also carry no cost data — cost aggregation is session-based and doesn't extend to markers.

One specific trap worth knowing about: a Co-authored-by: Copilot commit trailer is not counted as generation. GitHub adds that trailer automatically whenever a developer accepts a Copilot code-review suggestion — it marks code that was reviewed, not code Copilot generated. A fully human-written PR that accepted one review nit would otherwise show up as AI-managed, which would be wrong. Copilot generation is instead detected from PR authorship (the Copilot coding agent's bot identity), not from that trailer.

What gets tracked​

For every merged pull request in a configured repository, AIControls records:

  • Repository, PR number, title, author, and merge time
  • Lines added/removed
  • Whether the author is a bot
  • Every commit SHA and commit message in the PR (squash and rebase merges rewrite the head SHA, so the full commit list — not just the head — is what witnessed matching checks against)
  • The AI session(s) attributed to it, if any, and the attribution tier
  • Any self-declared marker found, its tool, and where it was found (PR body, commit trailer, or PR authorship)

Automated PRs​

Every merged pull request lands in exactly one of three buckets, checked in this order:

  1. Witnessed or self-declared — a linked AI session, or a marker, beats everything else. Any AI evidence at all — witnessed or self-declared — outranks the bot heuristic below, so a bot-authored PR that carries a marker (e.g. GitHub Copilot's coding agent opening the PR itself) is still reported as AI-managed, never as automated.
  2. Automated — the author is a bot (Dependabot, Renovate, and similar) and neither of the signals above fired. This is the catch-all for dependency bumps, generated changelogs, and similar bot housekeeping that never touched an AI coding tool.
  3. Manual — everything else: no witnessed session and no marker, and not caught by the bot check above. This also covers PRs whose author identity hasn't been resolved to human or bot yet — not just confirmed human authors.

Automated PRs are excluded from the AI-share percentage entirely — both the numerator and the denominator. A repository that merges a hundred Dependabot PRs a month shouldn't see its "AI-managed" number move at all, in either direction; automated work was never a human-vs-AI authorship decision to begin with. The definition of the AI-share % itself is unchanged by this split — it's still (witnessed + self-declared) / (witnessed + self-declared + manual), with automated PRs simply never entering that ratio.

Efficiency metrics​

Efficiency isn't a single panel — it's a set of outcome comparisons (not just volume) across witnessed, self-declared, and manual work, surfaced in the places on the page where each one is most useful:

  • Cycle time and cost per AI PR are stat cards in the top card row, so they're visible without opening a tab.
  • Review rounds (median) appears as a column in the Breakdown tab's Team grouping, and as part of the tooltip on individual rows in the Pull requests tab.
  • Revert rate and abandon rate are the "quality parity" strip on the Adoption tab.
MetricWhat it measuresCompared across
Cycle time (median hours)Time from PR open to mergeWitnessed · Self-declared · Manual
Review comments (median)Reviewer comment count on the merged PRWitnessed · Self-declared · Manual — API only, not surfaced in the UI
Review rounds (median)1 + count(CHANGES_REQUESTED), or 0 if the PR was never reviewedWitnessed · Self-declared · Manual
AI-reviewed shareFraction of merged, human-driven PRs that got at least one Copilot reviewSingle rate — see AI-reviewed share below
Cost per witnessed PRLinked AI session cost ÷ number of merged witnessed PRsWitnessed only
Revert rateFraction of merged PRs later revertedAI (witnessed + self-declared) vs. manual
Abandon rateFraction of opened PRs closed without mergingAI vs. manual
Sample sizesMerged PR counts backing every median aboveWitnessed · Self-declared · Manual · Automated

All medians are computed per range and filter selection, exactly like every other number on the page — narrow the scope bar and the medians recompute.

note

Small sample sizes produce misleading medians, so every AI-vs-manual comparison (cycle time, revert rate, abandon rate) checks both sides' sample size before rendering a number. If either side has fewer than 5 merged PRs in scope, the cell shows — with a "not enough data" tooltip instead of a number that would just be noise. Widen the time range or clear a filter to get past this.

Cost per witnessed PR inherits the same limitation as linked cost anywhere else on this page: only witnessed PRs have cost data, because cost is tracked per AI session and only witnessed attribution links back to a session. A self-declared marker PR has no session to charge cost against, so it never appears in this metric's denominator — a self-declared-heavy team's "cost per AI PR" will look artificially cheap if you mentally include those PRs; the metric never does.

note

A revert stamp is one-way: once a PR is marked Reverted, editing its reverting PR's body to remove the "Reverts …" line — or even reverting the revert itself — does not clear the original PR's Reverted mark. Treat it as a durable historical flag, not a live state that tracks the current branch contents.

note

Revert detection only recognizes a same-repo reference: a "Reverts owner/repo#N" line only marks a target PR when owner/repo is the same repository the revert PR itself merged into, matching what GitHub's revert button actually generates. A PR body naming a target in a different repository is ignored.

note

Revert detection isn't always instant. If a "Reverts …" PR references a same-repo target PR that AIControls hasn't ingested yet — for example, the revert's webhook arrives before that repository's initial backfill has finished — the Reverted mark is applied the next time a backfill runs for that repository. There is no automatic periodic backfill today: a fresh run otherwise means re-saving the GitHub integration in Settings → Integrations, or triggering a backfill manually. The one case that self-heals automatically is a GitHub API rate limit hit mid-backfill — AIControls schedules one automatic retry once GitHub's own rate-limit window is expected to have cleared, rather than leaving that run's history permanently partial. Outside of that case, if neither a manual re-save nor the admin trigger happens, a missing Reverted tag can persist indefinitely.

AI-reviewed share​

AI-reviewed share answers a different question from everything else on this page: not "did AI write this code?" but "did AI review it?" The two are independent — a fully human-written PR can get an AI review, and a witnessed AI-authored PR can be reviewed only by humans.

The signal is a review from GitHub's Copilot code-review bot (copilot-pull-request-reviewer[bot]) on the PR. Any state of that review — approval, change request, or a comment — counts; the share is the fraction of merged, human-driven PRs (witnessed, self-declared, and manual — automated PRs are excluded, same as everywhere else) that got at least one such review.

warning

Don't confuse this with the Co-authored-by: Copilot commit trailer covered above — that trailer is generation-adjacent authorship noise, not review data. AI-reviewed share only counts an actual review event from the Copilot reviewer bot; it has nothing to do with who wrote the code being reviewed.

Teams​

The Team grouping (in the Breakdown tab's group-by toggle, and the Adoption tab's team table) rolls PRs up by the developer's team in AIControls — the same team assignment used for cost and policy attribution elsewhere in the product, sourced from each user's teams field, not anything read from GitHub.

A developer whose AIControls user account has no team assigned — or whose PR author identity has no linked AIControls user at all — is bucketed under Unassigned rather than dropped. Unassigned always sorts last in the team table, and it isn't a real team: there's no team ID behind it, and it can't be picked as a filter value, only observed as a rollup bucket.

Team assignment is one-to-one: a user belongs to exactly one team at a time (AIControls enforces this invariant when teams are managed), so a PR's author contributes to exactly one row of the team rollup, never split across two.

Linkage coverage​

Linkage coverage is a descriptive metric, not a compliance score: of the AI sessions your developers ran, what fraction ended up linked to a merged PR? A low number doesn't necessarily mean waste — a session might have been exploratory, feed into a PR that hasn't merged yet, or produce a change too small to be its own PR. Read it alongside PR volume, not as a standalone "efficiency" figure. Coverage is scoped by developer, not by repository — a session belongs to the developer who ran it, not to any one repo.

Page anatomy​

The Productivity page (Dashboards section of the sidebar) is built around one shared scope for the whole page:

  • Scope bar — a time-range picker (presets: 4 weeks, 12 weeks, this quarter, last quarter, or a custom range; default is trailing 12 weeks) plus repository and developer multi-select filters. Every card, chart, and tab recomputes under the same scope, so numbers always reconcile with each other. Active filters render as removable chips, and the full scope — time range, repos, developers, active tab — is encoded in the URL, so a filtered view is shareable as a link (a Slack message, a QBR deck) and restores exactly on load.
  • Four stat cards — AI share of merged PRs (with a witnessed/self-declared/manual/automated subline), AI share of merged lines of code, cost per AI PR (witnessed only, with an inline weekly trend sparkline showing which way it's moving — directional only, not gated by the n<5 sample-size check since it isn't a point-in-time comparison), and cycle time (AI vs. manual, hidden behind the n<5 honesty check like every other comparison on the page).
  • Weekly trend chart — a stacked bar of witnessed, self-declared, manual, and automated PRs per week. Witnessed and self-declared are deliberately rendered as the same color at two intensities (not red) — this is a governance win being celebrated, not a risk being flagged. Clicking any bar segment focuses the whole page's scope bar on that bar's week (Monday through the following Sunday) — every card, the chart itself, and every tab recompute under that narrowed week. There's no separate "undo" action: pick a new range or preset in the date picker to move the scope back out.
  • Breakdown / Pull requests / Coverage / Adoption / Identities tabs:
    • Breakdown — one table, toggled between grouping by repository, developer, or team, with the AI-managed percentage, and the witnessed/self-declared split, per row. An Export CSV button downloads whichever grouping is currently active (repository, developer, or team) as a CSV file scoped to the current filters.
    • Pull requests — the full ingested PR list, searchable, with an "AI evidence" column (e.g. "Claude Code · witnessed (2 sessions)" or "Cursor · self-declared"), a Reverted tag on rows that were later reverted, a dedicated Reviews column showing the review-round count and a Copilot tag (showing "Copilot ×N" when Copilot reviewed more than once — it commonly re-reviews after every push, and each submission counts separately from review rounds), and filters for repo and evidence kind.
    • Coverage — the session-to-PR linkage view described above. The repository filter is disabled here, since sessions are scoped to developers, not repos.
    • Adoption — AI-reviewed share, a per-team adoption table (weekly AI-vs-total sparkline plus member breakdown), a "not yet using AI" list of developers with zero AI-assisted merged PRs in scope, and the quality-parity revert/abandon rate comparison from the efficiency metrics above. Admin and viewer accounts are excluded from the "not yet using AI" list even when mapped to a team — they're typically managers or read-only accounts, not developers — though one who genuinely authored a merged PR still appears with real numbers. An Export CSV button downloads every team's member rows (AI PRs, total PRs, AI share, last AI PR date) as raw numeric values, not pre-formatted percentages or strings.
    • Identities — every unresolved GitHub login (see above), sorted by merged PR count so the highest-impact fixes surface first. Each row lets an admin either map the login to a developer's email (an autocomplete suggests known platform users, but any email can be entered — some contributors are only known via commit-email matching and never have a full account) or mark it as a bot the automated-account detection missed. Either action re-attributes every past PR by that login immediately — it isn't limited to PRs merged after the fix — so the Breakdown and Adoption tabs reflect the correct developer and team right away. The tab's count badge disappears once nothing is left unresolved.
  • Drill-down clicks — clicking a developer or repository cell in the Breakdown tab sets that value as a global filter and jumps to the Pull requests tab, so a summary number always leads to the PRs behind it. A separate ↗ affordance on PR rows and on Breakdown cells opens the equivalent view on GitHub itself (the PR, or a GitHub search scoped to that developer's or repository's merged PRs).

Every CSV export follows RFC 4180 quoting and is safe to open directly in Excel or Sheets: any text field that would otherwise be read as a spreadsheet formula (starting with =, +, -, or @) is escaped so it opens as plain text instead of executing.

See also​

Last updated on