Skip to main content

Configure GitHub Delivery Attribution

Tasks

Connect GitHub so AIControls can ingest merged pull requests and link them back to the AI sessions that produced them. See Outcomes for how the attribution model works.

Explore delivery outcomes​

  1. Open Productivity → Outcomes and choose a date range.
  2. Select a project from All projects to focus on its configured repositories. Add repository, developer, or team filters to narrow the work. Configure repository groups in Settings → Projects.
  3. Read the weekly chart's AI and Manual totals. Choose AI breakdown to see Automated, Self-declared, and Witnessed separately; choose Overview to return to two series.
  4. Choose Project, Repository, Developer, or Team under Breakdown. To investigate low AI adoption, click the row's Manual PRs count. The PR list opens with that entity, the same date range, and the Manual and Merged filters applied.
  5. Open a PR row to inspect its attribution evidence and recorded AI reviews. Change or remove the filters above the list to compare categories. Use Back to breakdown to restore the original table, including its search, sort, and page.

The chart's combined AI series includes automation. The AI-share cards and breakdown percentages exclude automated PRs. See Productivity analytics for overlapping projects and coverage scope.

Managed connection availability

The managed GitHub App connection is being introduced through a staged rollout. When enabled for your workspace, it removes the need to create your own App or enter its signing key and global webhook secret. The instructions below describe the existing customer-managed App and personal access token options. They remain available while the managed connection is not enabled.

Choose an authentication method​

GitHub App (recommended)Personal access token
IdentityOrg-level — not tied to any one personTied to the developer who created it
Rate limit5,000 requests/hour per installation5,000 requests/hour, shared with that user's other API usage
Breaks if...The App is uninstalledThe token owner leaves the org, or the token expires/is revoked
Webhook setupAutomatic — GitHub delivers events to every repo the App is installed onManual — you add a webhook on each repository yourself
Setup effortOne-time App creation, then reusable across reposFaster for a single repo / quick test

Use a GitHub App for anything beyond a quick trial. Use a PAT only to evaluate the feature against one or two repos before committing to App setup.

Set up authentication​

  1. In your GitHub organization, go to Settings → Developer settings → GitHub Apps → New GitHub App.

  2. Fill in the required fields:

    • GitHub App name — any name, up to 34 characters (e.g. AIC Delivery Attribution)
    • Homepage URL — your AIControls workspace URL
    • Webhook → Active — checked
    • Webhook URL — https://YOUR_WORKSPACE_URL/webhooks/github
    • Webhook secret — generate a random string and save it; you'll enter the same value in AIControls in a later step

  3. Under Repository permissions, grant:

    • Pull requests — Read-only
    • Contents — Read-only, required for the Push event subscription that supplies repository commit evidence
    • Metadata — Read-only (selected automatically)

    No repository write permissions are required.

  4. Under Subscribe to events, check Pull request and Push.

  5. Set Where can this GitHub App be installed? to Only on this account, then click Create GitHub App.

  6. On the App's settings page, scroll to Private keys and click Generate a private key. A .pem file downloads — keep it safe, you'll paste its contents into AIControls and it can't be downloaded again (you can generate a new one if you lose it).

  7. Note the App ID shown near the top of the same page.

  8. Click Install App in the left sidebar and choose your organization. Grant access to the repositories the integration may discover: either selected repositories or all repositories in that installation. AIControls onboarding rules determine which accessible repositories are collected.

  9. After installing, the URL bar shows .../settings/installations/<NUMBER> — that number is the Installation ID.

You should now have four values: App ID, Installation ID, the private key (.pem contents), and the webhook secret you set in step 2.

When updating an existing App, enable the Push subscription and approve the new Contents permission for its installation. Commit evidence appears after a signed branch-push event for a verified repository; enabling the subscription does not backfill older commits.

Configure the integration in AIControls​

  1. Go to Settings → Integrations, find the GitHub card, and open it. At the end of the setup wizard, the Set up more list on the You're protected screen links to it. See Set up more.

  2. Enter the Webhook secret — the same value you set on the GitHub App or on the repo webhooks above.

  3. Fill in either of the following, not both:

    • App ID, Installation ID, and Private key (PEM), under GitHub App (recommended) — if you're using the GitHub App method (paste the full contents of the downloaded .pem file, including the -----BEGIN/-----END lines)
    • Access token (PAT), under Or a personal access token — if you're using the personal-access-token method

  4. Enable the integration and click Save. Credential configuration is separate from repository onboarding. The Productivity repositories summary links to the inventory and shows onboarded, pending, and skipped counts.

  5. Select Manage repositories, or open Settings → Productivity → Repositories. Configure allowed owners and choose Manual, Discover and review, or Automatic. Onboard repositories through discovery or bulk import. Each newly onboarded repository queues its own initial history collection; the default history is 90 days. Follow Manage Productivity Repositories for the complete workflow.

Repository decisions apply consistently to history collection and webhook processing. Installing an App or receiving a webhook does not bypass the onboarding decision. A configured integration with zero onboarded repositories shows the discovery/setup state.

Verify it worked​

  1. Open the Outcomes page (in the Productivity section of the sidebar). Backfilled pull requests appear within a few minutes, depending on repo size and count.

  2. Open or merge a test pull request in one of the tracked repos, and confirm it shows up on the page shortly after.

  3. If nothing appears, check delivery health:

    • GitHub App: on the App's settings page, go to Advanced → Recent Deliveries and confirm deliveries show a 200 response
    • PAT / webhook: on the repo, go to Settings → Webhooks, click your webhook, and check Recent Deliveries

Re-running backfill after an upgrade​

Detection of self-declared markers (see Witnessed vs. self-declared) was added after Tier 1 shipped. If your workspace was already ingesting pull requests before marker detection was enabled, its historical PRs won't have markers recorded yet — the PR bodies and commit messages needed to detect them weren't stored at ingestion time.

You don't need a new integration or a fresh install to pick this up:

  1. Go to Settings → Integrations, open the GitHub card, and click Backfill again (or use the Backfill button on the Outcomes page itself). This re-fetches the trailing 90 days of merged pull requests and their commits through the same ingestion path as the original setup.

  2. As PR bodies and commit messages are re-fetched, marker detection runs against them and existing rows are updated in place — no PRs are duplicated or re-counted.

  3. Your historical AI-managed percentage may increase after this completes: PRs produced by tools that don't route through the proxy (for example, a session using a direct provider API key) but that carry a self-declared marker will move from "Unknown" into "self-declared."

note

This only affects the trailing 90-day backfill window. PRs merged further back were never ingested and won't gain markers retroactively.

Recover historical Codex attribution​

Backfill also checks the trailing 90 days of retained Codex server captures for commit results. Candidate PRs must fall within the same window, using the merge date or, for unmerged PRs, the opening date. It uses only intact tool-result deltas with the original recorded session identity, matches commit SHAs to ingested PRs, and preserves existing links. Quoted examples, assistant prose, inherited conversation snapshots, and missing or expired captures are skipped. When new links are recovered, leaderboard facts and scores are queued for recomputation.

This requires full capture to have been enabled when the affected requests were recorded and the corresponding payloads to remain available. Enabling capture now cannot recreate past requests. Recovery does not upload local Codex transcripts or alter GitHub PRs and commits.

If the evidence was never captured, the PR remains Unknown. An author can add an accurate self-declaration such as Generated by OpenAI Codex to the PR description and rerun backfill. This records a self-declared signal, not a witnessed session. GitHub backfill alone cannot infer missing AI authorship.

Configure a project's repositories​

  1. Open Settings → Productivity → Repositories and onboard the repositories you want to group. Session discoveries in Pending review must be onboarded first.
  2. Open Settings → Projects and select Add Project or edit an existing project.
  3. Enter the project name, optional description, and optional Owning team. The team is metadata; it does not restrict contributors or grant access.
  4. Search by host, provider, owner, or repository. Select across 10-, 25-, or 50-row pages, or choose Select all matching repositories for the full search result. The table checkbox selects the visible page only. You can select up to 1,000 repositories, including quiet repositories and those needing access or collection attention. Open View selected repositories to inspect or remove selections.
  5. Select Create or Save. Project membership groups attribution; it does not start, stop, or change repository collection.

Use Manage repository onboarding in the editor to open repository management in a separate tab. Existing assignments remain selected if a repository becomes skipped or is absent from the inventory. You can keep or remove them without onboarding them again. To add a removed repository again, onboard it first.

The Projects table shows the configured repository count. Open Edit or History to inspect the repository names.

Changes save with the revision you opened. If another administrator saves first, reload the project and reapply your changes. Your stale form cannot overwrite the newer membership.

Choose History to inspect the observed baseline and subsequent revisions, including the actor, time, repository additions/removals, owner, and status. History known from marks the first captured state; membership before that time is unknown. Repository renames preserve the saved repository ID. You can share the history drawer’s URL; opening it still requires administrator access.

If the repository count shows Unknown, its earlier repository selection could not be read. Open Edit, review the repository selection, and save it explicitly. The repaired selection applies from that revision; the earlier history remains unknown.

Set Status to Archived to retain the project configuration and history without stopping repository collection. Use Archived projects to find it later. Delete still removes the project configuration and its membership history; it does not delete the recorded sessions or PR evidence.

Group unmapped repos into a project​

A project groups one or more repos so session cost and activity roll up together on the Projects view of Productivity → Usage (/ui/usage). As soon as AIControls sees developer activity in a repo that isn't covered by any project yet, it appears as an unmapped repo — a N Repos Detected button shows above the Projects table. Open a project to see its page; the breadcrumb above the title (Usage, Projects, then the project name) takes you back to the Projects view.

The Contributors column of the Projects table and the contributor lists on a project's page show people only. Anonymous agents (anon:*), service accounts (sa:*), Kubernetes agents, and any owner that matches an agent in Inventory → Identities by its ID, agent ID, or display name are left out; an email address always counts as a person. The project's session and cost totals still include every session, so they can be higher than the sum of the listed contributors. The Avg sessions per contributor and Avg cost per contributor figures on a project's page use only the listed people's sessions and cost, divided by the number of people listed.

  1. Click N Repos Detected to open the Unmapped Repos drawer, and select one or more repos from the list. All selected repositories must be onboarded before you can create a project or add them to an existing one. Use Manage repository onboarding to review pending discoveries or restore skipped repositories, then return to the drawer.

  2. Under Add to, either:

    • Leave it on New project to group the selected repo(s) into a brand-new project (single repo: named automatically from the repo slug; multiple repos: you're prompted for a name), or
    • Pick an existing project to merge the selected repo(s) into that project's repo list instead.

  3. Click Create Project (or Add to Project, if an existing project is selected). The repo no longer shows as unmapped, and its historical sessions are re-attributed to the project.

Adding detected repositories preserves the project’s existing repository IDs, including unavailable or renamed members. If its membership needs repair, use Project settings to review and save the complete selection first.

Suggestions labeled Repository identity not detected are project labels without a usable repository name. Use Configure projects to select the actual onboarded repositories, or dismiss the suggestion. These labels are not sent to repository onboarding.

Repos you don't want tracked as a project can be dismissed from the same drawer — dismissed repos stop being suggested as project candidates. Dismissing a project suggestion does not skip the repository or stop collection.

Map GitHub accounts to gateway users​

Administrators manage reporting identities under Settings → Users → Identity mappings. The External identities column in the Users table also opens the mappings for an individual gateway user. One gateway user can have several external accounts; each external account maps to one person or is classified as a bot. No linked identities is informational and does not mean the user is misconfigured.

  1. Open Settings → Users → Identity mappings, then choose Unmapped. Search by external login or email. The count and pagination cover the complete matching list.
  2. Open an identity row. Review its provider, PR count, and any suggested match.
  3. Choose a Gateway user email, or enter an email for a contributor without a gateway account. To classify an automated account, choose Bot as the account type.
  4. Click Save mapping. If the account already belongs to another person or is classified differently, review and confirm the change.

Use Link identity to add a GitHub login that has not yet appeared in reports. Use the Mapped and Bots filters to review existing mappings. Open a row to see its match method, confirmation time, and confirming actor. Unlink identity removes the association after confirmation; automatic matching may link it again later if supporting evidence still exists.

GitHub logins match regardless of letter casing. For example, a mapping for jimbugwadia also attributes PRs imported as JimBugwadia. PR details retain the original GitHub display login. Reports and leaderboards group unresolved GitHub authors under the lowercase login, so case variants appear as one account. Unresolved accounts from other providers include the provider in their reporting key (for example, gitlab:Alice) and retain the original login casing, so they cannot be combined with an unrelated GitHub account.

If older mappings contain case variants confirmed to different people, or to both a person and a bot, the account appears as Conflicting mappings. AIControls preserves the saved owners and pauses attribution instead of choosing one. Open the identity to review them, select Unlink identity, confirm, and then save the correct mapping. Unlinking clears all GitHub case variants; the mapping history remains available.

Reports label unresolved authors with Unmapped identity and provide a Map to user link to Settings. Links from the productivity comparison preserve the report filters through Back to report. Existing bookmarks to the old Outcomes identity tab redirect to Settings.

Mappings apply across all dates, teams, and projects for repositories eligible for productivity collection. They apply to existing PRs when saved and to new PRs as they are imported. Background reconciliation repairs older PRs that missed a saved mapping, including accounts explicitly marked as bots. Skipped or historical-only repository evidence is not changed. Saving or unlinking a mapping queues affected PRs for attribution and derived-metric refresh, so those metrics may take time to catch up. The original external login remains available in PR details. A reporting mapping does not grant gateway access or connect an OAuth credential. Viewers can inspect mappings but only administrators can change them.

Automatic identity resolution​

An unresolved login (see Repository eligibility and attribution) doesn't require an admin to manually map every one. AIControls tries several automatic sources first, in order of confidence, before falling back to a manual or self-service fix:

SourceWhat it usesWhat it requires
vcs_tokenA GitHub token observed at the proxy — the login and email pair is confirmed by calling GitHub's own APIAny GitHub-authenticated session that routes through the proxy
sso_lookupThe org's SAML external identities, matched against the GitHub loginGitHub App with Organization members: read-only and Organization administration: read-only
verified_domainMember emails on the org's verified domain(s) — only when a member has exactly one verified-domain emailGitHub App with Organization members: read-only and Organization administration: read-only
commit_emailA commit email matches one known AIControls user. Always suggests a mapping for admin confirmation, even for signed commits or emails seen in gateway sessions: a PR can contain another contributor's commitsNo extra permission — uses data already ingested with each PR
session_linkagePR sessions trace back to one AIControls user. Always suggests a mapping for admin confirmation: the session user can differ from the person who opened the PRNo extra permission — derived from existing session data
noreply_witnessedThe commit author's email is a GitHub-generated noreply address, confirmed when every proxy-witnessed commit under that login traces back to a single AIControls userNo extra permission — uses data already ingested with each PR

Commit-email and PR-session suggestions never attribute PRs or change an existing confirmed mapping. Review the suggestion under Settings → Users → Identity mappings, choose the correct gateway user, and select Save mapping to confirm it.

When you upgrade, existing automatic commit_email and session_linkage mappings become unconfirmed suggestions. Admin-confirmed mappings, connected GitHub accounts, and other account-based matches remain in place. Affected eligible PRs return to unresolved and their derived metrics refresh in the background. If repository setup is pending or productivity is disabled, identity maintenance completes the repair after the repositories become eligible. Skipped and historical-only repository evidence stays unchanged.

sso_lookup and verified_domain both call GitHub's organization API, so each needs both GitHub App permissions above — granting only one leaves both lookups skipped. Both permissions together are optional: without them, the two lookups are skipped silently — no error is shown, and resolution just falls through to the remaining sources. The sweep that runs these lookups executes once daily and again immediately after every backfill.

Developer self-service​

Developers can also connect their own GitHub identity, without waiting for an automatic match or an admin to map them manually. This is available both to signed-in portal users (admin or viewer accounts) on the Connected Accounts page, and to Builder Portal developers on their dashboard's Integrations tab:

  1. An admin fills in Client ID and Client Secret on Settings → Integrations → GitHub. These are the same GitHub App's OAuth credentials — find them on the App's settings page under Client secrets.

  2. Once set, a Connect GitHub card appears: under Setup → Integrations → Connected Accounts for admin and viewer users, and on the Integrations tab of the Builder Portal dashboard for developers signed in with their personal access token.

  3. The user clicks it, authorizes the App on GitHub, and their identity is linked — one click plus the GitHub authorization screen.

note

In an HA (multi-replica) deployment, every replica must share the same encryption key for this flow to work — see the encryption key guidance for your deployment.

Suggestions​

When none of the automatic sources resolve a login, Settings → Users → Identity mappings may still show a pre-filled suggestion in its detail drawer. A suggestion only appears when the login's email local-part or name pattern is an exact match after normalization (case and punctuation stripped) to exactly one known user — this is not fuzzy or similarity-based matching, and an ambiguous match (more than one user could fit) produces no suggestion at all. Review the suggested email and choose Save mapping; saving records the mapping as an admin_override.

Manual mapping saves and deletions require successful configuration audit persistence. A failed save preserves the previous mapping and pull-request attribution. Managed GitHub integration saves also preserve the previous credentials on failure; reload the form before retrying a save rejected by a concurrent credential change.

Troubleshooting​

"Configure either an access token or a GitHub App — not both"​

The integration form rejects a config with both a PAT and App fields filled in, to avoid ambiguity about which credential is actually used. Clear whichever method you're not using.

Webhook deliveries show a signature failure​

The Webhook secret in AIControls must exactly match the secret configured on the GitHub App (or on every repo's webhook, for the PAT method). Re-enter it on both sides if you're not sure they match.

Backfill shows 0 pull requests​

Open Settings → Productivity → Repositories. Confirm each repository is Onboarded, in scope, and has GitHub access. Inspect its collection state and errors. For a GitHub App, check Install App → Configure to see which repositories are accessible. Use Sync selected to queue collection for eligible onboarded repositories. A skipped or pending repository is not collected, including through webhooks.

See also​

Compare developers on a project​

  1. Open Productivity → Outcomes and choose Overview, Productivity, or AI Effectiveness.
  2. Select a searchable Project scope, then set Compare to Developers.
  3. Choose a common relative or custom period and up to eight project contributors. Leave team scope empty to include contributors from different teams.
  4. Optionally select a repository, team, work type, or target branch. Review the included repositories and coverage before interpreting the charts.
  5. Select a chart bar to inspect scoped PR evidence. Use Export comparison CSV for the selected data or bookmark the URL to retain the comparison.

See Developers within one project for metric semantics, missing evidence, and shared-monorepo limitations.

Last updated on