Skip to main content

Configure GitHub Delivery Attribution

Tasks

Connect GitHub so AIControls can ingest merged pull requests and link them back to the AI sessions that produced them. See Productivity for how the attribution model works.

Choose an authentication method​

GitHub App (recommended)Personal access token
IdentityOrg-level — not tied to any one personTied to the developer who created it
Rate limit5,000 requests/hour per installation5,000 requests/hour, shared with that user's other API usage
Breaks if...The App is uninstalledThe token owner leaves the org, or the token expires/is revoked
Webhook setupAutomatic — GitHub delivers events to every repo the App is installed onManual — you add a webhook on each repository yourself
Setup effortOne-time App creation, then reusable across reposFaster for a single repo / quick test

Use a GitHub App for anything beyond a quick trial. Use a PAT only to evaluate the feature against one or two repos before committing to App setup.

Set up authentication​

  1. In your GitHub organization, go to Settings → Developer settings → GitHub Apps → New GitHub App.

  2. Fill in the required fields:

    • GitHub App name — any name, up to 34 characters (e.g. AIC Delivery Attribution)
    • Homepage URL — your AIControls workspace URL
    • Webhook → Active — checked
    • Webhook URL — https://YOUR_WORKSPACE_URL/webhooks/github
    • Webhook secret — generate a random string and save it; you'll enter the same value in AIControls in a later step

  3. Under Repository permissions, grant:

    • Pull requests — Read-only
    • Metadata — Read-only (selected automatically)

    No other permissions are required — AIControls only reads pull request and commit metadata, never file contents.

  4. Under Subscribe to events, check Pull request.

  5. Set Where can this GitHub App be installed? to Only on this account, then click Create GitHub App.

  6. On the App's settings page, scroll to Private keys and click Generate a private key. A .pem file downloads — keep it safe, you'll paste its contents into AIControls and it can't be downloaded again (you can generate a new one if you lose it).

  7. Note the App ID shown near the top of the same page.

  8. Click Install App in the left sidebar, choose your organization, select Only select repositories, and check every repository you want tracked.

  9. After installing, the URL bar shows .../settings/installations/<NUMBER> — that number is the Installation ID.

You should now have four values: App ID, Installation ID, the private key (.pem contents), and the webhook secret you set in step 2.

Configure the integration in AIControls​

  1. Go to Settings → Integrations, find the GitHub card, and open it.

  2. Enter the Webhook secret — the same value you set on the GitHub App or on the repo webhooks above.

  3. Fill in either of the following, not both:

    • Access token (PAT) — if you're using the personal-access-token method
    • App ID, Installation ID, and Private key (PEM) — if you're using the GitHub App method (paste the full contents of the downloaded .pem file, including the -----BEGIN/-----END lines)

  4. Under Repositories to backfill, add each repository as owner/name (e.g. nirmata/aicontrols), pressing Enter after each one. This list controls which repos are included in the 90-day historical backfill — webhook deliveries are ingested regardless of this list, as long as the App (or webhook) is installed on that repo.

  5. Click Save. AIControls validates the credentials, and a 90-day backfill of merged pull requests starts automatically in the background.

Verify it worked​

  1. Open the Productivity page (in the Dashboards section of the sidebar). Backfilled pull requests appear within a few minutes, depending on repo size and count.

  2. Open or merge a test pull request in one of the tracked repos, and confirm it shows up on the page shortly after.

  3. If nothing appears, check delivery health:

    • GitHub App: on the App's settings page, go to Advanced → Recent Deliveries and confirm deliveries show a 200 response
    • PAT / webhook: on the repo, go to Settings → Webhooks, click your webhook, and check Recent Deliveries

Re-running backfill after an upgrade​

Detection of self-declared markers (see Witnessed vs. self-declared) was added after Tier 1 shipped. If your workspace was already ingesting pull requests before marker detection was enabled, its historical PRs won't have markers recorded yet — the PR bodies and commit messages needed to detect them weren't stored at ingestion time.

You don't need a new integration or a fresh install to pick this up:

  1. Go to Settings → Integrations, open the GitHub card, and click Backfill again (or use the Backfill button on the Productivity page itself). This re-fetches the trailing 90 days of merged pull requests and their commits through the same ingestion path as the original setup.

  2. As PR bodies and commit messages are re-fetched, marker detection runs against them and existing rows are updated in place — no PRs are duplicated or re-counted.

  3. Your historical AI-managed percentage may increase after this completes: PRs produced by tools that don't route through the proxy (a session on a direct API key, or a tool other than Claude Code) but that carry a self-declared marker will move from "manual" into "self-declared."

note

This only affects the trailing 90-day backfill window. PRs merged further back were never ingested and won't gain markers retroactively.

Group unmapped repos into a project​

A project groups one or more repos so session cost and activity roll up together on the Activity → Projects tab. As soon as AIControls sees developer activity in a repo that isn't covered by any project yet, it appears as an unmapped repo — a N Repos Detected button shows above the Projects table.

  1. Click N Repos Detected to open the Unmapped Repos drawer, and select one or more repos from the list.

  2. Under Add to, either:

    • Leave it on New project to group the selected repo(s) into a brand-new project (single repo: named automatically from the repo slug; multiple repos: you're prompted for a name), or
    • Pick an existing project to merge the selected repo(s) into that project's repo list instead.

  3. Click Create Project (or Add to Project, if an existing project is selected). The repo no longer shows as unmapped, and its historical sessions are re-attributed to the project.

Repos you don't want tracked as a project can be dismissed from the same drawer — dismissed repos stop being suggested as candidates.

Troubleshooting​

"Configure either an access token or a GitHub App — not both"​

The integration form rejects a config with both a PAT and App fields filled in, to avoid ambiguity about which credential is actually used. Clear whichever method you're not using.

Webhook deliveries show a signature failure​

The Webhook secret in AIControls must exactly match the secret configured on the GitHub App (or on every repo's webhook, for the PAT method). Re-enter it on both sides if you're not sure they match.

Backfill shows 0 pull requests​

Confirm the repositories are spelled correctly as owner/name under Repositories to backfill, and that the GitHub App (or PAT) actually has access to them — for a GitHub App, check Install App → Configure to see which repos are currently selected.

See also​

Last updated on