One pull request, from the AI’s commit to the merge
Take one pull request an AI agent wrote. This is what happens to it between opening and merging, what your engineers do along the way, and what your team learns from it.
For the engineer
One pull request, start to finish
Pull request 4127 in acme/payments-api, written with Claude Code: eleven lines added to src/dispatcher.ts and one removed.
Written with Claude Code: retries with backoff for the webhook dispatcher. Eleven lines added and one removed.
Someone will approve it. What the diff cannot tell anyone is whether the person who approves it understood it.
A risk score of 68 against a threshold of 60, and what it is made of: a sensitive path, reach, the kind of change, history and size. The check is waiting for answers.
It is scored as soon as it opens, from what it changes: the paths, how far the change reaches, what kind of change it is, the files’ history and its size. Which AI tool wrote it is recorded, and never moves the score.
This one scores 68, over your threshold of 60, so the check waits for answers. A pull request under your threshold is never touched.
Question 1 of 2, above the six lines of src/dispatcher.ts it is about, highlighted.
The questions come from the diff and point at specific lines. Here: what happens to a webhook that has already failed four times?
Someone who read the change answers in a couple of minutes. The check on the pull request links straight to the questions, so there is no new tool to learn.
The reviewer’s answer about lines 42 to 47, and the AI’s code rated Mixed.
In your own words, what happens to a webhook that has already failed four times?
Answer · lines 42 to 47
After four attempts it stops retrying — it parks the event as “exhausted”, bumps webhook.parked and returns before posting, so the caller gets nothing back and no other system is told. The delay line below it never runs for that event.
How does the AI’s code read here?
Rates the code, not the person who wrote the prompt.
04
Answer, then rate the AI
The reviewer answers in their own words. An answer that misses brings another question on the same lines, and a hunk nobody can explain is recorded as a finding about the AI.
Then they rate how the AI’s code reads: good, mixed or poor. It is the only rating in the product, and its subject is always the code.
The required check deslopping/comprehension has passed with both questions answered, and the pull request is ready to merge.
deslopping/comprehension2 of 2 questions answeredRequired
ci / buildSuccessful
All checks have passed
Break-glass, when production is down: one reason, a logged use, and the review still owed within 24 hours.
05
The check turns green
Once its questions are answered, deslopping/comprehension passes and the merge goes ahead.
When production is down, break-glass is one short reason away for the people allowed to use it. Every use is logged, and the review is still owed within 24 hours.
For the team
The AI, made manageable
How ratings of the AI’s code split into good, mixed and poor for four modules, with the tool that wrote each. A fifth module shows only “Not enough reviews yet”.
deslopping.com/acme/ai-performanceSample data · 06 / 08
AI performance
Last 90 days, by moduleGoodMixedPoor
payments/webhooksClaude Code84
billing/invoicesCursor61
auth/sessionClaude Code47
sdk/public-apiCopilotNot enough reviews yet
infra/terraformClaude Code38
06
Where the AI writes badly
Every rating is about a specific hunk of code, so they add up: by module, to where the AI writes code people can’t follow, and by tool, with Claude Code, Cursor and Copilot on the same scale.
Nothing shows below five reviewers. A module with fewer reads “Not enough reviews yet”, even when you know who they are.
Coming next: a rule for webhook handlers, and how ratings of payments/webhooks split in the 30 days before it and the 30 days after.
deslopping.com/acme/ai-performanceSample data · 07 / 08
AI performance
Last 90 days, by moduleGoodMixedPoor
Rule · CLAUDE.md · coming next
Webhook handlers: a retry needs a cap, a dead-letter path and a metric.
Will arrive as a pull request from the rules app, if you opt in
payments/webhooks
30 days before
30 days after
07 · Coming next
A rule, and what it changed
When a module keeps reading poorly, you write one line into your agent’s instructions. A second app you opt into delivers it to your repositories as a pull request, and the same chart shows what changed after it.
The colours stay neutral: the chart shows the shape, not a verdict that the rule worked.
The promise
What nobody sees
Two views: the reviewer’s own answer, which only they can read, and the lead’s dashboard, which shows modules and tools and no names.
deslopping.com/acmeSample data · 08 / 08
Who can read what
The same pull request, from both sides
What the reviewer sees
Your answer · lines 42 to 47
Encrypted, and readable by you alone
Export it or delete it any time. It is gone after your retention window.
What the lead sees
payments/webhooks
billing/invoices
auth/session
sdk/public-apiNot enough reviews yet
Modules and tools. No names, and nothing below five reviewers.
08
No score on any person
Answers stay with whoever wrote them: encrypted, theirs to read, export or delete, and gone after your retention window. Leads see modules and tools, never a name.
No per-person score, ranking or streak is computed anywhere. The dashboards read through a database role that cannot reach an answer, so a query for one person’s words fails at the database.
acme/payments-api #4127deslopping.com/acme/ai-performancedeslopping.com/acmeSample data · 01 / 08
In your own words, what happens to a webhook that has already failed four times?
Answer · lines 42 to 47
After four attempts it stops retrying — it parks the event as “exhausted”, bumps webhook.parked and returns before posting, so the caller gets nothing back and no other system is told. The delay line below it never runs for that event.
How does the AI’s code read here?
Rates the code, not the person who wrote the prompt.
deslopping/comprehension2 of 2 questions answeredRequired
ci / buildSuccessful
All checks have passed
Break-glass, when production is down: one reason, a logged use, and the review still owed within 24 hours.
payments/webhooksClaude Code84
billing/invoicesCursor61
auth/sessionClaude Code47
sdk/public-apiCopilotNot enough reviews yet
infra/terraformClaude Code38
Rule · CLAUDE.md · coming next
Webhook handlers: a retry needs a cap, a dead-letter path and a metric.
Will arrive as a pull request from the rules app, if you opt in
payments/webhooks
30 days before
30 days after
What the reviewer sees
Your answer · lines 42 to 47
Encrypted, and readable by you alone
Export it or delete it any time. It is gone after your retention window.
What the lead sees
payments/webhooks
billing/invoices
auth/session
sdk/public-apiNot enough reviews yet
Modules and tools. No names, and nothing below five reviewers.
The beta is small on purpose
We are taking a handful of teams at a time so the questions stay good. Tell us what you ship with and we will tell you honestly whether you are a fit.
Design partners get half off Team for their first year. Read-only on your code; rules for your AI agents come next, as pull requests from a second app you opt into.