Faster Coding Is Not Faster Delivery: Measure What Your Coding Assistants Changed in 30 Minutes

Why it matters now

The 2025 DevOps Research and Assessment report, based on nearly 5,000 technology professionals, describes artificial intelligence as an amplifier. Teams with good platforms, reliable tests and small batches become stronger. Teams with slow reviews and manual releases get more of the same, faster.^1,2

In 2025 a randomised trial by METR found that experienced open-source developers took 19% longer with these tools on their own repositories, while they believed they were 20% faster.⁴ In February 2026 METR wrote that its newer data gives an unreliable signal, because more developers declined to work on study tasks without these tools.^5,6

Delivery is a sequence of queues: review, continuous integration, tests, release approval and deployment. When pull requests arrive faster and review capacity stays the same, they wait longer. A measurement of your own system shows where the gain stops.

Check your delivery numbers in 30 minutes

Step 1. Pick the five numbers

Median time from pull request opened to merged, median time to first review, time from merge to production, change failure rate, and pipeline duration with rerun rate. Use medians, since one pull request that stayed open for three months distorts an average. Compare the same quarter across two years to remove seasonal effects.

Step 2. Median hours from opened to merged

This uses the GitHub command-line tool and jq for one repository over a quarter.

gh pr list --repo ORG/REPO --state merged --limit 1000 \
  --search "merged:2026-07-01..2026-09-30" --json createdAt,mergedAt \
  | jq '[.[] | select(.mergedAt != null and .createdAt != null)
        | ((.mergedAt | fromdateiso8601) - (.createdAt | fromdateiso8601)) / 3600]
        | sort | .[length / 2 | floor]'

Run it again with merged:2025-07-01..2025-09-30 for the same quarter last year.

Step 3. Median hours to first review

gh pr list --repo ORG/REPO --state merged --limit 1000 \
  --search "merged:2026-07-01..2026-09-30" --json createdAt,reviews \
  | jq '[.[] | select(any(.reviews[]; .submittedAt != null))
        | ((.reviews | map(select(.submittedAt != null) | .submittedAt | fromdateiso8601) | min)
           - (.createdAt | fromdateiso8601)) / 3600]
        | sort | .[length / 2 | floor]'

On GitLab, the merge requests interface returns created_at and merged_at for the same calculation:

curl -s --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
  "https://gitlab.example.com/api/v4/projects/PROJECT_ID/merge_requests?state=merged&updated_after=2026-07-01T00:00:00Z&per_page=100"

Step 4. Add release, failure and pipeline data

Take the time from merge to production from deployment events or release tags, the change failure rate from incidents, rollbacks and hotfixes per deployment, and pipeline duration and reruns from your continuous integration history.

What the result means

The plan, quarter by quarter

1. Rebuild the earlier picture. Pull the five numbers for the same quarter last year and this year for your main repositories.
2. Find where work waits. The longest wait, in review, pipeline or release, is the constraint to address first.
3. Fix one constraint. Pick one change, such as smaller pull requests, a reviewer on duty or quarantined unstable tests, and run it for one quarter.
4. Measure every quarter. Report the same five numbers each quarter so that the trend guides decisions and budget.

What is still open

• The studies differ in size, method and population, so their numbers describe tendencies. Your own measurement is the reliable guide for your organisation.
• Tools change quickly, and METR could not repeat its 2025 trial with reliable results. Read its February 2026 update before you quote the 19%.^5,6
• Deployment and incident data often sit in other systems, so the merge-to-production number may need a manual sample at first.
• The same research programme published a separate report on the return on investment of assisted development in April 2026, for teams that need to defend a budget.⁷

A note to forward to your team

Coding assistants and delivery: our numbers

Research links artificial intelligence adoption to higher throughput and lower stability. We want to know where our gain goes.

Please run the queries in this note on [repositories] for July to September 2025 and July to September 2026, and reply by [date] with five numbers for each quarter:
1. Median hours from pull request opened to merged
2. Median hours to first review
3. Time from merge to production
4. Change failure rate
5. Pipeline duration and rerun rate

Owner: [name]
Note: https://stonetusker.com/insights/ai-delivery-baseline.html