In September 2026, more than 40% of pull requests on GitHub began with an agent-generated first draft. That figure was 17% in March, and GitHub notes it is a floor, since not every agent can be detected. That shift makes AI coding agent productivity a management question. How much of the extra code becomes reliable software in production? A credible answer must cover review, testing, deployment, and maintenance.
The evidence is mixed, and for good reasons. Agents can speed up implementation while sending more code into review. Gains also vary with task complexity, codebase maturity, developer experience, and downstream capacity. So the best measure covers the whole delivery process. This article reviews recent research and offers a practical way to track AI coding agent ROI for CTOs, VPs of Engineering, and delivery leaders.
Key Takeaways
- AI coding agent productivity is quality-adjusted delivery. It counts work that is completed, reviewed, released, and sustained, and it does not count code volume alone.
- A Harvard working paper found that pull requests rose 35.7% after agent adoption, while resolved Jira issues rose 13.7%. Review time rose 56.1%.
- Okta saw 37% more pull requests per engineer and 16% shorter PR cycle times. That study covers one repository and is preliminary.
- A useful AI coding agent ROI model uses cost per accepted change. It includes review, retries, testing, and rework, plus the cost of the tools.
- Hours saved are capacity, and they become savings only with evidence of an economic benefit.
What Metrics Show AI Coding Agent Productivity?
The most useful answer is quality-adjusted delivery. That means work that is completed, reviewed, released, and sustained without heavy rework or failures. Teams should track the full change lifecycle. They should pair delivery outcomes with the time and money spent to produce them.
A practical scorecard uses five categories:
| Measurement area | Useful indicators | What the measures reveal |
|---|---|---|
| Completed delivery | Accepted work items, lead time, deployment frequency | Whether useful work reaches production |
| Review and validation | Review queue time, active review time, review rounds, first-pass CI acceptance | Whether generated changes create downstream congestion |
| Quality and stability | Reopened issues, corrective changes, change failure rate, escaped defects | Whether speed gains survive release |
| Engineering effort | Human review hours, debugging time, agent retries, CI usage | How much work the apparent speedup requires |
| Business value | Outcomes delivered, avoided rework, cost per accepted change, customer impact | Whether the investment creates value beyond code volume |
These categories matter together. For example, a team may merge more pull requests and still take longer to deliver a feature. On the other hand, extra review effort can be worth it when the result is fewer production incidents.

What Does the 2026 Research Say About AI Coding Agents?
Recent studies give different answers because they observe different companies, tools, and stages of work. Their findings should stay separate. They should not merge into one industry-wide estimate.
| Research | Observed result | Important limitation |
|---|---|---|
| GitHub Octoverse 2026 | More than 40% of pull requests had an agent first draft in September, against 17% in March | Measures adoption and activity, and counts only detectable agents |
| Chen and Stratton, Harvard working paper (November 2026 revision) | After agent adoption, pull requests opened rose 35.7%. Jira issues resolved rose 13.7%, a borderline result | Observational study of 655 firms. Completed work grew less than code output |
| Same Harvard study | Time from review submission to approval rose 56.1%. Review comments per change rose 38.6% | Estimates describe the sample and adoption window |
| Okta engineering study | Monthly pull requests per engineer rose 37%. Average PR cycle time fell 16% after a Claude Code rollout | One repository, 295 engineers, preliminary. Measures output, not value |
What the Studies Actually Measure
The Harvard analysis uses 445 million work events from 720,542 engineers at 655 companies, as summarized by BERI. Its authors find that coding agents raise code production a lot, while finished output grows more modestly. They name review as a constraint, because changes wait longer for attention and need more effort once review starts.
That finding does not contradict Okta’s shorter cycle times. Okta’s study compares three months before and after a Claude Code rollout in one large repository. The author calls it “a preliminary look” and says it measures code produced, not the quality or value shipped. The two studies cover different companies, periods, and units of performance. For engineering leaders, the gap is useful. The impact of an agent depends on the delivery system around it.
The DORA AI Capabilities Model takes the same system-level view. It names seven capabilities that amplify the benefits of AI. They include working in small batches, a clear AI stance, AI-accessible internal data, strong version control, and quality internal platforms. Those practices decide whether faster coding becomes better delivery.
Why More Pull Requests May Not Mean More Shipped Software
Pull requests are a useful signal, since each one is a change sent for review. However, their count cannot show product value or engineering efficiency on its own.
An agent can open a pull request fast. The change may still need architecture review, security checks, test coverage, integration fixes, and release coordination. In a mature product, dependencies and backward compatibility can dominate the final stages.
The Harvard paper separates two causes of longer reviews. One is queue time, when a change waits for a reviewer. Another is active review time, when reviewers inspect and discuss the code. According to the study, both grew after agent adoption.
This distinction points to different fixes. For a queue problem, smaller batches, clear review ownership, or new assignment rules may help. In contrast, a long active review may point to weak acceptance criteria, extra complexity, thin tests, or changes that break the existing architecture.
Measure Review Queue Time and Active Review Time Separately
Teams can track review performance from repository events:
- Review queue time: time from PR submission to the first substantive review.
- Active review time: time from the first substantive review to approval or merge.
- Review iterations: the number of rounds with requested changes.
- Reviewer effort: the human time spent reviewing, when it can be measured reliably.
Elapsed time and human effort should stay separate. A pull request can wait two days for scheduling reasons and still need only 20 minutes of attention. In contrast, a fast first response can lead to several hours of hard technical work.
Which Delivery Metrics Matter Beyond Code Output?
Track Completed Work and Delivery Lead Time
A completed feature, a resolved defect, or an accepted work item ties more closely to business delivery than lines of code. Even so, teams should group work items by comparable task type and complexity.
Lead time can run from an agreed starting event, such as the start of development, to release in production. Teams should document the definition and use it the same way every time. DORA’s delivery approach also links throughput with stability, so it does not reward frequent deployments with bad outcomes.
Track Rework and Post-Release Quality
Rework is the effort spent fixing changes after the first implementation. Therefore, useful signals include reopened issues, follow-up fixes within 30 days, reverted changes, escaped defects, and production incidents tied to a release.
Attribution needs care. Some failures come from unclear requirements, dependent services, or old code. An AI-generated change should pass the same quality gates as a human-written one. The same rules should assign defects and rework to both.
Track Review Burden and Senior Engineering Capacity
More agent-written code can shift work toward experienced reviewers. In the Harvard sample, 82.8% of reviews went to engineers already familiar with the code. Random assignment would have given them 17.2%.
That makes senior review time a key constraint, so teams should watch how review load is spread. They should also check whether specialists lose time needed for architecture, mentoring, incidents, or hard development. Sending every approval to a small group creates a bottleneck, even when agents speed up coding.
Track Full Engineering Cost
Licenses and model tokens are the visible costs. However, they do not make up the full cost of delivery. Other costs include human verification, agent retries, CI runs, test environments, debugging, security checks, onboarding, and maintenance.
IBM’s guidance on measuring the ROI of AI-assisted development points the same way. It favors cost per outcome over token counts. It also says freed capacity should not be reported as cost savings, and that payback should be shown as a range.
For AI coding agent productivity, a useful operating indicator is cost per accepted change:
Cost per accepted change = (AI tools + compute + human implementation + review + testing + rework costs) / accepted changes
This is a team measurement model, not a universal accounting standard. Also, it should compare similar types of changes and count failed attempts in the cost. Business ROI needs one more step. The team must estimate the value of the outcomes and compare it with the full investment.
How to Calculate AI Coding Agent ROI: A Hypothetical Example
A financial ROI estimate starts with a defensible baseline. Consider two eight-week periods, each with 100 accepted changes of similar scope. Assume the same acceptance criteria and no drop in post-release quality. All figures below are illustrative assumptions. They are not results from Folder IT, GitHub, Okta, or Harvard.
| Cost or outcome | Without agents | With agents |
|---|---|---|
| Comparable accepted changes | 100 | 100 |
| Engineering, review, testing, and rework hours | 1,000 | 920 |
| Assumed fully loaded cost per hour | $100 | $100 |
| Human delivery cost | $100,000 | $92,000 |
| Agent licensing and compute | $0 | $3,000 |
| Training and implementation | $0 | $1,000 |
| Total attributable cost | $100,000 | $96,000 |
| Cost per accepted change | $1,000 | $960 |
In this scenario, 80 fewer engineering hours equal $8,000 of potential gross value. The extra AI investment totals $4,000. If those 80 hours produce a documented financial benefit, the net benefit is $4,000 and the cost-based ROI is 100%:
ROI = (verified gross benefit − incremental AI investment) / incremental AI investment × 100 = ($8,000 − $4,000) / $4,000 × 100 = 100%
The qualification matters. Salaried hours saved do not become cash savings on their own. The company needs proof of avoided spending, more sellable delivery, or another measurable economic benefit. Without it, this example shows a possible capacity gain and no verified financial ROI. A higher defect rate or a different task mix would also invalidate the comparison.
That is why AI coding agent productivity needs two levels of review. The operational level measures accepted work and effort. The financial level needs evidence that the efficiency gain created economic value.
How to Run an AI Coding Agent Productivity Pilot
A useful evaluation can start with an eight-week observation plan. The steps below are a proposed method. They do not report Folder IT’s internal results.
Step 1: Define the Unit of Work
First, choose work that can be tracked the same way across Jira or another issue tracker, the repository, CI, and production releases. Link each accepted work item to its pull requests and deployments.
Then define what counts as complete. For an enterprise application, a merged pull request may be progress. Production release and acceptance are a different milestone.
Step 2: Establish a Baseline
Before expanding agent use, record the same metrics for a suitable baseline period. Capture delivery time, review effort, failure rate, and the mix of work.
For a credible comparison, note changes in headcount, release cadence, project complexity, tooling, and team duties. Those factors can change results without any link to agents.
Step 3: Tag Agent Involvement Without Assuming Perfect Attribution
Teams can identify changes made with GitHub Copilot, Claude Code, Cursor, or other agents. They can use supported tooling, PR metadata, or a consistent declaration process. A practical split separates human-written work, AI-assisted work, and agent-started changes.
These categories are imperfect. Developers often edit agent output, and tools differ in what they record. So teams should document the reporting method and its limits next to the metrics.
Step 4: Compare Similar Work
Next, segment results by task type, repository, risk, and rough complexity. A routine dependency update and a multi-system integration should not count the same in a simple productivity score.
Where possible, a staged rollout or comparable teams can strengthen the analysis. Even so, differences in team experience and codebase familiarity still limit causal claims.
Step 5: Review Both Leading and Lagging Indicators
In the first weeks, teams should look at changes in PR size, review backlog, first-pass test acceptance, and agent retries. Over a longer window, assess accepted work, releases, reopened issues, incidents, and cost.
A useful operating review asks whether one stage improved while another got worse. For instance, faster drafts with longer review queues call for a workflow fix. That fix should come before the organization raises agent throughput.
A Practical Scorecard for AI Coding Agents
The template below is for engineering leaders who evaluate agent-enabled delivery. It shows what to collect. It does not offer industry benchmarks or promise gains.
| Indicator | Consistent definition | Suggested evidence source | Interpretation |
|---|---|---|---|
| Accepted work completed | Work items accepted in a set period, segmented by complexity | Jira / Azure DevOps | Closest operational proxy for completed scope |
| Change lead time | Time from a consistent starting event to production | Repository + CI/CD | Tests end-to-end delivery speed |
| Review queue time | PR submission to first substantive review | GitHub / GitLab | Shows reviewer capacity limits |
| Active review time | First review to approval or merge | Repository reviews | Shows validation complexity |
| First-pass validation | Share passing agreed checks on the first run | CI + test systems | Shows readiness for review |
| Rework rate | Share of accepted changes needing fixes within 30 days | Issues + commit links | Shows downstream correction burden |
| Change failure rate | Share of production changes causing agreed failure events | Deployment + incident data | Connects delivery with stability |
| Cost per accepted change | Full attributable cost divided by accepted changes | Finance + engineering logs | Compares resource use with validated output |
Avoid Individual Targets
However, no single metric should become an individual performance target. If teams earn rewards for pull request counts alone, they can raise activity while lowering reviewability. A shared team scorecard suits organizational delivery better.
What Helps Teams Realize the Value of Coding Agents?
Recent evidence suggests that the practices around the agents matter as much as the choice of model. GitHub’s 2026 findings show wide adoption. The other studies show varied downstream results.
Three investments deserve attention:
- Manageable change sizes. Smaller pull requests cut review complexity and make failures easier to trace. DORA lists working in small batches as one of its seven capabilities.
- Clear engineering context. Repository conventions, dependency needs, acceptance criteria, and architecture limits reduce avoidable mismatches. Folder IT’s Spec Kit article covers how structured specifications support this.
- Reliable validation and ownership. Automated tests, security checks, named reviewers, and clear escalation paths help teams keep control as candidate changes pile up.
For a broader look at speed, code quality, and maintenance, Folder IT earlier examined AI in software development. This article covers the next question. How do those trade-offs show up in measurable delivery results?
Frequently Asked Questions
Do AI coding agents increase developer productivity?
AI coding agents can raise coding activity and cut implementation time on some tasks. GitHub’s 2026 Octoverse documented fast growth in agent-started pull requests. However, Harvard research and an internal Okta study show that downstream results differ across organizations. Completed work, review time, stability, and cost give a stronger view than code volume alone.
What is the best metric for AI coding agent productivity?
No single metric works for every team. Engineering teams should measure accepted work and end-to-end delivery time next to review effort, rework, production stability, and full cost. The most useful comparison covers similar work before and after adoption. It also notes other changes that affected the team.
Why can AI-generated code slow down code review?
AI-generated code can raise the volume of changes that wait for human validation. Some changes also need extra fixes, tests, or architecture review. In a November 2026 Harvard working paper covering 655 firms, review time from submission to approval rose 56.1% after agent adoption. The result describes the studied firms. It is not a universal prediction.
How should AI coding agent ROI be calculated?
AI coding agent ROI equals the verified financial benefit minus the extra AI investment, divided by that investment, times 100. A hypothetical team that validates $8,000 in avoided costs against $4,000 in added agent and adoption costs would calculate 100% ROI. However, saved engineering hours are not confirmed savings unless they deliver a measurable economic benefit. Teams should also track quality, accepted work, rework, and maintenance costs.
Which DORA metrics apply to AI-assisted engineering?
DORA-style delivery measures, such as lead time, deployment throughput, and stability, help evaluate AI-assisted engineering at the system level. Teams should read them next to agent-specific measures like review queue time, validation failures, and corrective work. Higher deployment activity means more when quality and reliability stay acceptable.
Should companies compare developers by AI-generated lines of code?
Counting AI-generated lines of code is a weak way to compare developers. Tasks differ in complexity, maintenance burden, and business value. Generated code can also add review and correction work for colleagues. Team-level, quality-adjusted delivery measures give a fairer view and discourage needless changes.
Measuring AI Coding Agent Productivity and ROI
AI coding agents are now part of a large share of software engineering work. Their value depends on how their output moves through review, testing, deployment, and maintenance. The research shows real gains in some settings. It also shows limits that can absorb part of those gains.
Engineering organizations can decide better by collecting the full delivery evidence and comparing work of similar scope. That shows whether an agent rollout adds delivery capacity, moves effort to reviewers, or creates costs that a first productivity dashboard misses.
Folder IT supports US engineering teams through nearshore software development and AI engineering services. Its teams work on enterprise applications, AI systems, integration, and production delivery. Organizations that want to scale AI-enabled engineering can also look at AI staff augmentation. Get in touch to discuss the delivery capabilities a specific roadmap needs.
Written by the Folder IT engineering team, a nearshore software development company working in-hours with US-based engineering and product teams.