- 1. The paradox: more output, flat outcomes
- 2. Why AI-generated code fails quietly
- 3. What “quality” has to mean now
- 4. The roadmap: six phases to quality-first AI engineering
- 5. Metrics that hold up, and metrics that mislead
- 6. Quality and evaluation: two halves of one system
- 7. The people side: keep engineers able to judge
- 8. A 90-day starting plan
- 9. Conclusion
- Let’s talk
Almost every engineering team now uses AI to write code. Far fewer can show that their software got better because of it. This article looks at what the independent research says, explains why quality has become the variable that decides whether AI pays off, and gives a practical roadmap: the development patterns, gates and metrics that keep quality high, and the evaluation practice that proves it.
1. The paradox: more output, flat outcomes
The adoption question is settled. Google’s 2025 DORA report, based on a survey of nearly 5,000 technology professionals, found that about 90% of them use AI at work, a 14% increase over the previous year. Most say it makes them more productive.
The outcome question is not settled, and the most careful studies point in an uncomfortable direction.
- People can’t feel the difference. In METR’s randomized controlled trial, 16 experienced open-source developers worked on 246 real issues in repositories they knew well. With AI allowed, they took 19% longer. Before starting, they expected a 24% speedup. After finishing, they still believed AI had made them about 20% faster. The lesson that lasts is not the slowdown, which is specific to that setting and those early-2025 tools. It’s that self-reported productivity is not reliable evidence.
- Output rises, and so do its costs. Faros AI’s telemetry across more than 10,000 developers found that teams with high AI adoption completed 21% more tasks and merged 98% more pull requests. But pull requests grew 154% larger, review time rose 91%, bugs per developer rose 9%, and company-level delivery metrics stayed flat. Their 2026 follow-up reported the review bottleneck getting worse, with more pull requests merging with no review at all.
- Codebases age faster. GitClear’s analysis of hundreds of millions of changed lines shows copy-pasted code overtaking refactored (“moved”) code for the first time in 2024, and duplicated blocks rising sharply. Their 2026 study found a real output gain of roughly 25% when developers were compared with their own past performance, alongside an 81% rise in block duplication.
- The winners win on quality. McKinsey’s survey of nearly 300 public companies found that top performers improved productivity, customer experience and time to market by 16–30%, and software quality by 31–45%. The largest gains were in quality, not speed.
DORA summarizes all of this in one idea: AI is an amplifier. It magnifies the strengths of organizations with good engineering foundations and the weaknesses of those without them. AI adoption now correlates with higher delivery throughput, but it still correlates with lower delivery stability.
So the useful question isn’t “how do we generate more code?” It’s “what does AI amplify in our organization, and how do we make sure it’s quality?”
2. Why AI-generated code fails quietly
AI-written code rarely fails loudly. It compiles, it’s neatly formatted, and it reads like something a careful engineer wrote. The problems are underneath.
- More defects, and more serious ones. CodeRabbit analyzed 470 open-source pull requests and found about 1.7× more issues in AI-co-authored ones (10.83 per PR versus 6.45). Logic and correctness issues were 75% more common, error handling gaps were nearly twice as common, and readability issues more than tripled. Critical and major findings were 1.4–1.7× higher.
- Security doesn’t improve with model size. Veracode tested more than 100 models on 80 coding tasks designed around known weakness types. About 45% of the generated samples contained OWASP Top 10 vulnerabilities. Newer, larger models wrote more syntactically correct code, but their security pass rate stayed flat. Cross-site scripting defenses failed in 86% of relevant samples.
- Maintainability erodes slowly. Duplication, missing reuse and skipped refactoring don’t break the build. They make every future change slower and riskier, which is exactly the cost a sprint-level productivity metric can’t see.
Put these together and you get the core risk: AI makes it cheap to produce code that looks finished. A review process designed for human-paced, human-written changes waves it through. Problems then show up later as incidents, rework and security debt.
3. What “quality” has to mean now
Before improving quality, define it. For AI-assisted engineering I use six dimensions. Each one can be measured, and each maps to a known AI failure pattern.
| Dimension | Question it answers | Typical AI failure |
|---|---|---|
| Correctness | Does it do what the specification says? | Plausible logic that handles the happy path only |
| Security | Can it be abused? | Insecure defaults, missing input validation |
| Maintainability | Can the next person change it safely? | Duplication instead of reuse, local style violations |
| Operability | Can we run, observe and roll it back? | Missing logging, error handling and timeouts |
| Fitness for purpose | Does it serve the user and the business goal? | Solves the literal prompt, not the actual need |
| Understanding | Does the team know how and why it works? | Code nobody on the team can debug |
The last dimension is new, and it’s the easiest to ignore. In a randomized trial published by Anthropic in January 2026, developers learning a new Python library with AI assistance scored 17% lower on a comprehension quiz than those who coded by hand, with debugging skills showing the largest gap and no statistically significant time saved. How people used AI mattered: those who delegated everything understood least, and those who stayed engaged (asking questions, checking explanations) kept most of their understanding. Code that no one on the team understands is a quality problem, even when it passes every test today.
4. The roadmap: six phases to quality-first AI engineering
This roadmap combines what the research above shows with patterns I’ve seen work when evaluating AI systems in production. The phases build on each other, but you don’t need to finish one before starting the next.
Phase 0: Measure before you change anything
You can’t show improvement without a baseline, and you can’t trust developer surveys alone (see METR). Before you roll out tools or new practices, record at least a quarter of:
- The DORA delivery metrics: deployment frequency, lead time for changes, change failure rate, and time to restore service. DORA also tracks rework rate, the share of deployments that are unplanned fixes.
- Review health: pull request size, time to first review, and the share of pull requests merged without review.
- Codebase health: duplication, how much new code is changed again within two weeks, and open security findings.
Phase 1: Give AI the context, and give people a clear stance
DORA’s AI Capabilities Model identifies seven capabilities that amplify AI’s benefits. Several of them are about context and clarity: a clear, communicated AI stance; healthy data ecosystems; AI-accessible internal data; strong version control practices; working in small batches; a user-centric focus; and quality internal platforms.
In practice:
- Write repository instructions for the AI, versioned with the code: architecture, conventions, how to run tests, what not to touch. Many teams use files like
CLAUDE.mdorAGENTS.md. Without them, the model fills gaps with generic patterns, which is a direct cause of the “violates local patterns” issues CodeRabbit measured. - Publish an AI policy that says which tools are allowed, what data can be shared with them, and what is expected of a person who submits AI-assisted code. The short version: you own it as if you wrote it.
- Encode style as policy, not as review comments. CodeRabbit’s own recommendation is to enforce formatting, linting and style in CI so whole categories of AI-introduced issues never reach a reviewer.
Phase 2: Change how code gets written
This is where design and development patterns matter most. AI changes what is cheap. Writing code is now cheap. Specifying it, verifying it and understanding it are not. The patterns below move effort to where it now belongs.
| Pattern | What to do | What it protects against |
|---|---|---|
| Spec-first | Write a short specification (inputs, outputs, constraints, edge cases, non-goals) before asking the AI to build anything | Solving the literal prompt instead of the real need |
| Test-first (TDD with agents) | Have the tests written and reviewed first, commit them, then ask the agent to make them pass without editing them | Tests written to match the code instead of the requirement |
| Small batches | Keep each change small enough to review in one sitting | Oversized AI pull requests that stall in review |
| Explicit interfaces | Define contracts, types and module boundaries yourself; let AI fill in implementations | Architecture decided by accident |
| Reuse before generate | Tell the AI where existing utilities live and require it to search before writing new helpers | Duplication and drifting copies of the same logic |
| Refactor budget | Reserve a fixed share of each cycle for consolidation and cleanup | The refactoring decline GitClear measured |
| Secure by default | Use vetted libraries and templates for auth, input handling and secrets; never let the model choose | The security choices Veracode found models get wrong |
| Explain before accept | Require the author to explain, in the PR, how the change works and why this approach was chosen | Loss of understanding, as in the Anthropic skills study |
Anthropic’s own guidance for agentic coding recommends exactly this test-first loop: start from test specifications rather than a broad feature request, then grow scope one tested piece at a time. The reason is simple. When an agent writes the code and the tests together, the tests tend to check what the code does, not what it should do. Locked tests turn “done” into something that can be checked.
Phase 3: Automate the gates
Gates turn intentions into rules that hold under deadline pressure. A reasonable set for AI-assisted code:
- Tests must pass, and new code must include tests. Coverage is a floor, not a goal. If you can, add mutation testing on critical modules, since it checks whether tests would actually catch a bug.
- Static analysis and security scanning (SAST, dependency and secret scanning) on every pull request, blocking on high-severity findings.
- Duplication and complexity thresholds on changed files, so maintainability problems are caught at the point they’re introduced.
- Size limits on pull requests, with a required justification to exceed them.
- Labeling of AI-assisted changes, through PR labels or commit trailers, so you can later compare defect and incident rates for AI-assisted and human-written code. Without this, you can’t tell whether your AI practice is working.
Phase 4: Redesign review
Every independent dataset above points to the same bottleneck: review. Generation got faster; verification didn’t. Fixing it means changing what humans spend review time on:
- Automated first pass. Let tools catch style, known vulnerability patterns, missing tests and duplication before a human opens the file.
- Human review for judgment. Does this solve the right problem? Does it fit the architecture? What happens when it fails? Is the specification itself right?
- Risk-based depth. Changes to authentication, payments, data handling or safety-relevant logic always get deeper review, whoever or whatever wrote them.
- Watch the “merged without review” number. Faros’s 2026 data shows it rising with AI adoption. It should be close to zero for anything that reaches production.
Phase 5: Measure outcomes and evaluate continuously
The last phase closes the loop: measure whether quality and delivery actually improved, and run evaluations that catch regressions before users do. The next two sections cover both.
5. Metrics that hold up, and metrics that mislead
The research gives a clear lesson about measurement: the numbers that are easiest to collect are the ones most likely to mislead.
Metrics to rely on
| Layer | Metric | Why it’s trustworthy |
|---|---|---|
| Delivery outcome | Lead time, deployment frequency | Measures what reaches users, not what gets typed |
| Stability | Change failure rate, time to restore, rework rate | The metric DORA still finds moving in the wrong direction with AI adoption |
| Escaped quality | Production defects and incidents per change | Captures problems review missed |
| Security | New high-severity findings per change; time to fix | Tracks the risk Veracode measured |
| Maintainability | Duplication, short-term churn, refactoring share | Leading indicators of future slowdown |
| Review health | PR size, time to first review, % merged without review | Shows whether verification is keeping up with generation |
| Understanding | Incident debugging time; on-call escalations | Shows whether the team still understands its own system |
Metrics to treat with care
- Lines of code, pull request counts and commit counts. They go up automatically with AI and say nothing about value.
- Suggestion acceptance rate. It shows a suggestion was kept, not that it was good.
- Self-reported time saved. METR’s participants were wrong about the direction of the effect, not just its size. Use surveys to understand experience, not to calculate ROI.
Two rules for reading the numbers. First, give changes time. Practices take a while to settle, and an early dip is common, so judge trends over quarters, not weeks. Second, always read throughput and stability together. A throughput gain paired with a rising change failure rate is not a gain. It’s the pattern Faros calls whiplash.
6. Quality and evaluation: two halves of one system
Evaluation is where quality stops being a claim and becomes evidence. The relationship works in three directions, and teams that miss any one of them end up with either unproven quality or untrustworthy evals.
6.1 Quality needs evaluation
Quality you haven’t measured is a belief, and METR showed how far beliefs can drift from reality. This matters even more when the system itself is non-deterministic: an AI feature, an agent, or a coding agent working in your repository. Traditional tests check that a function returns the same answer every time. AI systems can give a different answer on each run, so “it worked when I tried it” proves very little.
There are two places this applies, and teams often think only about the first:
- Evaluating the AI that writes your code. Before standardizing on a coding agent, model or prompt setup, test it on tasks from your repository with your tests as graders. Public benchmarks don’t reflect your codebase, conventions or risk profile. A small internal suite (real issues with known good solutions) tells you far more.
- Evaluating the AI features you ship. If your product includes an LLM feature or an agent, quality means the behavior users experience, and only evaluation can measure that.
6.2 Evaluation ensures quality
Anthropic’s engineering guide to agent evals describes the building blocks clearly. A task is one test with defined inputs and success criteria. A trial is one attempt at it, and you run several because outputs vary. A grader scores the result. The transcript is the full record of what the agent did, and the outcome is the actual end state, for example whether the database record exists, not whether the agent said it created one.
Several practices from that guide and from Hamel Husain and Shreya Shankar’s work on evals turn this into a quality system:
- Start from real failures, not imagined ones. Husain and Shankar insist on error analysis first: read real traces, label what went wrong, group the failures, and only then write evals for the failure modes that matter. Writing a rubric before looking at data produces evals that measure the wrong things.
- Start small. Anthropic suggests 20–50 tasks drawn from real bugs and support tickets is enough to begin, because early improvements have large, visible effects.
- Separate capability evals from regression evals. Capability evals measure what the system can’t do well yet and start with low pass rates. Regression evals protect what already works and should stay near 100%. When a capability eval is consistently passing, move it into the regression suite.
- Measure consistency, not just possibility. pass@k is the chance that at least one of k attempts succeeds. pass^k is the chance that all k succeed. For anything user-facing, pass^k is the honest number, because users experience every run, not the best one.
- Wire evals into the gates. Run regression suites in CI on every prompt, model or code change, exactly like unit tests. A model upgrade that drops a regression eval should block the release just like a failing test would.
- Combine evals with production monitoring. Offline evals are the first line of defense and run on every change. Production monitoring shows real behavior and catches what evals miss. Periodic human review keeps both calibrated. Failures found in production become new eval tasks, so the suite keeps learning.
6.3 Evaluation needs quality
This is the direction most teams miss. An eval suite is software, and it has bugs. A grader that rejects valid answers, an ambiguous task, or a test harness with a constraint nobody noticed can make a good system look bad, or a bad one look good. Anthropic’s guide warns against taking scores at face value for this reason.
Apply the same quality discipline to evals that you apply to code:
- Unambiguous tasks. Two domain experts should reach the same verdict on each task. Include a reference solution to prove the task can be solved.
- Balanced cases. Test both when a behavior should happen and when it shouldn’t, or you’ll optimize the system toward one side.
- Calibrated judges. If you use an LLM as a grader, check its verdicts against human labels before trusting it, and recheck after model changes.
- Read the transcripts. Regularly review transcripts from failed and passed trials. That’s how you find grader bugs and agents that pass by exploiting a loophole.
- Retire saturated evals. When a suite sits at 100%, it still guards against regressions but no longer tells you anything new. Add harder tasks.
- Version everything. Datasets, graders, prompts and thresholds go in version control with the code they test, so every result can be reproduced.
6.4 Set the criteria before you test
One more principle ties quality and evaluation together: agree on the acceptance criteria before you see the results. Define what “good enough” means (the metrics, thresholds and test set) up front, with the people who own the outcome. Criteria chosen after seeing the numbers tend to be chosen to fit them. Starting from the business goal also keeps evaluation honest: every eval should trace back to something the system is supposed to achieve for its users, not just to a score that’s easy to compute.
7. The people side: keep engineers able to judge
Every practice above depends on engineers who can judge what AI produces. That ability is under pressure.
- Understanding can erode quietly. The Anthropic study showed lower comprehension, especially for debugging, when people delegated fully. It also showed the fix: people who stayed engaged kept their understanding.
- Early-career engineers carry the most risk. Stanford Digital Economy Lab researchers found a 13–16% relative decline in employment (depending on the paper’s revision) for 22- to 25-year-olds in the most AI-exposed occupations, with young software developers among the hardest hit. The authors note that other factors, including interest rates, explain part of the timing. Either way, if juniors only learn to prompt and accept, no one will be able to review AI output in five years.
Practical responses:
- Make juniors reviewers of AI output against the specification, not just its users. Reviewing well is how they learn to judge.
- Use the explain-before-accept rule from Phase 2 for everyone.
- Hold regular no-assist sessions, such as debugging exercises or design reviews done without AI, to keep core skills sharp.
- Treat skeptics as a source of test cases. Their objections often point to failure modes your evals don’t cover yet.
8. A 90-day starting plan
Days 1–30: Baseline and foundations
- Record DORA metrics, review health and codebase health for the current state.
- Publish the AI policy and add repository instruction files to your most active repositories.
- Turn on formatting, linting and security scanning as blocking CI checks.
- Start labeling AI-assisted pull requests.
Days 31–60: Patterns and gates
- Pilot spec-first and test-first development with one or two teams.
- Add duplication, complexity and PR size gates.
- Set up an automated first-pass review and track time to first review and merges without review.
- Build a first eval suite of 20–50 tasks from real bugs, both for your coding agent and for any AI feature you ship.
Days 61–90: Evidence and the feedback loop
- Compare AI-labeled and human-written changes on change failure rate, escaped defects and security findings.
- Move stable capability evals into a regression suite that runs on every change.
- Feed production incidents back into the eval suite and the repository instructions.
- Report results using outcome metrics, never self-reported time savings alone.
9. Conclusion
The research is consistent. AI reliably increases the amount of code a team produces. It doesn’t, on its own, increase the amount of working, secure, maintainable software a team delivers. The teams that get real value treat quality as the foundation and speed as the result: they give AI the right context, change their development patterns so specification and verification come first, automate the gates, redesign review, and measure outcomes instead of activity.
Evaluation holds all of this together. Quality needs evaluation to be proven, evaluation enforces quality through gates and regression suites, and evaluation needs its own quality discipline to be trusted. If you’re using AI to build software, the most important question is no longer how fast you can generate. It’s how quickly you can tell whether what you generated is right.
Let’s talk
How is your team measuring whether AI-assisted code is actually better? Which gates or evals have caught real problems for you? Tell me on LinkedIn or at mohcine@evaltrust.cloud.
Related reading on this site: A practical guide to agentic AI evaluation and Evaluation, guardrails and observability.
Sources
- Google Cloud DORA, 2025 State of AI-assisted Software Development, September 2025, and the DORA AI Capabilities Model (adoption, the amplifier finding, throughput and stability, the seven capabilities).
- Becker, Rush, Barnes and Rein (METR), Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity, July 2025 (paper).
- Faros AI, The AI Productivity Paradox Report 2025 and AI Engineering Report 2026: The Acceleration Whiplash.
- GitClear, AI Copilot Code Quality research (2025) and the 2026 analysis of 623 million changes, as reported by The New Stack.
- McKinsey & Company, Unlocking the value of AI in software development, November 2025.
- CodeRabbit, State of AI vs Human Code Generation Report, December 2025.
- Veracode, 2025 GenAI Code Security Report, July 2025 (updated since).
- Shen and Tamkin (Anthropic), How AI Impacts Skill Formation, January 2026 (paper).
- Brynjolfsson, Chandar and Chen, Canaries in the Coal Mine? Six Facts about the Recent Employment Effects of Artificial Intelligence, Stanford Digital Economy Lab, revised November 2025, and the lab’s February 2026 note on timing and interest rates.
- Anthropic, Demystifying evals for AI agents, January 2026, and Claude Code best practices.
- Hamel Husain, Your AI Product Needs Evals and AI Evals FAQ; Husain and Shankar, Building eval systems that improve your AI product.
Note: the roadmap, pattern table and 90-day plan are practical recommendations built on these sources and on my own evaluation work. Thresholds should be set with your own team and adjusted to your risk profile.