How to Measure Developer Productivity (When the Code Is Written by AI)

Every engineering leader is being asked the same question right now. The AI bill keeps growing. Copilot, Cursor, Claude Code, all of it. And someone upstairs wants to know a simple thing: is it working?

Most teams cannot answer. Not because they are careless, but because the metrics they have were never built for this. This post walks through why measuring AI developer productivity is so hard, why the common metrics fail, and what actually tells you whether the AI spend is producing value.

Why measuring AI developer productivity is so hard

The old way of measuring engineering output assumed a human wrote the code. Lines of code, commits, story points, velocity. Once AI writes most of the code, these numbers stop meaning what they used to.

Here is the problem in one sentence. When an AI agent can produce the output of a whole team in a night, counting output tells you nothing about value. You can ship ten times more code and be worse off if half of it needs rework.

A CTO we spoke with put it plainly: lines of code do not tell you how much value you ship. He had tried story points too. They inflated the moment his team got good with AI, so the number went up while the real work stayed the same.

The metrics that break

Lines of code. AI generates volume by default. More lines is not more value. It is often more to review and more to maintain.

The productivity multiplier. Comparing before-AI to after-AI works exactly once. Once your team is fully on AI, there is no pre-AI baseline left to compare against. The multiplier becomes a number you measured one time and can never measure again.

Story points. They adapt as the team learns. A story point after AI adoption is not the same size as one before it. The scale quietly shifts, so the trend line lies.

Tool adoption and seat usage. Knowing who has a license or who opened the tool tells you about access, not impact. The unit of value is the task, not the license.

What actually shows whether AI is producing value

The leaders who have moved past gut feel stopped measuring activity and started measuring outcomes. A few things they watch:

• Bug-to-feature ratio. For every feature shipped, how many bugs surface afterward? This says something about quality, which is what output alone hides.
• Incidents per release. As AI accelerates delivery, does stability hold, or do incidents climb with the speed?
• Rework and reopen rate. When AI-assisted work ships, how often does it come back? Rework is the hidden cost that raw output numbers erase.
• Test coverage and fitness. AI writing code fast without strong testing is a risk, not a gain. Whether the team is set up to test what AI produces matters more as the pace increases.

The common thread is simple. Measure what is linked to customer value, not how many lines were generated.

The layer most tools miss

Here is the part most measurement tools cannot see. The metadata approach, reading commits, pull requests, and merge status, only sees the finished code. It misses what happened between the engineer and the AI to produce it.

Two engineers can ship the same feature. One prompts well, gets a clean result, and moves on. The other loops for an hour, burns tokens, feeds the agent context over and over, and gets there eventually. The output looks identical. The cost and the efficiency are not.

That difference lives at the session level, in how the work actually happened. If you only measure the output, you never see it. This is where the real signal on AI productivity and AI cost sits, and it is the layer Tommbo measures.

Where to start

You do not need a perfect system to start. You need to stop measuring the things that break and start measuring the things linked to value.

• Drop lines of code and the one-time multiplier as your headline numbers.
• Track bug-to-feature ratio and incidents per release, so quality stays visible as speed climbs.
• Watch rework and reopen rate to catch the hidden cost of fast but fragile work.
• Look at the session layer, not just the finished code, to see cost and efficiency you would otherwise miss.

The question your board is asking is fair. Is the AI spend paying off? The answer is knowable. It just is not in the metrics most teams are still using.

Tommbo measures whether AI is producing real value for engineering teams, at the layer where the work actually happens. If proving your AI spend is worth it is the question on your desk, that is the problem Tommbo is built for.