跪拜 Guibai
← Back to the summary

The Hidden Costs of Measuring Developer Productivity by the Numbers

This article discusses a management approach based on quantitative metrics for evaluation. The viewpoints come from multiple rounds of debate between me and an AI, and were ultimately organized into an article by the AI.

1. What Quantification Does to the Evaluated

Evaluations often face a pile of ambiguous matters. Who is working hard, who is slacking off, which architectural refactoring is truly necessary—without a set of hard standards, it all comes down to who has a good relationship with the supervisor, who is more eloquent, or what mood the supervisor is in today.

One management solution is to rely purely on numbers and metrics, with rules laid out on the table, quantifying all work as much as possible into a task list with scores. Difficult, time-consuming tasks get high scores; introducing bugs leads to point deductions.

1.1 The Most Obvious Benefits and Drawbacks

You can clearly count how many requirements a person fulfilled this month, how many bugs they fixed, and how many points each ticket was worth. These numbers on Jira match the code commit records, and no one can deny them. But does adding up these numbers truly represent this person's entire contribution?

It does not. They are only that small slice the scoring system happens to be able to count. The rest—

All of this is zeroed out in the system. Not a low score, but no score at all.

Is quantification completely useless? Not at all. Among those uncountable things, there is also a large amount of muddled freeloading. People who talk grand architecture all day but can't actually write a line of code; directions pursued under the banner of "technical exploration" for half a year with nothing deliverable—a quantification system, while mistakenly hurting those who silently defuse landmines, also cuts down the slackers with one clean stroke. In an organization without a quantitative yardstick, this type of slacker can survive for a long time because their (lack of) value is equally "perceptible but unprovable."

The contradiction of quantification lies right here: It indiscriminately eliminates both hidden good contributions and hidden scams, discarding signal and noise together. The small portion that remains is not the most perfect, but it is at least a baseline that everyone can mutually confirm.

Quantification is not injustice. It replaces the random, untraceable bias of relying on your private intuition with a systematic, transparent, rule-explicit bias. The victim of the former at least knows what harmed them, which can be discussed and improved. The victim of the latter cannot even identify what harmed them.

1.2 When Metrics Are Tied to Rewards and Punishments, Optimization Behavior Begins

There is a law in economics called Goodhart's Law:

"When a measure becomes a target, it ceases to be a good measure."

This happens especially fast in R&D teams, because programmers are the people in a company most skilled at analyzing system rules and finding optimal paths. The whole process happens in three steps:

Stage State Phenomenon
One Metrics are still relatively reliable High scorers indeed do more, with solid output
Two Everyone figures out the rules and optimizes directionally Delivery rates rise, per-capita output soars, but business value does not grow proportionally
Three A pure score-grinding game No one remembers what questions these metrics were originally meant to answer

The second stage is the hardest to identify. Because all the numbers are improving—the delivery curve looks beautiful, the bug escape rate is decreasing. This is not an accident; this is growth. It is very difficult to make anyone stop and think when looking at a rising curve: What exactly is it that we are rising?

1.3 Scores Have Their Own Gravity

People always think score manipulation is intentional, but often, no one needs to act maliciously. The evaluation form itself carries a bias.

Everyone calculates cost-effectiveness when working:

Is there even a choice? This is not a moral issue; the rules are pushing everyone toward low-risk, quick-scoring tasks. Those truly tough nuts that have long-term benefits for the architecture are systematically left untouched. And the supervisor cannot see any problem in the reports—all they see is everyone enthusiastically picking up tickets and scoring points. Every ticket got the points it deserved.

Even worse, the update cycle of the scoring table always lags behind strategy. The company says this year it wants to shift toward componentization and platformization, but on the scoring table, these long-term refactoring tasks either have low point values or no corresponding point values at all—they can only be shoehorned into historical old standards. This gravity field runs quietly for several quarters; high-value refactoring remains untouched, and the supervisor wonders why everyone suddenly lacks self-drive.

2. The Maintenance Cost of a Quantification System

Chapter 1 discussed how the evaluated react once a quantification system is up and running. This chapter switches perspectives: the cost of maintaining the system itself.

2.1 The System Expands

Every time a gaming loophole is discovered, a new check-and-balance dimension is added. After a few rounds, the metric library itself becomes a complex system. Plug a loophole today, add a metric; discover a new trick tomorrow, add another. Eventually, the cost of maintaining it may exceed the management bandwidth it saves.

2.2 AI Expands the Boundaries of Quantifiability, but the Ruler Itself Is Aging

Some might think that AI will drastically reduce maintenance costs.

Traditional quantification can only capture output. AI advances the quantifiable object from "results" to "process"—AI code review can evaluate the logical coherence and architectural consistency of a PR. The past dilemma of "raising a preventive question that cannot be proven" is technically being approached; although it can never be fully covered, the boundary is moving.

The positive side: Some previously zeroed-out implicit contributions can now be partially captured.

The side requiring vigilance: The increasing precision of quantification tools creates an illusion that "everything can be quantified." An AI score is not objective truth—it reflects the designer's preferences and the biases of the training data just like traditional metrics, and because the model itself is a black box, the bias is even harder to audit than explicit rules.

AI quantification tools also face a triple degradation:

  1. Implicit drift. Prompt templates become outdated, and evaluation models fall out of sync with the latest architectural practices. Lint rule errors will scream at you, and you will fix them; the deviation of an AI scoring model can silently drift for months before anyone notices.
  2. Output optimization. When AI scores are tied to performance, an engineer's goal slides from "writing good code" to "writing code the AI will give a high score to." Concise, unconventional code based on deep domain knowledge is systematically undervalued, while verbose, boilerplate code consistently gets high scores.
  3. Honest double loss. No one is acting maliciously—the engineer is honestly optimizing for what the AI rewards, and the AI is honestly scoring according to its training distribution. Two honest behaviors superimposed result in global absurdity: the codebase has never been healthier on the metrics, and never been more mediocre in maintainability.

AI expands the territory of quantification but does not change the underlying trade-off of quantification: any quantification system is always an instantaneous snapshot of the designer's current judgment, and the world will continue to move. Precision does not equal truth. More precise incompleteness is still incompleteness.

2.3 Making Scores Chase the Business Is a Never-Ending Task

Assigning points to each type of task is essentially a manager's judgment on "how much this type of work is worth at the current stage." But this judgment needs to be updated as the business phase, tech stack, and team capabilities change—

This is a task that becomes more exhausting the more you do it, and it will never be "done." Scores are always chasing the business from behind. You finally correct the weighting of last year's technical debt, and this year's AI-native development brings new imbalances. You fix one historical omission, and a new omission has already been generated elsewhere.

3. How Quantification Backfires on Those Who Use It

The previous two chapters discussed the quantification system itself—how it affects the evaluated and what it costs to maintain. This chapter switches to a third angle: what happens to the managers who use the quantification tool.

3.1 Liberation or Replacement

When a team grows beyond a certain size, monoliths become microservices, and hundreds of APIs run daily. Without tools to monitor progress and quality, no one's brain is sufficient.

Tools do two fundamentally different things simultaneously:

Liberation Replacement
What you are doing Consciously offloading low-value judgments to the tool Unconsciously handing over the act of judgment itself
Effect Saving mental energy to ponder architectural direction, talk to key members, identify structural risks You no longer get close to the signals that can only be captured by being in the thick of things
Consequence Judgment does not atrophy Judgment atrophies

The dividing line is not in the tool itself. It lies in whether you have asked yourself: Does this judgment still need to be made by me? When you stop asking—replacement has already occurred.

For a supervisor with judgment, the tool makes them stronger—after offloading low-dimensional repetitive judgments, the freed bandwidth flows to higher-dimensional matters. For a supervisor without judgment, the tool provides a perfect hiding place—looking at dashboards, scoring by metrics, making decisions based on rankings; every decision is traceable and defensible. Superiors see all-green indicators, and no one can question them.

Tools elevate those with judgment and allow those without judgment to never have to develop it. Under the same tool system, two types of people move in completely opposite directions. But the organization's dilemma is this: when the tool system is mature enough, these two types of managers may appear completely identical on the dashboard—the latter may even be more stable, because they don't take risks, don't challenge rules, and don't make long-term investments. In a system that only rewards a good-looking dashboard, the person without judgment actually gets promoted faster.

3.2 Metrics Reverse-Shape the People Who Use Them

There is an even more subtle effect. Metrics are not only gamed by the evaluated; they also reverse-shape the managers who use them.

A person who long-term only looks at a few specific metrics to make decisions will have their cognitive framework slowly align with the metrics—the directions they care about converge to what the metrics can measure, and the questions they ask are limited to what the metrics can answer. They will feel that things outside the metrics are unimportant, not because they truly are unimportant, but because they can no longer perceive them.

The dashboard uses what you thought was important in the past to continuously train the present you to continue thinking these things are important. The more you rely on it, the less you question it; the less you question it, the more you rely on it.

Dashboards need to be interrupted, but not through occasional random deep-dives—a manager randomly slicing a micro-view, reading a piece of code, attending a review, will most likely get meaningless fragments, instead creating the dangerous illusion that "I still understand the front lines."

Calibration is not personally sampling; it is structurally testing whether the dashboard is still measuring what it claims to measure. For example, pick a metric like "requirement delivery speed" and trace whether its recent rise is because high-value requirements are genuinely accelerating, or because requirements are being broken into smaller pieces. This is not feeling the front lines; this is targeted flaw detection.

4. Where Quantification Cannot Reach

None of the above means quantification is bad. Pure personal judgment in an organization of any size is a disaster—unfair, unstable, and unscalable. Quantification provides transparency, consistency, and scalability, which are the foundations upon which management can exist.

The question has never been whether to quantify. It is what you have quantified, and what you do with the quantification results.

But the previous three chapters have made it clear: a quantification system ages, expands, and fails at its boundaries. The following two things cannot be solved internally within the quantification system and must be handled outside the system.

4.1 Leave an Escape Hatch

When a quantification system runs for too long, metrics become outdated, and gaming paths solidify. At a certain node, any internal patching will only cause complexity to continue climbing.

At this point, someone needs to stand up and say: In this specific scenario, this set of metrics no longer applies—and then bear the consequences.

This is not a retreat to private opinion. It differs from deciding based on personal preference: the latter has no rules, no records, and no way to trace back. Judgment with a quantitative chassis presupposes that quantification has already covered the vast majority of daily scenarios, and judgment only acts on the boundaries it cannot reach.

To prevent this escape hatch from becoming a new privilege, three constraints are needed:

Three constraints ensure the escape hatch does not degenerate into a new round of private opinion, nor expand into another layer of metrics needing management.

4.2 Preserve Some Things That Cannot Currently Be Valued

An organization's long-term survival does not depend solely on efficiency. Breakthroughs that truly change the landscape almost never fit the definition of a "good investment" when they occur—returns are unpredictable, probabilities incalculable. A quantification system naturally only recognizes output that is predictable, verifiable, and consistent with historical patterns. When this strategy runs long enough, it will automatically drain away things that produce no current value but carry future possibilities.

"Unquantifiable" does not in itself confer value on anything—Chapter 1 has already argued that what the scoring table misses includes both genuine substance and water. So the premise for preservation is not "admitting it is unquantifiable," but setting verification nodes for it:

A direction for which you cannot even describe what failure looks like is not exploration; it is self-deception.

This does not require overturning the scoring system. Carve out an independent channel outside the scoring system, with a hard upper limit on quantity; each direction entered carries an expiration date and verifiable stop conditions; the judgment criterion switches from "how much value is produced" to "how many false hypotheses have been eliminated"—the former can only be assessed retrospectively, the latter can be tracked in process.

The same logic applies to all management behaviors. Any management tool, no matter how finely designed, faces the same fork: Is it helping you see reality more clearly, or is it helping you avoid the discomfort of directly facing reality. The hallmark of the former is that the manager clearly knows what the ruler measures, what it misses, and when it should be thrown away. The hallmark of the latter is that the manager has stopped asking.