The Estimation Paradox

[🇩🇪 - Das Schätz-Paradoxon]

🃏 Forty Seconds to Consensus: Why Planning Poker Fails After the Reveal

Sprint planning, Tuesday afternoon, FluxIon Grid Sachsenhausen, nine-person engineering team. Chris, a junior in his second sprint, looks at the story Settlement reconciliation: account for the new tenant structure in the report distribution. He does not know whether this is a three or an eight. He does not know the module. He was not on the last refactor. He is unsure whether tenant structure here means the same thing it means in the authentication module.

The Planning Poker tool counts down. Chris glances left at Sarah, the lead architect. Sarah has raised a five. Chris raises a five. Anika, the second-line developer on the right-hand screen, has raised an eight. She sees the eight fives, says half-aloud “okay, I was probably too cautious, let’s go with five”, and changes her card. Forty seconds. Story estimated. Next story.

In those forty seconds no estimation happened. A conformity ritual happened that looks like estimation. Chris checked what the senior had raised. Anika revised her own dissenting number because eight cards appearing together carry social weight. The number that lands in the tool, a five, gets summed into sprint velocity, extrapolated into the quarterly forecast, and aggregated onto the roadmap. It looks like a measurement.

It is the loudness of the highest-status voice in the room, rounded to the nearest Fibonacci number.

📐 The Cohn Construction: What Story Points Were Built to Measure

Mike Cohn introduced story points in 2005 as a measure of relative complexity. Not hours. Not person-days. An ordinal scale that ranks comparable stories against each other. The logic was defensive. Hour and day estimates are systematically biased toward optimism, as Magne Jørgensen documented across two decades of empirical software-estimation studies. An ordinal measure without a time anchor was meant to bypass the bias while still letting the team forecast.

The mathematics works under specific conditions. The estimates must be relative, anchored to a stable reference story that the room knows. And the estimates must be independent, which means nine people looking at the story in parallel and choosing a number unaffected by each other.

Both conditions fail in most scaled corporate planning sessions. The reference story rotates or is not maintained at all. Independence collapses the second the senior architect’s card becomes visible.

The unit has value. The conditions under which the unit produces a valid measurement are absent.

🧪 The Asch Inheritance: Why Group Estimation Defaults to a Social Reflex

Solomon Asch published the line experiment in 1955. Subjects sat in groups of eight, seven of whom were confederates of the experimenter, and were asked which of three lines matched a reference line in length. The task is trivial. Performed alone, error rates sit below one per cent. When the seven confederates unanimously named the wrong line, three quarters of the genuine subjects gave the wrong answer at least once across the trial sequence. Roughly a third conformed in the majority of trials. The correct line was visually obvious. The subjects could solve the task in isolation without difficulty. They folded anyway.

Asch’s finding is brutal in its clarity. Conformity pressure overrides direct perception even when the right answer is in front of your eyes and the cost of dissent is zero. A graduate student you will not see again may disagree with you about the length of a line on a card. Most people conform.

Story points are not lines. They are an ordinal estimate over an inherently uncertain future. The correct answer is structurally not independently verifiable in the moment. The conformity conditions in sprint planning are systematically worse than in Asch’s lab. The group is not anonymous. The team members know each other, and they will code together tomorrow. The highest-status voice physically places a number on the screen, with their name attached.

There is time pressure, because planning must finish in two hours. There is a visible cost function for dissent: anyone who deviates has to justify the deviation in front of the room, which costs discussion time the whole team pays for. There is little reward for being the lone correct estimator, because the estimate is rarely independently checked against reality in a way that ties back to the original card.

The most rational individual strategy under those conditions is conformity.

The mechanism layered on top is anchoring, formally documented by Tversky and Kahneman in 1974 and replicated in software estimation by Jørgensen. Estimates that follow an arbitrary first number converge on that number, even when the anchor is acknowledged as random. The first visible card in the room acts as the anchor against which the subsequent cards are unconsciously calibrated. Planning Poker with simultaneous reveal attempts to prevent this. In practice, the team looks at the senior after the reveal and adjusts on the second pass, the third pass, or the discussion round.

🪞 The Suspiciously Stable Curve: When Velocity Stops Behaving Like a Measurement

An honest estimate over inherently variant work shows sprint-to-sprint dispersion of plus or minus thirty to forty per cent. The work is variable. Stories are different sizes. People fall sick. Dependencies block. The system is noisy.

In many corporate teams, velocity moves in bands of plus or minus ten per cent across twelve, sixteen, twenty sprints. That is not delivery stability. That is stability of self-description. The team has learned that velocity appears in next month’s board slide, and estimates so the number stays roughly constant. A story that fits the current sprint’s remaining velocity becomes a five. The same story dropped into an empty sprint would be an eight. Once the measure enters the steering loop, the estimate becomes a function of the report.

The second symptom is quieter and more expensive. Chris raises the five because Sarah raised the five. Nobody learns that Chris does not understand the story. That information, the most precise leading indicator of the story’s delivery uncertainty, falls out of the process before it is spoken. When the story takes three times longer six sprints later, it gets attributed to “technical complexity” in the retrospective. The underlying cause was a junior who sat through estimation in silence because the conformity cost of saying I don’t know what tenant structure means here outweighed the throughput cost of guessing.

Both symptoms share the same root. The estimate is a social artefact. The chart drawn from it is the chart of the social artefact, not the chart of the work.

🧮 Goodhart’s Law: How a Team-Internal Aid Becomes a Boardroom Currency

Cohn’s velocity was a private dialect, owned by the people who wrote the code. The moment velocity is summed across the programme, presented to a steering committee, and tied to a bonus pool or a quarterly target, Charles Goodhart’s 1975 observation takes over. Any observed statistical regularity tends to collapse once pressure is placed on it for control purposes. The teams under that pressure do not invent new ways to lie. They reach for the few moves the unit allows: the unit re-priced upward, the work re-counted as more tickets, the slip absorbed into the inflation of the next sprint’s estimate. The chart climbs. The shipped value remains flat.

This article does not re-litigate that mechanism, which the velocity-fetish chapter handles in its own right. The point here is one step earlier. The Goodhart corruption begins at the moment of the estimate, not at the moment of aggregation. By the time the number arrives in the boardroom, it has undergone two filters: once through the conformity pressure of the planning session, once through the team’s quiet calibration to the expected report. The aggregation adds a third filter on top.

This explains why coaching teams on “better Planning Poker” produces no outcome. The unit was honest in the room before it became corporate currency. The corruption does not sit in the technique. The corruption sits in the boundary the number crosses on its way out of the team.

🔧 The Throughput Pivot: Replacing the Estimate With the Ticket Count

The structural alternative is older than the dysfunction it cures. Dan Vacanti laid it out in 2015: stop estimating each story, count how many tickets the team finishes per week, and forecast against that distribution.

The mechanics are unromantic. The team no longer plays Planning Poker. Stories that look enormous get split until they look comparable to the team’s recent work. Each item that lands in the team’s queue counts as one. The team records the count of items finished each week. After eight to twelve weeks, the throughput distribution is dense enough to drive a Monte Carlo simulation. Given this team’s empirical week-to-week variation, what is the probability of finishing the next seventeen items by week four? By week six? By week eight?

The forecast emerges as a probability band, not a single number. The board sees “85% likelihood of finishing by week six, 50% likelihood by week four”, and the team has not estimated a single ticket along the way. The number driving the forecast is recorded after the fact. It cannot be conformity-biased because it is observation, not negotiation.

What a coach can do:

• Replace the Planning Poker step in the next sprint with a thirty-minute right-sizing conversation. The question is not how big is this? but is this comparable to the kind of work we have been finishing in a week? Items that fail the test get split. Items that resist splitting get marked oversized and quarantined.
• Record weekly throughput from the existing board. The data is already in the system.
• After eight weeks, run a Monte Carlo against the throughput sample. Replace the next quarterly forecast slide with the probability band.
• Stop reporting story-point velocity to anyone outside the team. Watch the team’s estimation behaviour shift when the audience disappears.

The Monte Carlo step sounds technical. A spreadsheet template, three columns wide, runs ten thousand trials in under a second.

The disciplinary hurdle is whether the steering committee can read a probability band. That is a translation problem, not an analytical one. “Seventeen items have an 85% chance of being done by week six, and a 50% chance by week four” is a CFO-grade statement. It is closer to an option price than to a budget line. It happens to be accurate.

🛑 The Structural Amplifier: Why the Senior’s Card Is Not the Core Problem

The behavioural-bias account, anchoring plus conformity plus a junior who folds, is a true description of the symptom. It is not a complete description of the cause.

The cause is structural. In a standard Planning Poker session at FluxIon, distinct organisational asymmetries operate in the room before anyone raises a card.

Power asymmetry: the senior architect’s card carries weight because the architect’s annual review is closer to the manager’s ear than the junior’s.

Information asymmetry: the senior has the system context the junior is missing, and the format does not separate “I have context” from “I have a number”; both arrive as a single Fibonacci card.

Career-incentive asymmetry: the cost of being wrong about an estimate falls on whoever delivers the ticket, which is rarely the person who anchored the estimate. The senior pays no penalty for the five. The junior pays the cost of pretending to understand it.

A coach who asks the senior to raise their card last so the juniors avoid anchoring is treating the symptom. The senior’s card is loud because the room is structured to amplify it. Asking the loudest voice to whisper does not change the acoustics. It ensures that the next-loudest voice gets the same effect.

The throughput pivot defuses the structural layers at once by removing the moment in which a number must be produced under social observation. The estimate disappears. The forecast remains. The senior’s status is no longer denominated in a number that the team has to ratify in real time.

⚖️ The Boundary Condition: Where Story Points Still Earn Their Keep

The case against story points is not absolute. They still function as a team-internal planning aid under three specific conditions. The team must be small enough that the highest-status voice shifts depending on the domain. The reference story is actively maintained, not silently rotated. And the resulting number never leaves the room: it is never aggregated, summed, or compared across the department. Under those parameters, Cohn’s original construction holds. The unit stays meaningful, and the friction cost of throughput sampling is not yet worth paying.

The strongest counter-argument deserves equal honesty. Throughput sampling has its own gaming pattern. If a team knows its forecast relies on a weekly count, they can split tickets aggressively to inflate the data, or absorb scope into completed items to keep the line steady. The structural protection here is identical to the one that fails for story points. Keep the metric out of the reward path. Use the probability band to forecast, not to judge. The moment a steering committee sets a target throughput and ties it to a bonus, you invent a new dialect of velocity inflation.

Every estimation technique degrades when weaponised. The choice is between a metric that breaks visibly under pressure and one that breaks invisibly. Throughput sampling fails visibly. Story-point velocity fails invisibly. That is the structural reason to switch.

🏁 The Bottom Line: Stop Asking the Room to Vote on the Future

Planning Poker fails because it is a vote, and a vote is the wrong instrument for measuring uncertain work. Votes find consensus. Estimates require information.

Whether your team is small or your enterprise is scaled, the mechanical fix remains the same. Count the finished items. Forecast from the distribution. If a department insists on keeping their cards because the ritual provides psychological comfort, let them. But accept that the resulting chart measures the room’s social hierarchy, not the work’s complexity. Read it accordingly. Do not paste it onto a boardroom slide.

The number on the wall is not a measurement of future delivery. It is a recorded echo of the highest-status voice, dressed up in a Fibonacci sequence and stamped with a corporate seal.

⏱️ TL;DR: The Forty-Second Recap

• Planning Poker is a conformity ritual: Asch measured what happens when confederates name the wrong line; three quarters of subjects fold at least once. Sprint planning conformity conditions are systematically worse than Asch’s lab.
• The unit was honest before it crossed the team boundary: Cohn’s story points were a private team-internal aid. The moment they get summed across teams and reported to a steering committee, Goodhart’s Law consumes the measure.
• Suspiciously stable velocity is the diagnostic signature: A team that delivers variable work but reports velocity inside tight bands for sixteen sprints is calibrating to the report, not to the work.
• The structural alternative is throughput sampling: Stop estimating each ticket. Count what the team finishes per week. Run a Monte Carlo against the distribution. The forecast comes out as a probability band, not a single number.
• Throughput sampling fails if weaponised: Estimation techniques degrade when tied to a bonus. The protection lies in keeping the metric out of the reward path, not in the metric itself.
• The senior’s card is not the real problem: Power, information, and career-incentive asymmetry produce the conformity. Asking the loudest voice to whisper does not change the room’s acoustics.

🧾 The Receipts: The Psychology and the Mechanics

• Conformity Under Group Pressure: Asch, S. E. (1955). “Opinions and Social Pressure.” Scientific American. DOI, PDF. The line experiment is the foundational demonstration that visual perception folds under unanimous group dissent.
• Anchoring as a Measurable Bias in Estimation: Jørgensen, M. (2004). “A review of studies on expert estimation of software development effort.” Journal of Systems and Software. DOI, PDF. Two decades of empirical studies confirm that the first visible number reshapes every estimate that follows.
• The General Anchoring Effect: Tversky, A., & Kahneman, D. (1974). “Judgment under Uncertainty: Heuristics and Biases.” Science. DOI. The original demonstration that arbitrary first numbers contaminate subsequent estimates across domains.
• The Original Construction of Velocity: Cohn, M. (2005). Agile Estimating and Planning (ISBN: 978-0131479418). The explicit definition of story points as relative, ordinal, time-decoupled, and team-internal.
• Throughput-Based Forecasting (Monte Carlo): Vacanti, D. S. (2015). Actionable Agile Metrics for Predictability (ISBN: 978-0986436338). End-to-end throughput sampling with worked Monte Carlo examples; the single most important practitioner reference for replacing single-bar estimates.
• Velocity as a Steering Metric: Reinertsen, D. G. (2009). The Principles of Product Development Flow (ISBN: 978-1935401001). The standing reference for why metrics that enter the steering loop stop measuring the system and start measuring themselves.
• Goodhart’s Law (The Metric Trap): Goodhart, C. A. E. (1975). “Problems of Monetary Management: The UK Experience.” Monetary Theory and Practice (Macmillan, 1984, ch. 4). When a measure becomes a target, it ceases to be a good measure. Read the chapter here