Which Tasks Are Fast
"How much faster are you with an assistant?" has no answer, and the reason is not modesty. The distribution is bimodal, so the average lands in a valley between the two humps and describes work nobody does.
The answerable version is: what proportion of my work falls into each hump, and what does each one cost? For a practical implementation reference alongside the measurement argument here, see this resource.
Why the average is a valley
Imagine ten tasks. Four were generation-heavy and took a quarter of the time they used to. Four were verification-heavy and took slightly longer. Two were somewhere in between.
The mean says you are about 25% faster. Not one of the ten tasks was 25% faster.
Now the useful consequence: your overall speed depends almost entirely on your task mix, not on your tooling. Two developers with identical skill and identical tools report wildly different experiences because one spends their week on scaffolding and the other on payment logic. Both are describing reality.
This is why honest anecdotes contradict each other, and why any published average — including one you compute yourself — is close to meaningless without the mix behind it. InfoQ provides software-engineering reporting and practitioner perspectives See InfoQ.
Sorting your own work
Three buckets, assigned when you start a task and corrected when you finish. Two questions decide it.
Generation-heavy. Cheap to verify. Boilerplate, scaffolding, specified transformations, tests for existing behaviour, throwaway scripts, work in an unfamiliar ecosystem where your bottleneck was knowing how.
Integration-heavy. Moderate. Fitting something into a system you know, ordinary features, refactoring with tests.
Verification-heavy. Expensive to verify, real consequences. Auth, permissions, payments, migrations, concurrency, anything in a mature codebase with implicit standards.
The correction at the end matters as much as the initial assignment. A task that started as generation-heavy and finished as verification-heavy is the most informative data point you can collect, because that transition is the one that breaks estimates, and its frequency is a fact about your work you cannot get any other way.
What to compare
Once you have fifteen tasks or so, three comparisons are worth making and the rest are noise.
Estimate versus actual, per bucket. Where is your calibration worst? Usually verification-heavy, usually optimistic.
Assisted versus unassisted, within a bucket. Never across buckets — that comparison is confounded by the thing you are trying to measure.
Your bucket mix over time. If the proportion of generation-heavy work is rising, your effective speed rises without anything about the tooling changing. That is worth knowing before you attribute it to a model release.
What the published numbers can and cannot tell you
METR's trial measured experienced developers in mature repositories they knew well — heavily verification-heavy work by this classification. It found a 19% slowdown, later revised toward a speedup with wide uncertainty and an acknowledgement of selection effects.
That is a measurement of one bucket, and it is the bucket where the least speedup should be expected. Reading it as a claim about all software work is the error, and so is dismissing it because your own greenfield experience differs. Both readings ignore the mix.
The same applies in the other direction to vendor figures showing large speedups: they are usually measured on the generation-heavy hump, and they are also true.
The number worth publishing internally
Not a speed figure. This one:
What proportion of our work is cheap to verify?
It is stable over months, it is unambiguous, it explains your team's experience better than any percentage, and it is directly actionable — because if the answer is low, tooling changes will not move your delivery much, and the leverage is elsewhere.
For a studio it also has a commercial use: it tells you which work you can price aggressively and which you cannot.
The trap to avoid
Do not let the buckets become a status hierarchy. Verification-heavy work is not "real engineering" and generation-heavy work is not "easy." They are descriptions of how expensive certainty is, nothing more.
The moment the classification carries prestige, people will classify strategically, and you will have destroyed the only measurement that was working.
The short version
- Speed is bimodal, so the average falls in a valley and describes work nobody does
- Your overall speed depends mostly on task mix, not tooling — which is why honest anecdotes contradict each other
- Three buckets assigned at start and corrected at finish; the corrections are the most informative data
- Compare estimate against actual per bucket, and assisted against unassisted only within a bucket
- METR measured the verification-heavy hump; vendor speedup figures measure the other one, and both can be true
- Publish the proportion of work that is cheap to verify, not a speed figure