Productivity math

The productivity math (naive vs human oversight)

❓ If AI is “100× faster,” why does the honest number look more like 1.1–1.4×—and why is that still a big deal?

This section is the economic core. All averages below are from Patwardhan et al. appendix A.2 on the 220-task gold set, unless noted. Arithmetic was independently recomputed from the published formulas and Table 2; it matches.

The ingredients

Wages are BLS medians, while experts are seniors—so dollar savings are conservative underestimates of true senior-loaded cost. Time figures are validated self-reports, not stopwatch industrial engineering.

Scenario A — Naive “robot alone” ratio (the viral 100×)

For GPT-5 the paper reports about 90× faster and 474× cheaper on pure inference vs human completion cost. Across frontier models, OpenAI’s blog rounds the spirit of this to “roughly 100× faster and 100× cheaper.”

What this measures: silicon time and API invoice vs a human sitting down cold.
What it ignores: reading the output, fixing it, integrating it, liability, rework, and the cases where the model is worse than the human.

Naive 100× is a capability physics statement (“generation is cheap”), not an operating model statement (“your department ships 100× more”).

Scenario B — Try once, review, else do it yourself (Try 1×)

This is the first honest workplace proxy in the paper:

1. Ask the model.
2. Pay an expert to review (R_T).
3. If the model did not win (probability 1 − w), the human does the full task.

Expected time and cost:

Speed / cost improvement vs unaided human:

Reproduce GPT-5 (Table 2)

Using w = 0.39, H_T = 404, R_T = 109, M_T = 404/90:

Cost side likewise → about 1.18× cheaper (C₁ ≈ $307 vs $361).

Read the table left to right: the miracle shrinks when you pay for judgment—and for weak models it goes negative.

Scenario C — Try many times, then fix (Try n× → ∞)

If the expert may resample the model up to many times before giving up, expected time converges (for w > 0) to:

so speed → H_T·w / (M_T+R_T).

For GPT-5 that limit is about 1.39× faster and 1.63× cheaper—the paper’s Try n× column. Authors note this still over-penalizes models because real users raise w with better prompts and cut R_T as they learn the task.

Break-even: when does review-still-required AI beat pure human?

Under Try 1×, AI helps on time when:

With gold-set averages and GPT-5-like M_T:

Interpretation: once a model beats human deliverables more than ~28% of the time on your task mix (under this review cost structure), even a clumsy “draft then maybe redo” process is faster in expectation than always starting from a blank page. GPT-4o at 12.5% fails that test (0.87×). GPT-5 at 39% clears it. That single threshold is why the quality curve and the budget curve are the same story.

One average task in plain numbers (GPT-5, Try 1×)

Eleven percent does not sound like science fiction. It is still enormous at portfolio scale—and it is the pessimistic end of the paper’s framing because:

1. R_T = 109 minutes is first-time external grading, not a warmed-up internal reviewer.
2. Redo is modeled as full human restart, not “edit the AI draft for 20 minutes.”
3. w is frozen across retries.
4. Human-alone path is charged zero self-review/supervisor review, which real orgs often pay anyway.

Sensitivity: review time is the lever you control

Hold GPT-5 w = 39%, vary only review time:

Manager translation: every process that turns “re-do the whole grade from scratch” into “structured checklist review in 15–30 minutes” multiplies the value of the same model. Model vendors sell w and M_T; you own R_T and task routing.

Scale picture: 100 experts, one gold-average task per working day

Assumptions: 100 professionals, 220 working days, one 404-minute-class task each day, GPT-5 Try 1× economics.

That is not headcount you must fire. It is capacity you can redeploy to judgment-heavy work, client ambiguity, exception handling, and volume you previously could not accept. If your real R_T is 30 minutes and you only send high-fit task families (higher w), the freed band widens quickly toward the paper’s 1.4× time / 1.6× cost zone—or beyond if edits replace full redos.

Concrete scene: A senior associate used to bill a full day on a recurring package. Now the model drafts in four minutes. She spends half an hour on a red-team checklist. Four times out of ten she ships with light edits; six times out of ten she rebuilds. On average she still gets the afternoon back often enough that the practice takes more matters without hiring. That is GDPval economics in a hallway, not a press release.

Why “we still need humans to review” is not a refutation

The skeptical sentence is: If people must check the AI, why bother?

The math answers:

1. Review is cheaper than creation on these tasks (109 min vs 404 min even in the harsh paper setup).
2. Creation is skipped whenever the model wins (probability w).
3. API cost is rounding error next to expert wages ($0.76 vs $361).
4. Below ~28% win rate, the skeptic is right for Try 1×. Above it, the skeptic is leaving money on the table.
5. The honest win is not 100× headcount deletion; it is double-digit capacity and cost improvement with quality gates still on—exactly what regulated industries can actually deploy.

What the math still misses (do not hide)

• Catastrophic errors with nonlinear cost (bad medical plan, bad legal advice).
• Integration, compliance logging, security review beyond content quality.
• Multi-turn client clarification time (may add work models currently skip).
• Benefits when humans also need review (two-sided QA).
• Gains when the human edits instead of fully redoing (paper understates upside).

For decision use: treat Table 2 as a lower-bound constructive proof that oversight-compatible AI can already beat unaided experts on average gold tasks for strong models—not as a universal ROI calculator for every workflow.