METR 曲线没画错，是单位骗了你（中英切换）

Someone in the group chat posts that chart again: release date on the x-axis, "how long a software task an AI can finish on its own" on the y-axis, dots climbing to the upper right. The caption: "GPT-4 could handle 4-minute jobs, now we're at hours, doubling every few months, at this rate..." And a row of people start counting on their fingers: which year can AI do a week of work, which year a month.

That's METR's time-horizon plot, probably the most forwarded chart in AI over the past year. The New York Times compared it to Moore's law, others to the Keeling Curve of rising CO2, and people calling for an AI slowdown cite it too.

My take: the curve is mostly drawn right. It's the unit on the y-axis that's wrong. "Minutes" look like an even scale, but the ruler is soft in the middle: the 10× from 3 to 30 minutes is cheap, the 10× from 30 minutes to 5 hours is the expensive one. Using "doubles every N months" to predict when AI does a week of work is extending a trend line on a ruler nobody calibrated.

The evidence is Drew T. Nguyen and William Fithian (UC Berkeley Statistics), "On the estimation and validity of AI time horizons — a statistical look at the METR plot" (arXiv:2610.12466, 8 Oct 2026). They refit the plot from METR's public data: Time Horizon 1.1, 228 tasks, 79 task families, 26 AIs, running up to Claude Mythos in May 2026.

How the ruler is made

METR first gives each task a "human time": professional software engineers attempt it, about 4 attempts per task, and the times of the successful attempts are combined with a geometric mean. Each AI then runs each task repeatedly, and its time horizon is the task length it completes with 50% probability. GPT-4's estimate, for example, is 4 minutes: on a task a human finishes in 4 minutes (like "Google this fact"), it succeeds on its own about half the time.

The key assumption hides in the fit: how hard a task is for an AI is a straight line in the log of human time. Every doubling of human time adds the same fixed step of AI difficulty, everywhere on the scale.

Between 2 and 30 minutes, the ruler is flat

The authors replace that straight line with a monotone spline, and add an item-response (IRT) model with task-family effects and correlated repeat runs (their Model 2), so the data decides what "human time → AI difficulty" looks like. Most of the range is close to linear. But the stretch from 2 to 30 minutes is nearly flat. A task that takes a human 3 minutes and one that takes 30 are about equally hard for an AI.

Hence the line in the abstract: a time-horizon jump from 3 minutes to 30 minutes is much easier than one from 30 minutes to 5 hours, despite the same 10× multiplier. Look at success rates at fixed task lengths and the AI with a 4-minute horizon and the one with a 15-minute horizon are much closer at every length than any other jump.

On concrete models: GPT-4o's 50% horizon is 7.2 minutes under METR's baseline method and 4.4 minutes under the new model. The biggest changes hit the GPT-4-era AIs with horizons of 2–15 minutes, and some new estimates fall outside the original plot's 95% confidence intervals. Across 12 proper scoring rules under 5-fold cross-validation, the point estimates of both new models beat the baseline on every one.

In other words, part of the steep-looking climb through the GPT-4 era comes from the ruler being soft right there.

The trend survives; "minutes" doesn't

To be fair, so this doesn't get forwarded as "the METR plot is fake":

The paper doesn't challenge the exponential growth. Refit with the new model, horizons still double roughly every 6.0 months (R² = 0.931). And there's a more interesting finding: take the model's abstract ability score (the Rasch ability), rescale it, and it lands almost exactly on METR's original minute values, R² = 0.996.

The authors' reading: the linear model claims to estimate "how long a task the AI can do", but it may be inadvertently estimating something else, an abstract capability score. Because the flat region is confined to 2–30 minutes, the straight-line assumption is good enough across the whole suite, so the trend and the ordering come out right. But they put it bluntly: if you believe the conversion curve isn't linear, you shouldn't call the estimate a time horizon.

So the accurate reading is: the METR plot shows AI ability scores rising steadily and exponentially. It does not show that the length of work AI can do on its own is rising at the same rate. The first is a capability ruler; the second is a promise. Nearly everyone forwarding the chart is forwarding the second.

Extrapolation is exactly where it's softest

The longest tasks in the suite are around 30 hours. The authors note there's no a priori reason longer tasks keep the linear relationship; another unusually flat or steep stretch could appear, and checks on a linear model won't detect it. "A week next year, a month the year after" is drawing the line precisely where there's no data and the ruler was never calibrated.

A few more things make the minutes softer:

• For 29% of tasks, valid human timings couldn't be obtained (unmeasurable, or too few successes), so an expert's estimate was used instead.
• When the same AI reruns the same task, independent runs would predict about 60% of run sets to be unanimous (all pass or all fail). The actual figure is 83%. Extra runs mostly re-confirm the same answer, carrying less information than their count suggests.
• In other domains the ruler may simply not work: METR itself found low correlation between human time and AI success on video understanding tasks, and Mertens et al. found weak predictiveness across a broad range of economically relevant tasks.

Why this is your problem

You probably won't refit the METR plot. But you're likely running a small version of it: an internal task set for your own agent, a pass rate or "how big a job it can handle", then used to convince your boss, set a budget or plan headcount. Both traps come along: the scale of task "size" isn't even, and rerun count isn't sample size.

One step you can run now: bucket your eval results by human time, check the success rate in each bucket, then check how unanimous your reruns are. Make a runs.csv with three columns, task,human_minutes,success (one row per run, success is 0 or 1):

import csv, math, collections
by_task, mins = collections.defaultdict(list), {}
for r in csv.DictReader(open("runs.csv")):
    by_task[r["task"]].append(int(r["success"])); mins[r["task"]] = float(r["human_minutes"])
bins = collections.defaultdict(list)
for t, ys in by_task.items():
    bins[4 ** round(math.log(mins[t], 4))].append(sum(ys) / len(ys))  # ~1, 4, 16, 64 ... min
for b in sorted(bins):
    v = bins[b]; print(f"~{b:>6g} min  tasks={len(v):3d}  success={sum(v)/len(v):.0%}")
multi = [ys for ys in by_task.values() if len(ys) > 1]
if multi: print(f"unanimous run sets: {sum(len(set(ys)) == 1 for ys in multi)/len(multi):.0%}")

If neighbouring buckets have about the same success rate, longer tasks in that range aren't actually harder for your agent, so don't announce progress there. If your reruns are highly unanimous, five more runs are worth less than five more distinct tasks.

Next time you see "the horizon doubled again", ask: on tasks of a fixed length, say 1 hour or 4 hours, what did the success rate go from and to? That's the reading the paper recommends, and it's far more honest than a single 50% horizon.

In one line: the METR plot is a good capability ruler, not a good stopwatch. Use it to compare who's stronger, fine. Use it to work out which year AI does a week of your work, and you're timing a distance the stopwatch was never calibrated for.