Methods

Each score is on its own test's scale. Most are the share of tasks solved; GDPval's is how often graders prefer the model's work to a professional's; FrontierSWE's is partial credit on each task, averaged over five runs; PostTrainBench's is the average score of the models the agent post-trained, across several benchmarks. The tests measure different things, so read each row against its own scale, not against the others. A record close to a perfect score leaves that test little room to show further progress. Scores sit at model release dates, often from runs with no limit on compute.

Benjamin Jones, an economist who studies where scientific progress comes from, argues that a high share of tasks solved is not a measure of progress in research: research needs every one of its steps done, so the steps a model still fails set the pace, however many others it solves. AI 2027's forecasters expect the remaining steps to fall to the models themselves.

Derived (benchmark_frontier) from: Epoch AI benchmarks and Capabilities Index; HAL AI Agent Reliability Tracker (reliability scores by agent). Epoch AI, Benchmarking Hub and Epoch Capabilities Index (CC BY 4.0). HAL: Holistic Agent Leaderboard, AI Agent Reliability Tracker, Princeton University.

HAL, a team at Princeton that runs agents on its own benchmarks, scores each agent two ways from the same runs. Accuracy is the share of tasks solved; overall reliability is HAL's index, from 0 to 1, averaging how consistently, predictably and robustly the agent behaves (safety is scored separately). Both run from 0 to 1 but measure different things, so compare how steeply each line climbs, not how high it sits. Across these agents the trend in reliability is +5.9 pts a year and in accuracy is +22.5 pts a year, where a point is a hundredth of that range; the lines are straight-line fits by this site to HAL's rows.

Derived (hal_trend_per_year) from: HAL AI Agent Reliability Tracker (reliability scores by agent). HAL: Holistic Agent Leaderboard, AI Agent Reliability Tracker, Princeton University.