Benchmark progress normalization

For Pro, m = Σ(tᵢ − t̄)(pᵢ − p̄) / Σ(tᵢ − t̄)² and c = p̄ − m·t̄. For each candidate τ less than the earliest DeepSWE date, fit a and b by the same OLS formula with xᵢ = ln(tᵢ − τ), then minimize Σ[dᵢ − a − b·ln(tᵢ − τ)]² over τ. The script scans and refines a broad range of time offsets, 0.0001–36,500 days before the earliest point. The optimum is 1.881551 days before that point, well inside the bounds. Moving the calendar’s coordinate origin and τ together leaves the fitted curve unchanged.

Equal weight is assigned to each new record. Long plateaus are not copied into repeated daily observations. Fit R²: Pro 0.863968; DeepSWE 0.997614. Score residual RMSE: Pro 4.918 points; DeepSWE 0.764 points. R² describes the observed fit; it is not a probability that the extrapolation is correct.

Fit only the earliest records, then predict the later records without refitting. The logarithmic curve describes this history much better than a full-history line. These small retrospective checks support the descriptive model; they do not validate its future extrapolation or establish a logarithmic benchmark scoring rule.

Single-record deletion within the shifted-log family gives approximately 14.1–16.7 equivalent Pro points in an independent replication. Removing the first two records changes the shape much more: the April–October DeepSWE records are close to linear. The early rapid-improvement period matters to the 14.8 estimate.

The logarithm models score progression over announcement dates; it does not mean DeepSWE is scored logarithmically. Its displayed score is a task-success proportion. Time-based progress is not a shared unit of capability, research effort, or engineering quality across benchmarks.

This is not a forecast of a Pro score above 100%. Adding 14.8 points to today’s 89.9% Pro leader would exceed that ceiling. The comparison uses a historical rate to express an interval, not an achievable future score.

The inputs combine model families, harnesses and configurations. Announcement dates can predate the publication of benchmark scores. Historical rows can be missing or revised. No task-level uncertainty or statistical-significance claim is inferred from the displayed aggregates.

A different question—fitting Pro score directly against DeepSWE score for shared model IDs—does not produce the same normalization. The shared models top out at 74.2% DeepSWE and have configuration differences. Neither that mapping nor this time-based one is a universal conversion between benchmarks.