Descant scores 82.3% on DeepSWE v1.1
What resolving 93 of 113 tasks tells us about autonomous software engineering—and what it leaves unmeasured about evolving a codebase over time.
Descant is a fully autonomous software engineering system built to advance and maintain substantial codebases: paying down technical debt, accelerating feature development, and carrying large refactors through to completion. It coordinates work across thousands of interconnected issues, learns from each run, and carries that knowledge forward as the codebase evolves over months and years. For new products, Descant can start from a single issue, a PRD, a technical design, or a backlog you’ve already structured—giving you control over how much is defined up front. It decomposes the remaining work into implementation issues, maintains their dependency graph, and carries them through to a working product.
We chose DeepSWE v1.1 because its original, long-horizon engineering tasks closely match an important part of that work. It tests whether a system can understand a codebase and implement a substantial change. The longer horizon—learning from previous work, coordinating dependencies, and carrying a product forward through thousands of changes—extends beyond what a single-task benchmark measures.
Source: llm-stats.com DeepSWE v1.1, as of October 14, 2026.
Descant resolved 93 of 113 tasks, scoring 82.3% on DeepSWE v1.1. The graphic above places Descant’s system-level result alongside the benchmark’s published frontier model scores.
Engineering across the life of a codebase
Descant decomposes complex requests into actionable issues, builds their dependency graph, and coordinates implementation and review. That work led us to create issuegraph, an open specification for expressing the relationships between issues.
As changes land, Descant reassesses the work around them. It closes issues already resolved by other changes, updates those whose requirements have shifted, and repurposes work that remains useful. It records learnings from each run and can draw on them in subsequent runs, carrying knowledge forward as the codebase evolves.
The system is designed to coordinate 1,000 concurrent runs and beyond, with dependencies determining which work can proceed together.
We have used Descant to refactor and evolve TriviaPot across thousands of issues, and to take OH HAI from a single issue to an operational product. Those projects exercise the continuity that isolated benchmark tasks leave unmeasured: each change becomes part of the environment in which the next decision is made.
Why DeepSWE
DeepSWE’s tasks and reference solutions are newly authored against existing codebases. They span TypeScript, Go, Python, JavaScript, and Rust, with purpose-written verifiers that evaluate the requested behavior. This makes it a useful test of solving unfamiliar engineering problems. DeepSWE’s methodology explains how that construction differs from benchmarks assembled from existing fixes.
We originally evaluated Descant on SWE-bench Pro, scoring 86.6%, with later runs approaching near-perfect scores. Published audits also identified weaknesses that could inflate results: insufficient test coverage allowed incomplete solutions to pass, and reference fixes accessible through repository history could be recovered rather than independently developed. DeepSWE’s original tasks and purpose-written verifiers offered a stronger basis for measuring how Descant handles unfamiliar, challenging engineering work.
FrontierSWE measures exceptionally difficult engineering and research work, including systems implementation, performance optimization, and scientific computing. It explores the technical capabilities of the frontier models Descant brings together. For this evaluation, we wanted to measure how effectively Descant combines those capabilities to deliver challenging, long-horizon engineering work in existing codebases. DeepSWE’s mix of substantial changes across varied projects was the closer fit for the work teams would entrust to Descant.
Broader agent benchmarks such as Terminal-Bench can measure useful component capabilities. They don’t establish whether a system can manage the continuing evolution of a software product. None of these scores, including ours, answers that larger question on its own.
What the results taught us
Reviewing the results surfaced omissions and contradictions between issue descriptions and hidden tests: an unstated name or value, or a test expecting behavior different from what the issue described.
Ambiguity is also part of real software engineering. Descant reasons through it using the requirements and evidence available in the repository. When two plausible interpretations can coexist cleanly, it can implement a solution that supports both, preserving valid behavior without making an unsupported choice.
When the alternatives are mutually exclusive, multiple models collaborate to examine the direct evidence and determine the most defensible direction. The reasoning effort is matched to the problem: a narrow uncertainty should not require the same deliberation as an architectural decision.
Our review also identified cases where Descant made a defensible engineering choice that the hidden tests rejected. Some tests required behavior the issue never specified or contradicted what it requested. In other cases, Descant deliberately departed from a stated requirement to preserve correctness or protect existing data, explaining that choice in the pull request.
Those tasks remain failures in the reported score. We stand behind choices where the evidence supports the engineering reasoning. Benchmarks provide useful evidence, but the outcome that matters is reliable software that evolves with your business and delivers value to your customers.
All 113 benchmark task repos and the solutions are published. If you’d like to dig into the code, PRs, and task notes, access is available on request.