Scaling, evidence and use-case thresholds
Why additional workers can hurt
❓ When does a larger swarm produce less useful work?
A task has an inherently serial part, work that can proceed in parallel, and coordination and integration overhead. Adding workers reduces the elapsed time of the parallel portion only until the remaining sequential work, contention, verification or cost dominates. This is an engineering explanation, not a claim that a fixed universal number of agents is optimal. Two agents that share an incorrect assumption can amplify it; two that independently discover the same issue may waste effort; two editing the same interface may produce incompatible designs.
In a 2025 study, Google Research and MIT evaluated multiple agent architectures and found that the benefit depended on task structure: parallelizable tasks could benefit, while sequential tasks could degrade. Their reported error-amplification figures apply to their defined experiments, not to every future swarm or real-world system (paper, Google Research). Anthropic's 2026 finding that comparable core-directory search had similar token efficiency despite the swarm finding many more total vulnerabilities illustrates why task scope and budget matter (Anthropic).
A useful independent counterpoint comes from Vals AI's October 2026 application-building study: on 50 web-app specifications, agent teams cost 1.8–5.1 times as much as their corresponding single-agent configurations, and only one of four team-versus-single differences in score was statistically significant. The test used a maximum of five simultaneous subagents, one run per setup, and browser-based UI grading. It does not invalidate large-scale swarm achievements on different tasks; it demonstrates why their extra cost and the alternative of giving a single agent more reasoning time must be measured.
Good benchmarks compare single agent, independent parallel agents, and coordinated swarm using the same task distribution, time and total budget; measure accepted outcomes, integration time, human intervention, security failures and confidence, not merely calls, lines or commits. Vary the worker count and observe where returns diminish. Test on new tasks not used to tune the prompt and harness. Report failures and uncertainty, not only the most impressive run.
The level needed depends on the use case
❓ Which capabilities must reach what threshold before a particular application is defensible?
There is no established universal numerical ladder. The following levels are proposed decision gates, not industry standards, predictions, or claims that agents have passed a quantified universal threshold. Each is defined by properties that can be measured on the intended task.
| Application | Required capability and evidence | Current evidence and limiting condition |
|---|---|---|
| Parallel search or document triage | Work can be partitioned; candidates include provenance; qualified people or reliable checks filter outputs; no external side effects. Compare against independent parallel calls. | Practicable as supervised workflows, but usefulness depends on retrieval coverage and reviewer load. |
| Vulnerability discovery in authorized code | Isolated workers, agreed scope, reproducible proof of findings, independent validation, no access to unauthorized targets. | Anthropic's 45-agent experiment found many candidate vulnerabilities, with a material difference in search scope versus the comparator. |
| Formalization and mathematical assistance | Precise formal statements, inspectable assumptions, proof dependencies, a trusted checker, and expert judgment about whether the statement matches the intended theorem. | Axiom reports CFSG formalization, with major human planning, intervention and mathematical consultation. Formalization of an existing result differs from discovering an entirely new theorem. |
| Bounded software implementation | Testable specification, isolated change sets, reliable integration, held-out tests, dependency and security checks; maintainers judge operational fitness. | Cursor's database reached reported held-out SQL test outcomes. This is not full database equivalence or a production-readiness certificate. |
| Scientific discovery | Hypothesis generation plus reliable experimental design, measurement, replication, domain review and a path to real-world intervention. | Computational search can accelerate candidate generation; in many domains the physical experiment and causal interpretation remain the bottleneck. A mathematical formalization is not a clinical result. |
| Operational enterprise workflows | End-to-end identity and access controls, externally enforced transaction limits, observability, audit, rollback, accurate exception handling, and measured reliability across real cases. | A successful benchmark does not establish authorization, data quality or compliance in a particular enterprise. Begin with read-only or reversible work and explicit human authorization. |
| High-impact autonomous action | Demonstrated robustness under distribution shift, independently enforced boundaries, high-confidence harm estimates, resilient monitoring, recovery, accountable ownership, and independent evaluation. | No universal acceptable error rate; use-case-specific tolerances and evidence are necessary. A one-off success is insufficient. |
A verifier checks whether output meets some condition. A proof kernel can check deductions under a formal specification; a test suite samples software behavior; a human reviewer judges purposes and context. None is a magic oracle: a formal proof can prove the wrong statement, a test suite can miss a crash or exploit, and humans can miss errors or rubber-stamp approvals. The harder the consequences and the weaker the verifier, the stronger the containment and human accountability required.
Trend analysis without invented curves
❓ Are agent swarms improving exponentially?
Swarm task performance: too data-poor to classify as exponential. Axiom's proof, Cursor's SQLite results, and Anthropic's vulnerability study span different tasks, test sets, numbers of agents, budgets and architectures during 2026. They are milestones, not repeated measurements of a common metric. They show that substantial multi-agent work is possible in controlled settings; they cannot establish a doubling time. More commit throughput does not automatically mean more correct work.
Coordination infrastructure: stepwise observed advances. Cursor's shift from its older setup to a custom agent-oriented version-control system is a documented architectural change, not a stable multi-year price-performance series. The causal pressure is understandable: agent-produced edits and conflicts can exceed the capacity of human-tempo integration, while verification and semantics remain limiting.
Agent-assisted replication: forecast, not observed exponential worm spread. Anthropic fitted a line to a proxy capability index for small models and forecast when a proposed floor might be reached. The proxy and simplified test conditions prevent translating the curve into a demonstrated real-world replication rate. The evidence supports monitoring this risk, not asserting a calendar date as fact.
Costs and the next bottleneck. Cheaper worker inference and better delegation may make large projects economically feasible, while external verification, integration, data access, compute availability and expert attention can remain scarce. The relative importance of these bottlenecks can change by domain. Axiom's cached proof checking and Cursor's coordination tooling are examples of infrastructure responding to observed bottlenecks, not evidence that a single fixed bottleneck always dominates.
A disciplined first deployment
❓ How should an organization decide whether a swarm is worth using?
Choose a bounded task with historical examples and a clear expected output. Establish the best practical human or single-agent baseline and an independent-parallel baseline. Assign read-only, least-privilege access, fixed budgets, separate protected logs and a human owner. Keep a genuinely held-out evaluation set. Measure validated outcomes, elapsed time, total compute and engineering costs, failure severity, duplicate work, intervention time and near misses. Expand permissions only after measured evidence supports it; halt on unexpected external behavior or inability to audit. This is a practical evaluation design, not a guarantee of safety.