What is QOBLIB, and how do you use a pre-registered benchmark?
What is QOBLIB?
QOBLIB — the Quantum Optimization Benchmarking Library, subtitled "The Intractable Decathlon" — is a public library of ten optimization problem classes with 1,264 fixed instances, built so that quantum and classical solvers can be compared on the same, unmovable test (QOBLIB site, accessed 2026-09-09). It was published in April 2025 by 27 authors (arXiv:2504.03832, v1 2025-04-04, revised 2025-08-28) from a consortium of roughly 17 organizations — Zuse Institute Berlin, TU Berlin, IBM Quantum, Purdue University, E.ON, Forschungszentrum Jülich, University of Southern California, National University of Singapore, and a list of quantum software companies, per the launch report by Quantum Computing Report (2026-08-15) — and the peer-reviewed version appeared in Nature Computational Science 6, 653–671 (2026). The instances, the reference results and the submission rules are open under Apache 2.0 on GitHub.
The ten classes, one line each — all rows sourced from the paper (arXiv:2504.03832):
| # | Class | What it asks | Where it bites |
|---|---|---|---|
| 01 | Market Split | multi-dimensional subset-sum feasibility | classic MIP stress test; hard for solvers at well under 200 binary variables |
| 02 | LABS | low-autocorrelation binary sequences | radar and communications engineering |
| 03 | Minimum Birkhoff decomposition | write a doubly stochastic matrix as few permutations | network switching, scheduling |
| 04 | Steiner tree packing | route disjoint trees through a graph | VLSI wire routing |
| 05 | Sports scheduling | build a feasible tournament calendar | leagues, broadcast planning |
| 06 | Multi-period portfolio | rebalance a portfolio over time with costs | finance (see our portfolio verdict) |
| 07 | Maximum independent set | largest set of mutually non-adjacent nodes | graph problems; the class neutral-atom vendors demo on |
| 08 | Network design | build a cheapest feasible communications network | telecom capacity planning |
| 09 | Capacitated vehicle routing | serve clients with capacity-bounded vehicles | logistics (see our routing verdict) |
| 10 | Topology design | graphs of minimum diameter under degree limits | network topology |
The design constraint that makes the library interesting: every class was chosen to become hard for state-of-the-art classical solvers at small sizes — the paper puts the range between under 100 and roughly 100,000 decision variables (arXiv:2504.03832) — which is the only regime near-term quantum hardware can even load.
What makes a benchmark "pre-registered"?
The instances and the rules are fixed and public before anyone makes a claim. That single property removes the two failure modes Rosetta Quantum keeps finding in quantum benchmarking: the movable test (pick the instance after you know your solver's sweet spot) and the weak baseline (compare against a classical method nobody would actually use — our baseline post covers why that invalidates a result, and our benchmark-cheating post catalogs the tricks).
QOBLIB's submission rules operationalize this (arXiv:2504.03832): report total wall-clock time on declared hardware, disclose software versions and compute specs, report repetitions for stochastic algorithms, and — for quantum runs — count the full pipeline (payload preparation, circuit execution, sample extraction) rather than just QPU-on time. Results go in by pull request in a fixed CSV format, and the leaderboard keeps one row per instance: the best feasible submission, dated and attributed (leaderboard, accessed 2026-09-09).
Why does the classical record keep moving?
Because building an honest test makes the ruler itself better — and QOBLIB documented that effect on its own hardest small class. During the library's construction and launch window, the largest Market Split instances solved classically grew from about 60 to 110 variables — the record nearly doubled with no quantum computer involved (IBM Quantum blog, 2026-08-12). IBM's own launch post makes the point Rosetta has been making since its first verdict: a quantum-advantage claim is only as strong as the classical baseline behind it, and that baseline climbs. A pre-registered benchmark turns that climb from a threat into data — the goalposts move in public, with dates.
That is also why the hero number of this post is a classical number. On a referee's ledger, the classical record IS the bar any quantum claim will have to clear, and today the bar is rising faster than quantum entries are appearing.
What does the scoreboard say today?
Read on 2026-09-09, the library's front page reports 1,264 instances, 2,798 submissions and 533 instances solved to proven optimality; the leaderboard lists 843 instance records — instances with at least one feasible submission on file (QOBLIB site and leaderboard, both accessed 2026-09-09). Classical solvers — Gurobi, CPLEX, specialized heuristics — dominate the board. In the portion of the board our tooling could enumerate (classes 01–03, 574 of the 843 records), exactly 2 rows are tagged Quantum Hardware: two Birkhoff instances whose best-known solutions were submitted on 2026-08-04. Best-known is not proven optimal — it means nobody has beaten it yet, which includes nobody having seriously tried.
The scoreboard, with a source per row:
| Reading (2026-09-09) | Value | Source |
|---|---|---|
| Problem classes / fixed instances | 10 / 1,264 | QOBLIB site |
| Total submissions on file | 2,798 | QOBLIB site |
| Instances with a recorded best submission | 843 | leaderboard |
| Instances solved to proven optimality | 533 | QOBLIB site |
| Market Split classical record | 110 variables, up from ≈60 | IBM Quantum blog, 2026-08-12 |
| Quantum-hardware best-knowns (our partial read, classes 01–03) | 2 (Birkhoff, 2026-08-04) | leaderboard |
| Quantum advantage claims recorded on QOBLIB that we could find | 0 | leaderboard, arXiv:2504.03832 |
How does a decision-maker use QOBLIB for free?
Before paying for a single QPU minute, in this order. First, map your problem to a class: routing operations map to class 09, portfolio rebalancing to 06, network planning to 08. Second, download the instances — they are free — and run your own classical solver on them; if your in-house stack cannot approach the reference results, your bottleneck is classical, and a quantum pilot would answer the wrong question (the budget logic is in our pilot-evaluation guide). Third, when a vendor claims advantage on a problem like yours, ask for the claim in QOBLIB form: fixed public instance, total wall-clock, disclosed hardware, reproducible artifacts. A claim that cannot survive that format is not a measurement — it is marketing. Fourth, re-read the leaderboard quarterly; it is dated, so the trend is checkable.
This is the same standard Rosetta Quantum applies on its own bench: our unit of account is the advantaged solve — beat your best classical baseline on the same instance at equal budget, measured — and our own ledger's count of verified advantaged solves still stands at 0. QOBLIB is the community-scale version of that discipline, and it is the infrastructure this series has been citing since our baseline post.
What we know / what we don't know
What we know. QOBLIB exists, is open (Apache 2.0), fixes 1,264 instances across 10 classes, and publishes dated, attributed reference results (arXiv:2504.03832; site, accessed 2026-09-09). Classical solvers hold the board; 533 instances are solved to proven optimality. The Market Split classical record grew ≈60 → 110 variables over the library's construction and launch window (IBM Quantum blog, 2026-08-12). Two quantum-hardware entries hold best-known results on Birkhoff instances, dated 2026-08-04 (leaderboard).
What we don't know. Whether quantum entries exist in classes 04–10 — our leaderboard read was partial (the page truncated in our tooling at class 03), so our quantum counts are lower bounds, not totals. How many of the 2,798 submissions used quantum hardware at all. Whether the two quantum best-knowns would survive a dedicated classical attack on those specific instances — best-known records the absence of a better submission, not the impossibility of one. Whether QOBLIB's ten classes represent your workload — a pre-registered benchmark measures what it registered, nothing else. And our own limit, stated plainly: Rosetta Quantum has not yet run QOBLIB instances on its own bench, so everything above is a reading of the public record, not an independent re-measurement.
Sources
- QOBLIB — Quantum Optimization Benchmarking Library (site, accessed 2026-09-09)
- QOBLIB leaderboard (accessed 2026-09-09)
- arXiv:2504.03832 — Quantum Optimization Benchmarking Library: The Intractable Decathlon (v1 2025-04-04, v2 2025-08-28)
- Nature Computational Science 6, 653–671 (2026)
- IBM Quantum blog — QOBLIB: tracking progress in quantum optimization (2026-08-12)
- Quantum Computing Report — Global consortium launches QOBLIB (2026-08-15)
- QOBLIB repository, ZIB-AOPT (Apache 2.0)
- Rosetta Quantum — Why a weak classical baseline ruins a quantum benchmark
- Rosetta Quantum — How to evaluate a quantum pilot without burning budget
- Rosetta Quantum — What is an advantaged solve?
Rosetta Quantum publishes verdicts with raw, reproducible data. This is educational content — not a product claim, and not procurement advice. Numbers above are readings of the public record on the dates stated; where our read was partial, it says so.