How do you read a quantum arXiv paper without being a physicist?
A new quantum computing preprint lands on arXiv about 20 times a day — the quant-ph category alone took 7,298 submissions in 2021, the most recent year with per-category totals on arXiv's own statistics page (updated January 2022). None of them arrives refereed: arXiv states plainly that «material is not peer-reviewed by arXiv» (arXiv, About, accessed September 29, 2026). Rosetta Quantum reads these documents daily with a fixed protocol, and the protocol does not require a physics degree. It requires knowing where papers keep their honesty. This post is that protocol, with seven worked examples from our own ledger.
Why do you need a reading method for quantum preprints?
Because the volume is industrial and the referee is optional. arXiv moderation checks that a submission is on-topic and scholarly in form — it does not check that the claim is true; that is what «not peer-reviewed» means in arXiv's own description of itself. The gap between a hedged 30-page document and the one-line verdict it becomes in coverage is where most quantum hype is manufactured, usually on day one. The protocol below closes that gap in the other direction — toward the fine print. It is the document-level companion of the claim-level credibility tree Rosetta published earlier in this series (how to tell if a quantum advantage claim is credible).
What is the abstract for — and what is it not for?
The abstract is the authors' press release: read it to learn the claim, never to weigh the evidence. Extract three things and write them down: the object claimed (a speedup, a demonstration, a bound, a simulation), the size it was shown at (qubits, photons, instances, matrix dimensions), and the comparison target (which classical method, on what hardware, at what budget). Then skip the middle and go straight to the discussion, the limitations paragraphs and the methods: that is where the authors negotiate with reality. In the seven cases documented below, the decisive sentence was written by the authors themselves — and it was rarely the sentence the coverage quoted.
Who ran the classical baseline?
This is the question that has retired more quantum claims than any other. A quantum result is only as strong as the classical rival it beat, and a weak classical baseline invalidates the benchmark. When you read, ask four things. Is the baseline the actual state of the art, or a convenient one? Did both sides run the same instances? At comparable budget? And who ran the classical side — because when the rival is fielded by the team whose claim it threatens, that is not fraud, but it is a structural conflict the reader must price in. The record shows why the question matters: the re-runs that reversed headline claims — four documented reversals, from Sycamore to doped Clifford — came from outsiders, and in hybrid pipelines the only independent audit Rosetta has found puts the QPU at 0.7% of the wall-clock (Lozano, arXiv:2605.17623).
Where is the fine print? Seven documented cases
In the primary source itself — that is the finding. In every case below, the sentence that undercuts the headline was written by the authors or the vendor, in their own document, before any critic arrived. Rosetta's ledger documented each one during 2026; each row links the primary source and our verdict post.
| Year | Primary source | The authors' own words | Documented in |
|---|---|---|---|
| 2021 | Babbush et al., PRX Quantum 2, 010103 | the «will not enable quantum advantage» verdict on quadratic speedups is scoped, in the paper itself, to today's error-correction overheads — not to physics | our ceilings post |
| 2022 | Madsen et al. (Borealis), Nature 606, 75 | the team left full verification of the advantage claim as an open question and invited spoofing attempts in print | our GBS post |
| 2023 | Parker (RAND), arXiv:2304.14344 | the 125 MW power figure rests on an «extremely rough estimate» of ≈6 W per qubit — the label is the author's own | our energy post |
| 2024 | Google Quantum AI, Willow announcement | «conservative assumptions» and «we expect classical computers to keep improving» sit under the 10²⁵-years estimate | our Sycamore post |
| 2025 | Aghaee et al. (Microsoft), Nature 638, 651 | the paper's own text says the measurements «do not, by themselves» establish that the states are topological | our Majorana post |
| 2026 | IBM, Qiskit v2.5 release notes | the «roughly half the time» compile figure is the vendor's own benchmark; no independent cross-SDK compiler benchmark exists | our SDK panorama |
| 2026 | Wiersema, arXiv:2609.01719 | the author's verdict on his own re-run: the results «sharpen, rather than settle» the dispute — and the ≈2,100 GPU-hours are not a lower bound | our analog-vs-digital post |
Two notes of honesty. First, the guillemet quotes above are verbatim from the primary sources; the Babbush and Borealis rows are paraphrases, documented at length with full citations in the linked posts. Second, this casebook is curated, not sampled: it shows that the pattern exists and repeats across vendors, national labs and independent authors — it does not measure how often the pattern occurs, and we will not quote a rate no one has measured.
What is an artifact, and why should you ask for it first?
An artifact is anything that lets a stranger re-run the result: code, data, problem instances, random seeds, raw measurement records. A paper with public artifacts converts trust into work — anyone can check it, which is what radical reproducibility means and why pre-registered benchmark suites like QOBLIB exist. Rosetta applies the rule to itself: our weekly prospective runs ship with sealed hashes so that our own claims can be audited by strangers, including the failed weeks. The rule of thumb for a reader: a big claim with no artifacts is not wrong — it is unfinished, and it should wait in your notes until the re-run arrives.
How do you know whether the claim is still standing?
Check its half-life before you cite it, because in this field headline claims get reversed after publication, repeatedly. The 2019 Sycamore margin fell from an estimated 10,000 years to 86.4 measured classical seconds (Zhao et al., Natl. Sci. Rev. 12, nwae317, 2025), and the first peer-reviewed history of the claim-versus-counterattack record (LaRose, Quantum 10, 2198, September 2026) formally lists the 2019 claim as «Refuted» and the current random-circuit claims as «Unrefuted» — a verdict vocabulary this series has tracked claim by claim. The practical test takes five minutes: search the paper's title plus «comment», «reply», «classical simulation» or «spoofing», and check whether the counterattack clock has already run. A claim that has survived years of attacks is information. A claim too young to have been attacked is information too.
What we don't know
- We read papers; we do not referee them. This protocol catches dropped caveats, missing baselines and absent artifacts; it will not catch a subtle error inside a proof or a fabricated dataset — only replication catches those.
- Our casebook of 7 is curated, not a survey. We have not measured what fraction of quantum preprints carry caveats that contradict their own coverage, and no such measurement exists that we know of.
- The per-category arXiv figure (7,298 quant-ph submissions) is from 2021, the most recent per-category year published on arXiv's statistics page as of this writing; current volume will differ.
- Peer review is a filter, not a verdict: LaRose (2026) is one peer-reviewed history, not a consensus body, and «Unrefuted» will mean «not yet» until it doesn't.
Rosetta Quantum publishes verdicts with raw, reproducible data. This is educational content, not a product claim.