The reproducibility badge

What it says

The badge is generated from the paper's report at every build, so it always shows the current outcome; if a report is corrected, the badge changes with it. It carries no number that is not on the paper's page, and it links nowhere on its own: the snippets on each paper page wrap it in a link to the report, so a reader can check the claims behind it.

Example badges for each outcome follow. Live badges contain the paper's own counts.

Reproduced. Every headline claim held within tolerance using the primary code version declared before any result was recorded (normally the authors' code at the paper date).

Reproduced. Every headline claim held within tolerance using the primary code version declared before any result was recorded (normally the authors' code at the paper date).

Reproduced. Every headline claim held within tolerance using the primary code version declared before any result was recorded (normally the authors' code at the paper date).

Reproduced only with earlier code. All headline claims held, but some only with an earlier version of the authors' code, under a fallback declared before any result was recorded.

Partially reproduced. Some headline claims held (with primary or declared earlier code) and others did not, or were untested.

Could not reproduce. None of the headline claims held, with the primary code or any declared earlier code.

Not verified: evidence incomplete. OpenRepro's own evidence for the attempt is incomplete (for example a result without its supporting log evidence, or a source quote that could not be resolved), so no reproduction outcome is given. It is a gap in OpenRepro's record, not a finding about the paper.

Not attemptable. The claims could not be tested at all, for example because the code or data is unavailable.

Where it lives

For a paper with arXiv id <arxiv-id>, listed on this site:

• https://openrepro.org/badge/<arxiv-id>.svg: the badge, a static SVG with no scripts, fonts or external resources.
• https://openrepro.org/badge/<arxiv-id>.json: the same outcome in the shields.io endpoint schema, for https://img.shields.io/endpoint?url=… if you would rather have shields.io draw it.
• https://openrepro.org/papers/<arxiv-id>/: the report page the badge should link to.

Each paper page has a Badge section with copy-ready Markdown and HTML, both linking the image to the report. A paper in its author right-of-reply period has no badge until it is listed; a paper that is not on this site has none at all.

Terms

Reports, verdicts and evidence logs, and the badge with them, are published under CC BY 4.0. The badge describes OpenRepro's re-run under the recorded conditions, not the paper's merit. Authors who disagree with a report have a right of reply: write to hello@openrepro.org.