Guides

Background on reproducibility, verification and replication packages, written for editors, researchers and developers.

Reproducibility vs replicability: what's the difference?
The two words are often swapped. Here is the distinction the US National Academies settled on in 2019, with examples from papers whose code was re-run.
Why do machine-learning results fail to reproduce?
Seeds, package versions, scripts rewritten after the paper, a narrow lead at the top of a ranking: the concrete causes seen when published ML code was re-run.
How to make your paper's results reproducible: a checklist
A checklist for authors of computational papers. Each item is tied to something that went right or wrong when a published paper's code was re-run.
What is a replication package?
What a replication package is, what journal data editors expect it to contain, and what re-running real packages showed about which parts matter.