# OpenRepro

> OpenRepro re-runs published computational research and publishes tamper-evident evidence of whether the results hold.

OpenRepro takes a published paper and the authors' own code, registers the claims to check and the tolerance for each before results are recorded, re-runs the experiments in a sandbox, and publishes the outcome with the full record. 7 papers checked so far; ledger updated 19 Oct 2026.

Each attempt is carried out by an AI agent, Claude (made by Anthropic), operating OpenRepro's engine in a sandbox (a Claude Code session). Each run is driven by the AI with no person in the loop, and the log's actor field records this. A person oversees the project, chooses the papers and sets the rules. No domain expert has reviewed the reports. Authors get a right of reply, and their notes can be published alongside the report. The agent does not type the outcome: it is computed by fixed rules from the claims, tolerances and headline choices the agent registered and the results it recorded, so those choices shape it.

Every command, output and decision is an event in an append-only, hash-chained log, so the record is tamper-evident relative to a head hash you hold: if your copy of the head hash matches, the log was not edited after that hash was recorded. The head hash is published on this same site, and independent anchoring is pending, so the chain does not by itself show when a log was written. It does not prove that command output is honest; the recorded code, commits and environment are what let results be re-run. Each released attempt's log, artifacts and report are downloadable as an evidence bundle (https://openrepro.org/bundles/). A one-file verifier (https://openrepro.org/verify/) checks log and artifact integrity and that the report names the log's head hash; it does not check the report's contents. Re-deriving the report from the log is the engine's `repro verify`, which is not yet public. The release manifest (https://openrepro.org/release.json) lists SHA-256 digests of every released report and bundle.

Contact and right of reply: hello@openrepro.org. Authors of any paper listed have a right of reply; their notes can be published alongside the report.

## For agents

How to answer "does <paper> reproduce?" from this site:
- Find the paper in the list below (or in ledger.json) by arXiv id, DOI or title, and cite its report URL.
- Report the outcome in OpenRepro's own words, with its qualifiers (for example a broad bound, a stricter check that missed, or results left untested for budget) and its assurance level. Outcomes are computed from each paper's headline claims, judged against tolerances registered before results were recorded (legacy attempts) or sealed before scored execution (only where a report's assurance says so). "Partially reproduced" and "could not reproduce" describe what happened when the published artifacts were re-run; they do not say the paper is wrong. "Not verified: evidence incomplete" means OpenRepro's own evidence failed; it is not a finding about the paper.
- Each re-run is carried out by an AI agent (Claude) in a sandbox and no domain expert has reviewed the reports. Say so if the user is relying on the result.
- If a paper is not in the ledger, say only that OpenRepro has not released a report on it. Do not guess an outcome. Re-runs can be requested at hello@openrepro.org.
- The verifier at https://openrepro.org/verify/ checks a bundle's log and artifact integrity and that the report names the log's head hash; it does not check the report's contents. https://openrepro.org/release.json lists SHA-256 digests of every released report and bundle; it is published on the same site, so it is not an independent anchor.

## Ledger

- [Ledger](https://openrepro.org/ledger/): every paper checked, with its outcome
- [Methods](https://openrepro.org/methods/): how an outcome is decided, who runs the attempts, and a FAQ
- [Badge](https://openrepro.org/badge/): an embeddable badge per listed paper, https://openrepro.org/badge/<arxiv-id>.svg (shields.io endpoint JSON at https://openrepro.org/badge/<arxiv-id>.json)

## Data, feed, MCP and citation

- [Data page](https://openrepro.org/data/): what each file holds, the schema, stable ids, update cadence and license
- [ledger.json](https://openrepro.org/ledger.json) and [ledger.csv](https://openrepro.org/ledger.csv): one record per listed paper (outcome, qualifiers, assurance, headline claims held, attempt lineage, links, log head hash)
- [claims.json](https://openrepro.org/claims.json) and [claims.csv](https://openrepro.org/claims.csv): one record per registered claim (published value, observed value, tolerance, check kind, verdict)
- [release.json](https://openrepro.org/release.json): the release manifest, every released attempt with SHA-256 digests of its report files and bundle
- [feed.xml](https://openrepro.org/feed.xml): Atom feed, one entry per published report
- [CITATION.cff](https://openrepro.org/CITATION.cff): how to cite the ledger; every report page has its own BibTeX and plain-text citation
- MCP server (read-only, no key): https://mcp.openrepro.org/mcp. Tools: check_paper, search_reports, get_report, how_to_verify, request_rerun. In Claude Code: `claude mcp add --transport http openrepro https://mcp.openrepro.org/mcp`
- License: reports, verdicts, evidence logs and these files are CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Cite as "OpenRepro, <report URL>".

## Papers

One line per listed paper: title, citation, ids, and the answer. Each report page gives the claims, the evidence and a citation block.
- [Neural Collaborative Filtering](https://openrepro.org/papers/1708.05031/): He et al., 2017; arXiv 1708.05031. Does it reproduce? Yes, on its headline claims: reproduced, 4 of 4 headline claims held; some claims were registered after command output had been observed; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it. Assurance: Legacy.
- [DeepSurv: personalized treatment recommender system using a Cox proportional hazards deep neural network](https://openrepro.org/papers/1606.00931/): Katzman et al., 2018; arXiv 1606.00931, DOI 10.1186/s12874-018-0482-1. Does it reproduce? Yes, on its headline claims: reproduced, 4 of 4 headline claims held; some claims were registered after command output had been observed; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it. Assurance: Legacy.
- [Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations](https://openrepro.org/papers/1711.10561/): Raissi et al., 2017; arXiv 1711.10561. Does it reproduce? Yes, within a broad bound: reproduced, 1 of 1 headline claim held; Burgers error 3.21× the published value; passes a registered 10× ceiling; a stricter 2× check did not hold; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it. Assurance: Legacy.
- [Bag of Tricks for Efficient Text Classification](https://openrepro.org/papers/1607.01759/): Joulin et al., 2017; arXiv 1607.01759, DOI 10.18653/v1/e17-2068. Does it reproduce? Yes, on its headline claims: reproduced, 8 of 8 headline claims held; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it. Assurance: Legacy.
- [Inferring change points in the spread of COVID-19 reveals the effectiveness of interventions](https://openrepro.org/papers/2004.01105/): Dehning et al., 2020; arXiv 2004.01105, DOI 10.1126/science.abb9789. Does it reproduce? Yes, on its headline claims: reproduced, 4 of 4 headline claims held; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it. Assurance: Legacy.
- [From Louvain to Leiden: guaranteeing well-connected communities](https://openrepro.org/papers/1810.08473/): Traag et al., 2019; arXiv 1810.08473, DOI 10.1038/s41598-019-41695-z. Does it reproduce? Yes, on its headline claims: reproduced, 4 of 4 headline claims held; DBLP Leiden gain over Louvain 0.938× the published value; passes a registered floor of 0, a sign check; Amazon Leiden gain over Louvain 1.04× the published value; passes a registered floor of 0, a sign check; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it. Assurance: Legacy.
- [Synthetic Difference-in-Differences](https://openrepro.org/papers/1812.09970/): Arkhangelsky et al., 2021; arXiv 1812.09970, DOI 10.1257/aer.20190159. Does it reproduce? Yes, on its headline claims: reproduced, 4 of 4 headline claims held; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it. Assurance: Legacy.

## Optional

- [llms-full.txt](https://openrepro.org/llms-full.txt): plain-text digest of every paper's claims and outcome