About OpenRepro

Why reproduction matters

A number in a paper is a claim about what some code produced on some data. Readers, reviewers and later work build on it, but it is rarely run again: re-running is slow, and a failed attempt is hard to interpret, because the paper, the code, the software environment and the person re-running it are all suspects.

OpenRepro does the re-running in the open, as an independent check that a paper's published code and data still produce its results. Each attempt says in advance what it will check and how close is close enough, then records everything it does. A result that holds can be relied on with more confidence. A result that does not usually comes with a reason in the record: a package that changed after publication, a script updated after the paper, a run that depends on its random seed.

How close is close enough is fixed per claim, before the run, with a written reason. The chart on the home page is drawn in those units: one band width is one tolerance, whatever the claim's own scale.

Who runs it

One person runs OpenRepro and is accountable for what it publishes. They are not named on this site; the project speaks only through its own address, hello@openrepro.org. OpenRepro has no funding, sponsor or institutional backing, and it is not affiliated with the authors of any paper it checks.

Who does the re-runs

The re-runs are carried out by AI agents: Anthropic's Claude, working in a sandbox. Each paper page names the model that ran the attempt as its log records it. OpenRepro is not affiliated with Anthropic, and is not funded or endorsed by it.

Each attempt is carried out by Claude, an AI model made by Anthropic, operating OpenRepro's engine in a sandbox. Each run is driven by the AI with no person in the loop, and the log's actor field records this. A person oversees the project, chooses the papers and sets the rules the agent works under. The agent does not type the outcome: it is computed by fixed rules from the claims, tolerances and headline choices the agent registered and the results it recorded, so those choices shape it. Each paper page names the model that ran it, as its log records it.

The driver is in the log. The model that ran each attempt is recorded in its log and printed under “The run” on the paper's page, next to the code commit, the container image and the sandbox limits. More on who runs the attempts.

What is not reviewed

OpenRepro has not arranged independent domain-expert review, and the reports are not peer reviewed. Authors can respond; a published response does not establish endorsement or independent validation. The outcome comes from fixed rules applied to what the agent registered and recorded (Methods explains the rules and their limits), and the registrations themselves are the agent's choices. A report says what happened when the published artifacts were re-run; it is not peer review and does not say whether a paper is right. Replies are published with the author's permission, in full or in the parts they choose, and any omission is marked.

What OpenRepro publishes

Reports. Each paper page lists the claims checked, with the published value, the re-run value and the rule registered for each. It gives the qualifiers, the assurance level and every deviation from the paper's setup, and links the full evidence report, which cites the log event behind every number. Papers by field · Suggest a paper

Data. The ledger and every claim are also published as data, rebuilt from the reports at every release. Only listed papers are included. Cite the ledger as a whole with CITATION.cff. ledger.csv · ledger.json · claims.csv · Atom feed

Verification. Each paper page links its raw report.json, the unmodified data the page is built from, and its evidence bundle when published: the hash-chained log, every artifact it names and the report. Attempts abandoned before any result have no evidence bundle. A single-file Python verifier checks the chain, the artifacts and the head hash printed on the page. It does not check the report's contents. release.json lists the digest of every released report and bundle. The verifier · Evidence bundles · What the chain proves

Corrections

Errors are corrected whichever way they cut, for or against a paper. Published reports are never edited in place: a correction is a new attempt, the earlier report stays public as a superseded version with the reason it was replaced, and the error is listed with its date on What we got wrong. To report an error, write to hello@openrepro.org (see Contact).

Contact

hello@openrepro.org. What to send, and the authors' right of reply. See also Privacy and Terms.