Data

The ledger as data

Every number on this site comes from an attempt's report.json. These files are the same facts, one record per paper and one per claim, rebuilt from the reports whenever an attempt is published. 7 papers and 53 claims as of 19 October 2026.

The files

ledger.json: One record per listed paper, under papers. The envelope carries schema_version, version (the build date; also the version in CITATION.cff), generated, license and the paper count.
ledger.csv: The same records as CSV, one row per paper; list fields are joined with "; ".
claims.json: One record per registered claim of every listed paper, under claims, with the same envelope.
claims.csv: The same records as CSV, one row per claim.
/papers/<arxiv_id>/report.json: The unmodified report.json of the attempt behind each page, for anything the two tables leave out (deviations, notes, protocol, environment).
Evidence bundles: The hash-chained log, every artifact it names and the report, one archive per published attempt, with checksums and a one-file verifier.
feed.xml: An Atom feed with one entry per published report, newest first. Subscribe to it to see new reports as they are listed.
CITATION.cff: How to cite the ledger as a dataset. Each report page also has its own BibTeX and plain-text citation.
llms.txt, llms-full.txt: Plain-text summaries for AI assistants and answer engines: every listed paper with its outcome and URL, and a digest of every claim.
/api/arxiv/: A lookup keyed by arXiv id, for tools that show OpenRepro's result next to a paper: one small JSON file per listed paper and an index of the ids. Details.
release.json: The release manifest: every released attempt, headline or superseded, with its release record and the SHA-256 of its report files and bundle. Details.

ledger.json and ledger.csv: one record per paper

arxiv_id: the paper's arXiv id; the stable key for a paper across every file and page (/papers/<arxiv_id>/)
title, authors, year, venue, doi: citation metadata from the report (display-only corrections noted in content/editorial.json)
field: a browsing label assigned by OpenRepro
outcome, outcome_label: the computed outcome, as a key and as printed (evidence_incomplete, "Not verified: evidence incomplete", means OpenRepro's evidence failed, not the paper)
qualifiers, qualifier_text, qualifiers_source: qualifier codes (bound_checks_only, broad_bound, other_checks_failed, untested_for_budget, fail_then_pass_without_policy, claims_registered_after_output, job_tree_modified, agent_selected_extraction, plan_not_sealed, evidence_not_bound), the same in plain words, and whether they come from the report or were derived by the site from a legacy report. A code this site does not know is still carried verbatim
coverage_note: what the report says (or, for a legacy report, cannot say) about results left untested for budget
assurance: plan seal, evidence binding, execution manifest, report re-derivation, engine version and the resulting level (assurance_level in the CSV); legacy reports say so
attempt_id, attempts: the released headline attempt, and the lineage: every released attempt of the paper with its status (released or superseded), outcome, superseded_by and reason
release_on, report_json_sha256: the release date from the release record, and the SHA-256 of the headline report.json (also in release.json)
headline_held, headline_held_earlier_code, headline_total: headline claims that held, of which how many only with earlier code, out of those scored
claims_checked: every claim registered, headline or not
checked: date the attempt ended (UTC)
page_url, evidence_url, badge_url: the report page, the full evidence report, and the SVG badge
log_head_hash, log_events: the head hash and length of the attempt's hash-chained log (printed on the page as the seal)
bundle_url, bundle_sha256: the downloadable evidence bundle and its checksum, when published
summary: the report's summary paragraph

claims.json and claims.csv: one record per claim

arxiv_id, claim_id: the paper and the claim; together the stable key for a claim (the page anchor is #claim-<claim_id>)
paper: the paper's short name
headline: true when the claim counts toward the outcome
statement, location, metric, unit: what was registered: the claim in words, where it sits in the paper, and what was measured
comparison, published, tolerance, accepted_if: the registered rule: within a band, at most, or at least; the published value; the tolerance; the rule as printed
observed: the value obtained by re-running the authors' code (null when untested)
verdict: consistent, consistent_earlier_code, inconsistent or untested
check_kind, broad_bound, ratio_to_published: equivalence or bound; whether the bound is broad; observed ÷ published. From the report when it carries them, else derived by the site (legacy)
earlier_code_observed: the value under the declared earlier-code fallback, when one was tried
registered_event, first_result_event: log events at which the claim was registered and its first result recorded, so the order can be checked
deviation_over_tolerance: (observed − published) ÷ tolerance; null when untested, "inf" or "-inf" when the tolerance was zero and the value missed

Outcome keys: reproduced, reproduced_with_earlier_code, partially_reproduced, not_reproduced, not_attemptable. What they mean.

Lookup by arXiv id

For a tool that shows OpenRepro's result beside a paper, such as an arXiv abstract-page tab, a browser extension or a reference manager. These are static files, not a service: there is no key, no query string, no cookie and no logging of who asks.

/api/arxiv/<arxiv_id>.json: One record per listed paper, keyed by the arXiv id without a version suffix (1606.00931, not 1606.00931v2). Any other id returns HTTP 404, which means OpenRepro has no listed report for it.
/api/arxiv/index.json: The ids that have a record, under ids, with the same envelope as ledger.json and the URL template under lookup.

Fields of a record:

schema_version, version: the record format (1) and the build date
arxiv_id, available: the id, and true (only listed papers have a file)
title: the paper's title from the report
outcome, outcome_label: the computed outcome, as a key and as printed
headline_held, headline_held_earlier_code, headline_total, headline: headline claims that held, of which how many only with earlier code, out of those scored, and the same as a phrase with any qualifier ("4 of 4 headline claims held")
qualifiers, qualifier_text, assurance_level: as in ledger.json; show the qualifier text next to the outcome
answer: the one-sentence answer printed under the paper's title
checked: date the attempt ended (UTC)
report_url, badge_url, badge_alt, evidence_url, attempt_id: the report page to link to, the SVG badge and its alt text, the full evidence report, and the attempt id
license, attribution, disclosure: CC BY 4.0, the attribution line to show, and the AI-run, no-expert-review caveat to show with the result

The files are served with Access-Control-Allow-Origin: *, so a page on another site can fetch them. Show the outcome with a link to report_url, and keep the attribution and the disclosure next to it. Papers without a due release record have no record here.

The release manifest

release.json lists every attempt this build released, and nothing else. Release is fail-closed: an attempt appears on any surface only when its record in OpenRepro's release allowlist says it is released and its date has come, and the build fails if any page or file names an attempt that is not in this manifest.

attempts[]: arxiv_id, attempt_id, status (released: the paper's current report; superseded: an earlier version, with superseded_by and reason), outcome, release_on, notified (date, kind and number of recipients of each notice to the authors), disposition (the authors' reply status)
files: SHA-256 of the engine's report.json, report.md and report.html for the attempt, as they are inside its evidence bundle. report.json is also served unmodified at the paper's report.json URL; the /evidence/ page is a restyled copy of report.html and is not covered by that digest.
bundle: The evidence bundle's URL and SHA-256 (the same as SHA256SUMS).
head_hash, events: The seal to pass to the verifier (--expect-head) and the log's length.
approved_on: When this exact report (by its report.json digest) was reviewed and its release decision made or renewed. A regenerated report is withheld until the decision is renewed.
schema: openrepro.release/v2: the schema the engine's own release manifest uses, so python3 openrepro_verify.py <attempt> --manifest release.json checks a report or bundle against this file directly.

The manifest is published on this same site, so it shows that a file is the one OpenRepro released, not when it was released; independent anchoring is pending. Keep a copy if you need to show later that a report changed.

From an AI assistant

The same facts are served by a read-only MCP server at https://mcp.openrepro.org/mcp, so an assistant can answer "does this paper reproduce?" with a link to the report. It serves only what this site publishes, stores nothing and needs no key. In Claude Code: claude mcp add --transport http openrepro https://mcp.openrepro.org/mcp. Tools: check_paper, search_reports, get_report, how_to_verify, request_rerun.

License and citation

Reports, verdicts and evidence logs: CC BY 4.0. So are these files. Cite the ledger with CITATION.cff (the version is the build date), or a single report with the citation block on its page; either way, link the report URL so a reader can reach the evidence. Attribution: "OpenRepro, https://openrepro.org".

Reports are produced by an AI agent and no domain expert has reviewed them; see who runs the attempts. Treat an outcome as what happened when the authors' code was re-run under the recorded conditions, not as a judgment of the paper.