Reproducibility report: Inferring change points in the spread of COVID-19 reveals the effectiveness of interventions

OpenRepro

Epidemiology · 2020

Inferring change points in the spread of COVID-19 reveals the effectiveness of interventions

Does COVID-19 change points (Dehning et al., 2020) reproduce? Yes, on its headline claims: when OpenRepro re-ran the authors' code, 4 of 4 headline claims held within tolerances registered before results were recorded; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it (checked 18 Oct 2026).

Jonas Dehning, Johannes Zierenberg, F. Paul Spitzner, Michael Wibral, João Pinheiro Neto, Michael Wilczek, Viola Priesemann. Science, 2020. arXiv 2004.01105 · doi:10.1126/science.abb9789

Does “COVID-19 change points” reproduce?

Reproduced

Yes, on its headline claims. 4 of 4 headline claims held within tolerances registered before results were recorded; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it.

How far the re-run landed from the published number, in tolerances7 claims re-run, 4 headline

Held (large: headline claim; small: other claim)

The data behind this chart

Claim	Published	Observed	Accepted if	Deviation ÷ tolerance	Result
Before the first change point the spreading rate in Germany was 0.43 per day. (headline)	0.43	0.4275	within ±0.02	-0.13	Held
After the first change point the spreading rate was 0.25 per day. (headline)	0.25	0.2509	within ±0.02	+0.05	Held
After the second change point the spreading rate was 0.15 per day. (headline)	0.15	0.1544	within ±0.02	+0.22	Held
After the third change point the spreading rate was 0.09 per day. (headline)	0.09	0.0937	within ±0.02	+0.19	Held
The first change point occurred around March 6, 2020.	6	5.79	within ±2	-0.10	Held
The second change point occurred around March 15, 2020.	15	15.22	within ±2	+0.11	Held
The third change point occurred around March 23, 2020.	23	22.63	within ±2	-0.19	Held

Headline claims are the results the paper is chiefly known for; they decide the outcome. Other claims are reported alongside.

Registration. Claims and tolerances registered at events 7–13; first result recorded at event 31. Registration came before results were recorded, which does not show that no output had been seen (a legacy attempt: no sealed plan). Event numbers refer to the attempt's log.

Checked 18 October 2026.

How this report was made. The re-run was carried out by an AI agent (Claude, made by Anthropic) in a sandbox. Each run is driven by the AI with no person in the loop, and the log's actor field records this. A person oversees the project. No domain expert has reviewed this report. The authors have a right of reply, and their notes can be published alongside it.

What the attempt found

Fitting the authors three-change-point SIR model with weekly reporting modulation (repository state on the date of arXiv v3) to German case data until 21 April 2020, using the JHU snapshot bundled with the repository, reproduces all registered values: spreading rates 0.43, 0.25, 0.15 and 0.09 (posterior medians 0.428, 0.251, 0.154, 0.094) and change points around March 6, 15 and 23 (posterior medians March 5.8, 15.2 and 22.6). The credible intervals also match Fig. 3 closely. Only the main three-change-point model was refitted; the 0-2 change-point comparisons and forecasts were not.

Assurance

Level: Legacy (what the levels mean)
Plan: Not sealed (legacy): claims were registered before results were recorded, which does not show that no output had been seen
Evidence: Legacy matching: each observed value was matched against the text of the logged output it cites
Execution manifest: No: not recorded for every cited run, or incomplete (no Git checkout, a driver file that could not be hashed)
Report re-derived from the log: Passed: the engine re-derived the report from the log
Engine version: 0.2.1
Predates: plan seal, evidence binding, execution manifest, paper text hash, scored job flag, retry reasons, extraction rule
Qualifiers: legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it

Headline claims

These decide the outcome. Values are as published and as observed by re-running the code; the accepted-if rule was registered before results were recorded. Each row has a ¶ link of its own.

Claim	Published	Observed	Accepted if	Result
Before the first change point the spreading rate in Germany was 0.43 per day.¶Abstract, p. 1 · registered at event 7 · quote verified on p. 1 · equivalence check	0.43per day	0.4275	within ±0.02	Held
After the first change point the spreading rate was 0.25 per day.¶Abstract, p. 1 · registered at event 8 · quote verified on p. 1 · equivalence check	0.25per day	0.2509	within ±0.02	Held
After the second change point the spreading rate was 0.15 per day.¶Abstract, p. 1 · registered at event 9 · quote verified on p. 1 · equivalence check	0.15per day	0.1544	within ±0.02	Held
After the third change point the spreading rate was 0.09 per day.¶Abstract, p. 1 · registered at event 10 · quote verified on p. 1 · equivalence check	0.09per day	0.0937	within ±0.02	Held

Other claims checked

Reported alongside the headline claims; they do not change the outcome.

Claim	Published	Observed	Accepted if	Result
The first change point occurred around March 6, 2020.¶Abstract, p. 1 · registered at event 11 · quote verified on p. 1 · equivalence check	6day of March	5.79	within ±2	Held
The second change point occurred around March 15, 2020.¶Abstract, p. 1 · registered at event 12 · quote verified on p. 1 · equivalence check	15day of March	15.22	within ±2	Held
The third change point occurred around March 23, 2020.¶Abstract, p. 1 · registered at event 13 · quote verified on p. 1 · equivalence check	23day of March	22.63	within ±2	Held

Deviations from the paper's setup

minorData end pinned to 2020-04-21 and data taken from the repository bundled JHU snapshot (confirmed_global_fallback_2020-04-28.csv) instead of the live JHU download; only the three-change-point model is fitted; pm.sample uses a fixed random_seed.
minorPython 3.7 with PyMC3 3.8, Theano 1.0.4, numpy 1.18.5, pandas 1.0.5, scipy 1.4.1 and arviz 0.7.0 from conda-forge, CPU only.

The run

Wall time: 53 min 40 s, 18 October 2026 to 18 October 2026
Job compute: 49 min 13 s across 2 jobs
Code the results name: Priesemann-Group/covid19_inference_forecast at 2960f28
Work tree at the end: Priesemann-Group/covid19_inference_forecast at 2960f28
Container image: repro-sandbox:0.2 sha256:36b1e94bee34…
Sandbox: 3.5 CPUs, 6 GB, no GPU
Run by: Claude (claude-opus-5-5), an AI agent made by Anthropic, in a Claude Code session
Attempt: arxiv-2004-01105-20261018T033025Z
Log: 44 events, head 90e4d597

Evidence

The full evidence report has the exact paper quotes, the method behind every number, the reason for each tolerance, notes made during the attempt and the environment it ran in. The raw report.json is the unmodified data this page is built from.

Log seal · 44 hash-chained events

90e4d597839451c42440893ea4dea7451ef11eaedc3887da05aa3c80dcc11e78

The head hash of the attempt's append-only log. The engine generated this report from that log; the verifier checks the log and its artifacts against this hash and that the report names it, but not the report's contents. Download the evidence bundle (the log, every artifact and the report; SHA-256 133120d227942583…) and check it with the verifier, which needs only Python.

Released 19 October 2026. Authors notified: courtesy note 19 October 2026. Reply status: Courtesy note sent to the corresponding author at publication. No reply on record.

Share image (1200×630), drawn from this report.

How to cite this report

OpenRepro (2026). Reproducibility report: Inferring change points in the spread of COVID-19 reveals the effectiveness of interventions. Reproduced; 4 of 4 headline claims held (legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it). Attempt arxiv-2004-01105-20261018T033025Z, log head 90e4d597839451c4, checked 18 Oct 2026. https://openrepro.org/papers/2004.01105/

BibTeX

@techreport{openrepro-2004.01105,
  author      = {{OpenRepro}},
  title       = {Reproducibility report: Inferring change points in the spread of COVID-19 reveals the effectiveness of interventions},
  institution = {OpenRepro},
  year        = {2026},
  url         = {https://openrepro.org/papers/2004.01105/},
  note        = {Attempt arxiv-2004-01105-20261018T033025Z; log head 90e4d597839451c4; checked 18 Oct 2026}
}

To cite the whole ledger, use CITATION.cff on the data page.

Badge

Authors and maintainers may embed this badge; it always reflects the current report, and a correction updates it.

Markdown

[![OpenRepro: Reproduced. 4 of 4 headline claims held; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it.](https://openrepro.org/badge/2004.01105.svg)](https://openrepro.org/papers/2004.01105/)

HTML

<a href="https://openrepro.org/papers/2004.01105/"><img src="https://openrepro.org/badge/2004.01105.svg" alt="OpenRepro: Reproduced. 4 of 4 headline claims held; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it." width="416" height="20"></a>

Right of reply, and reporting an error

Authors may submit a response, which is published alongside this report unedited. Write to hello@openrepro.org.

Anyone, not only an author, can report an error in an attempt to the same address. The same address takes suggestions of papers to check and questions about the method.