OpenRepro re-runs published computational research and publishes tamper-evident evidence of whether the results hold. One entry per published report. https://openrepro.org/ 2026-10-19T16:00:36.780Z OpenReprohttps://openrepro.org/hello@openrepro.org Reports, verdicts and evidence logs are CC BY 4.0. OpenRepro site build https://openrepro.org/papers/1708.05031/ 2026-10-19T16:00:36.780Z 2026-10-19T16:00:36.780Z OpenRepro

Does Neural Collaborative Filtering (He et al., 2017) reproduce? Yes, on its headline claims: when OpenRepro re-ran the authors' code, 4 of 4 headline claims held within tolerances registered before results were recorded; some claims were registered after command output had been observed; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it (checked 19 Oct 2026). The authors Keras code (commit 312aff0, identical in code to the current default branch) was run on MovieLens-1M with the exact README environment (Python 2.7.15, Keras 1.0.7, Theano 0.8.0) on CPU, using the README commands with 8 predictive factors and 20 epochs. All six pre-registered Table 2/3/4 values at 8 factors landed within 0.005 of the paper: NeuMF without pre-training HR@10 0.6848 vs 0.688 and NDCG@10 0.4088 vs 0.410; NeuMF with pre-training (initialised from the authors bundled GMF/MLP weights rather than retrained ones) 0.6798 vs 0.684 and 0.4054 vs 0.403; MLP-3 0.6707 vs 0.671 and 0.3979 vs 0.399. Each value is one run reported at the best-HR epoch, which is the scripts own convention. Mid-attempt the shared cloud droplet was destroyed by another process; the environment and code were rebuilt with the same recipe and only the in-flight MLP run was repeated.

CC BY 4.0 https://openrepro.org/papers/1606.00931/ 2026-10-19T00:51:39.239Z 2026-10-19T00:51:39.239Z OpenRepro

Does DeepSurv (Katzman et al., 2018) reproduce? Yes, on its headline claims: when OpenRepro re-ran the authors' code, 4 of 4 headline claims held within tolerances registered before results were recorded; some claims were registered after command output had been observed; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it (checked 19 Oct 2026). Rerunning the first author code at the commit matching arXiv v3 (079dee7, 2017-08-08), with the documented command for each experiment in a reconstructed Python 2.7 / Theano 0.9 / Lasagne environment on CPU, reproduces all six DeepSurv C-index values of Table 1 within 0.01: mean of 10 unseeded runs SUPPORT 0.6186 vs 0.6183, METABRIC 0.6435 vs 0.6434, Rotterdam & GBSG 0.6720 vs 0.6684, WHAS 0.8580 vs 0.8626 (run-to-run sd 0.009), simulated linear 0.7739 vs 0.7740, simulated nonlinear 0.6458 vs 0.6489. The WHAS model file in the repository does not match the WHAS hyper-parameters listed in the paper Table 3, and every run ends with a weight-saving error after the metrics are printed (fixed upstream in December 2017). CPH, RSF and the treatment-recommendation results were not scored.

CC BY 4.0 https://openrepro.org/papers/1711.10561/ 2026-10-19T00:06:41.787Z 2026-10-19T00:06:41.787Z OpenRepro

Does PINNs (Raissi et al., 2017) reproduce? Yes, within a broad bound: when OpenRepro re-ran the authors' code, 1 of 1 headline claim held within tolerances registered before results were recorded; Burgers error 3.21× the published value; passes a registered 10× ceiling; a stricter 2× check did not hold; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it (checked 19 Oct 2026). Running the authors Burgers.py (the earliest version in their repository, from five months after arXiv v1) with its fixed seeds on CPU gives a relative L2 error of 2.15e-3 for the continuous-time Burgers example, against 6.7e-4 reported. That meets the pre-registered order-of-magnitude headline check (at most 6.7e-3) but not the stricter within-a-factor-of-2 check. The paper own Table 1 shows errors varying between 1e-4 and 1e-2 across neighbouring settings, and this run used a CPU rather than the reported GPU. The Table 1 sweep was not run.

CC BY 4.0 https://openrepro.org/papers/1607.01759/ 2026-10-18T23:44:49.124Z 2026-10-18T23:44:49.124Z OpenRepro

Does fastText (Joulin et al., 2017) reproduce? Yes, on its headline claims: when OpenRepro re-ran the authors' code, 8 of 8 headline claims held within tolerances registered before results were recorded; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it (checked 18 Oct 2026). With fastText at the commit of arXiv v3 and its classification-results.sh, test accuracy with 10 hidden units and bigrams reproduces Table 1 on all eight datasets: 92.2, 96.7, 98.6, 95.7, 63.9, 72.4, 60.2 and 94.6 against 92.5, 96.8, 98.6, 95.7, 63.9, 72.3, 60.2 and 94.6. The unigram row also reproduces, within 0.2 points everywhere. The script own Google Drive download no longer works, so the datasets came from a public mirror of the same tarballs, preprocessed with the script own function. Training speed (Table 2) was not tested.

CC BY 4.0 https://openrepro.org/papers/2004.01105/ 2026-10-18T04:24:05.955Z 2026-10-18T04:24:05.955Z OpenRepro

Does COVID-19 change points (Dehning et al., 2020) reproduce? Yes, on its headline claims: when OpenRepro re-ran the authors' code, 4 of 4 headline claims held within tolerances registered before results were recorded; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it (checked 18 Oct 2026). Fitting the authors three-change-point SIR model with weekly reporting modulation (repository state on the date of arXiv v3) to German case data until 21 April 2020, using the JHU snapshot bundled with the repository, reproduces all registered values: spreading rates 0.43, 0.25, 0.15 and 0.09 (posterior medians 0.428, 0.251, 0.154, 0.094) and change points around March 6, 15 and 23 (posterior medians March 5.8, 15.2 and 22.6). The credible intervals also match Fig. 3 closely. Only the main three-change-point model was refitted; the 0-2 change-point comparisons and forecasts were not.

CC BY 4.0 https://openrepro.org/papers/1810.08473/ 2026-10-18T01:48:45.899Z 2026-10-18T01:48:45.899Z OpenRepro

Does From Louvain to Leiden (Traag et al., 2019) reproduce? Yes, on its headline claims: when OpenRepro re-ran the authors' code, 4 of 4 headline claims held within tolerances registered before results were recorded; DBLP Leiden gain over Louvain 0.938× the published value; passes a registered floor of 0, a sign check; Amazon Leiden gain over Louvain 1.04× the published value; passes a registered floor of 0, a sign check; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it (checked 18 Oct 2026). Using the authors Java implementation at the release tag matching the paper (networkanalysis 1.0.0) on the SNAP DBLP and Amazon networks, with the paper protocol of 10 random starts of 10 iterations each, all tested Table II values reproduced within 0.0005: DBLP Louvain 0.8267 vs 0.8262 and Leiden 0.8384 vs 0.8387; Amazon Louvain 0.9298 vs 0.9301 and Leiden 0.9340 vs 0.9341. Leiden found higher modularity than Louvain on both networks, as claimed. The other four networks in Table II were out of scope (download blocked, proprietary, or too large for the sandbox), and the badly connected community percentages in Fig. 4 were not tested.

CC BY 4.0 https://openrepro.org/papers/1812.09970/ 2026-10-18T01:36:21.344Z 2026-10-18T01:36:21.344Z OpenRepro

Does Synthetic Difference-in-Differences (Arkhangelsky et al., 2021) reproduce? Yes, on its headline claims: when OpenRepro re-ran the authors' code, 4 of 4 headline claims held within tolerances registered before results were recorded; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it (checked 18 Oct 2026). Running the authors own Table 1 code (vignettes/paper-results.Rmd) at the synthdid commit named in their session-info, with R 4.0.5 and the pinned MCPanel and glmnet commits, reproduced every entry of Table 1 at the reported precision: SDID -15.6 (8.4), SC -19.6 (9.9), DID -27.3 (17.7), MC -20.2 (11.5), DIFP -11.1 (9.5). Point estimates differ from the paper by at most 0.05, and placebo standard errors by at most 0.05. Deviations were limited to platform (x86_64 Linux instead of an M1 Mac) and one non-estimating build dependency. The paper-era synthdid commit is no longer on any branch of the repository but is still retrievable from GitHub by its hash.

CC BY 4.0