7 papers, by field
Each page answers whether the paper reproduces, lists every claim checked with its published and observed value, and links to the evidence report and raw data.
Economics
- Synthetic Difference-in-DifferencesDmitry Arkhangelsky, Susan Carleton Athey, David A. Hirshberg, Guido W. Imbens, Stefan Wager. American Economic Review, 2021. arXiv 1812.09970Reproduced4 of 4 headline claims held; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it
Epidemiology
- Inferring change points in the spread of COVID-19 reveals the effectiveness of interventionsJonas Dehning, Johannes Zierenberg, F. Paul Spitzner, Michael Wibral, João Pinheiro Neto, Michael Wilczek, Viola Priesemann. Science, 2020. arXiv 2004.01105Reproduced4 of 4 headline claims held; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it
Medical statistics
- DeepSurv: personalized treatment recommender system using a Cox proportional hazards deep neural networkJared Katzman, Uri Shaham, Alexander Cloninger, Jonathan Bates, Tingting Jiang, Yuval Kluger. BMC Medical Research Methodology, 2018. arXiv 1606.00931Reproduced4 of 4 headline claims held; some claims were registered after command output had been observed; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it
Natural language processing
- Bag of Tricks for Efficient Text ClassificationArmand Joulin, Édouard Grave, Piotr Bojanowski, Tomáš Mikolov. Conference of the European Chapter of the Association for Computational Linguistics, 2017. arXiv 1607.01759Reproduced8 of 8 headline claims held; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it
Network science
- From Louvain to Leiden: guaranteeing well-connected communitiesV. A. Traag, L. Waltman, Nees Jan van Eck. Scientific Reports, 2019. arXiv 1810.08473Reproduced4 of 4 headline claims held; DBLP Leiden gain over Louvain 0.938× the published value; passes a registered floor of 0, a sign check; Amazon Leiden gain over Louvain 1.04× the published value; passes a registered floor of 0, a sign check; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it
Recommender systems
- Neural Collaborative FilteringXiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, Tat-Seng Chua. arXiv, 2017. arXiv 1708.05031Reproduced4 of 4 headline claims held; some claims were registered after command output had been observed; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it
Scientific machine learning
- Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential EquationsMaziar Raissi, Paris G. Perdikaris, George Em Karniadakis. arXiv, 2017. arXiv 1711.10561Reproduced1 of 1 headline claim held; Burgers error 3.21× the published value; passes a registered 10× ceiling; a stricter 2× check did not hold; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it