Reproducibility report: Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations

OpenRepro

Scientific machine learning · 2017

Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations

Does PINNs (Raissi et al., 2017) reproduce? Yes, within a broad bound: when OpenRepro re-ran the authors' code, 1 of 1 headline claim held within tolerances registered before results were recorded; Burgers error 3.21× the published value; passes a registered 10× ceiling; a stricter 2× check did not hold; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it (checked 19 Oct 2026).

Maziar Raissi, Paris G. Perdikaris, George Em Karniadakis. 2017. arXiv 1711.10561

Does “PINNs” reproduce?

Reproduced

Yes, within a broad bound. 1 of 1 headline claim held within tolerances registered before results were recorded; Burgers error 3.21× the published value; passes a registered 10× ceiling; a stricter 2× check did not hold; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it.

How far the re-run landed from the published number, in tolerances2 claims re-run, 1 headline

Held (large: headline claim; small: other claim)
Did not hold

Reading the marks

Tail: a one-sided claim (at most, at least); the tail points to the open side, which is shaded faintly

The data behind this chart

Claim	Published	Observed	Accepted if	Deviation ÷ tolerance	Result
A physics-informed network (9 layers x 20 neurons, N_u = 100, N_f = 10,000) solves the Burgers equation with relative L2 error of order 1e-4 (reported 6.7e-4). (headline)	0.00067	0.00214873	at most 0.0067	+0.25	Held
The Burgers relative L2 error is within a factor of 2 of the reported 6.7e-4.	0.00067	0.00214873	at most 0.00134	+2.21	Did not hold

Headline claims are the results the paper is chiefly known for; they decide the outcome. Other claims are reported alongside.

Registration. Claims and tolerances registered at events 8–9; first result recorded at event 18. Registration came before results were recorded, which does not show that no output had been seen (a legacy attempt: no sealed plan). Event numbers refer to the attempt's log.

Checked 19 October 2026.

How this report was made. The re-run was carried out by an AI agent (Claude, made by Anthropic) in a sandbox. Each run is driven by the AI with no person in the loop, and the log's actor field records this. A person oversees the project. No domain expert has reviewed this report. The authors have a right of reply, and their notes can be published alongside it.

What the attempt found

Running the authors Burgers.py (the earliest version in their repository, from five months after arXiv v1) with its fixed seeds on CPU gives a relative L2 error of 2.15e-3 for the continuous-time Burgers example, against 6.7e-4 reported. That meets the pre-registered order-of-magnitude headline check (at most 6.7e-3) but not the stricter within-a-factor-of-2 check. The paper own Table 1 shows errors varying between 1e-4 and 1e-2 across neighbouring settings, and this run used a CPU rather than the reported GPU. The Table 1 sweep was not run.

Assurance

Level: Legacy (what the levels mean)
Plan: Not sealed (legacy): claims were registered before results were recorded, which does not show that no output had been seen
Evidence: Legacy matching: each observed value was matched against the text of the logged output it cites
Execution manifest: No: not recorded for every cited run, or incomplete (no Git checkout, a driver file that could not be hashed)
Report re-derived from the log: Passed: the engine re-derived the report from the log
Engine version: 0.2.1
Predates: plan seal, evidence binding, execution manifest, paper text hash, scored job flag, retry reasons, extraction rule
Qualifiers: Burgers error 3.21× the published value; passes a registered 10× ceiling; a stricter 2× check did not hold; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it

Headline claims

These decide the outcome. Values are as published and as observed by re-running the code; the accepted-if rule was registered before results were recorded. Each row has a ¶ link of its own.

Claim	Published	Observed	Accepted if	Result
A physics-informed network (9 layers x 20 neurons, N_u = 100, N_f = 10,000) solves the Burgers equation with relative L2 error of order 1e-4 (reported 6.7e-4).¶Section 3.1, Fig. 1 caption, p. 8 · registered at event 8 · quote verified on p. 8 · broad bound, observed 3.21× the published value	0.00067relative L2	0.00214873	at most 0.0067	Held

Other claims checked

Reported alongside the headline claims; they do not change the outcome.

Claim	Published	Observed	Accepted if	Result
The Burgers relative L2 error is within a factor of 2 of the reported 6.7e-4.¶Section 3.1, Fig. 1 caption, p. 8 · registered at event 9 · quote verified on p. 8 · broad bound, observed 3.21× the published value	0.00067relative L2	0.00214873	at most 0.00134	Did not hold

Deviations from the paper's setup

minorCPU (shared 4 vCPU droplet, 2 threads) instead of the NVIDIA Titan X GPU used in the paper; Python 3.6.15, TensorFlow 1.8.0, numpy 1.14.5, scipy 1.1.0, matplotlib 2.2.2 (2018-era versions; the repository pins none). Training took 974 s against about 60 s reported.

The run

Wall time: 20 min 3 s, 18 October 2026 to 19 October 2026
Job compute: 17 min 28 s across 1 job
Code the results name: maziarraissi/PINNs at 13ce051
Work tree at the end: maziarraissi/PINNs at 13ce051
Container image: repro-sandbox:0.2 sha256:f4b241036e33…
Sandbox: 3.5 CPUs, 6 GB, no GPU
Run by: Claude (claude-opus-5-5), an AI agent made by Anthropic, in a Claude Code session
Attempt: arxiv-1711-10561-20261018T234638Z
Log: 26 events, head 450dad5c

Evidence

The full evidence report has the exact paper quotes, the method behind every number, the reason for each tolerance, notes made during the attempt and the environment it ran in. The raw report.json is the unmodified data this page is built from.

Log seal · 26 hash-chained events

450dad5c4edbfc2ab0c9dce568dcddd793d19978beba85e4a64f26f6724759da

The head hash of the attempt's append-only log. The engine generated this report from that log; the verifier checks the log and its artifacts against this hash and that the report names it, but not the report's contents. Download the evidence bundle (the log, every artifact and the report; SHA-256 1fd033882f810ccf…) and check it with the verifier, which needs only Python.

Released 19 October 2026. No author notice was sent. Reply status: No courtesy note: no author address is printed in the arXiv version. No reply on record.

Share image (1200×630), drawn from this report.

How to cite this report

OpenRepro (2026). Reproducibility report: Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations. Reproduced; 1 of 1 headline claim held (Burgers error 3.21× the published value; passes a registered 10× ceiling; a stricter 2× check did not hold; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it). Attempt arxiv-1711-10561-20261018T234638Z, log head 450dad5c4edbfc2a, checked 19 Oct 2026. https://openrepro.org/papers/1711.10561/

BibTeX

@techreport{openrepro-1711.10561,
  author      = {{OpenRepro}},
  title       = {Reproducibility report: Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations},
  institution = {OpenRepro},
  year        = {2026},
  url         = {https://openrepro.org/papers/1711.10561/},
  note        = {Attempt arxiv-1711-10561-20261018T234638Z; log head 450dad5c4edbfc2a; checked 19 Oct 2026}
}

To cite the whole ledger, use CITATION.cff on the data page.

Badge

Authors and maintainers may embed this badge; it always reflects the current report, and a correction updates it.

Markdown

[![OpenRepro: Reproduced. 1 of 1 headline claim held; Burgers error 3.21× the published value; passes a registered 10× ceiling; a stricter 2× check did not hold; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it.](https://openrepro.org/badge/1711.10561.svg)](https://openrepro.org/papers/1711.10561/)

HTML

<a href="https://openrepro.org/papers/1711.10561/"><img src="https://openrepro.org/badge/1711.10561.svg" alt="OpenRepro: Reproduced. 1 of 1 headline claim held; Burgers error 3.21× the published value; passes a registered 10× ceiling; a stricter 2× check did not hold; legacy attempt: the plan was never sealed; claims were frozen only by their first result; at least one value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it." width="517" height="20"></a>

Right of reply, and reporting an error

Authors may submit a response, which is published alongside this report unedited. Write to hello@openrepro.org.

Anyone, not only an author, can report an error in an attempt to the same address. The same address takes suggestions of papers to check and questions about the method.