Replication record · arXiv:1711.10561

Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations

Maziar Raissi, Paris G. Perdikaris, George Em Karniadakis · 2017 · arXiv

Reproduced

Running the authors Burgers.py (the earliest version in their repository, from five months after arXiv v1) with its fixed seeds on CPU gives a relative L2 error of 2.15e-3 for the continuous-time Burgers example, against 6.7e-4 reported. That meets the pre-registered order-of-magnitude headline check (at most 6.7e-3) but not the stricter within-a-factor-of-2 check. The paper own Table 1 shows errors varying between 1e-4 and 1e-2 across neighbouring settings, and this run used a CPU rather than the reported GPU. The Table 1 sweep was not run.

Plan. Registered before results were recorded; not sealed (legacy).

Qualifiers. Every scored check is a one-sided bound, not an equivalence to the published number. At least one passing check accepts a band wider than half the published value. Non-headline claims (stricter checks or registered controls) did not hold; the claims list names each. The plan was never sealed (legacy attempt): claims were frozen only by their first result. At least one result was matched by the legacy rule rather than bound to an excerpt. At least one scored value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it. Burgers-l2-order: observed 3.21x the published value; passes a registered 10x ceiling.

Scored (headline): 1 of 1 consistent
Other checks: 0 of 1 consistent
Deviations: 1 (0 material)
Integrity flags: 0
Logged compute: 0.30 h over 3 commands and 1 jobs
Primary code (declared before the first recorded result): github.com/maziarraissi/PINNs at commit 13ce051 (2018-05-01), the earliest commit containing the Burgers code; the repository was created on 2018-01-20, after arXiv v1 (2017-11-28), so no code exists from the paper date
Commits the recorded results name: 13ce051: 2 results, named in the result's method text
Work tree when the attempt ended: maziarraissi/PINNs@13ce051750db in /work/PINNs, 1 file(s) modified locally (see code-diff.patch)

Protocol

Declared before the first recorded result: primary code is github.com/maziarraissi/PINNs at commit 13ce051 (2018-05-01), the earliest commit containing the Burgers code; the repository was created on 2018-01-20, after arXiv v1 (2017-11-28), so no code exists from the paper date; earlier versions of the authors' code are not used. Revision before any result: the declared rule (last commit on or before 2017-11-28) matches no commit. The earliest available code is used as primary, and there is nothing earlier to fall back to.

Claims

A physics-informed network (9 layers x 20 neurons, N_u = 100, N_f = 10,000) solves the Burgers equation with relative L2 error of order 1e-4 (reported 6.7e-4).headline

Consistent

paper 0.00067observed 0.00214873

Paper 0.00067 relative L2Observed 0.00214873 relative L2Accepted if at most 0.0067

The relative L2 error for this case is 6.7· 10−4

Section 3.1, Fig. 1 caption, p. 8 · quote verified on p. 8 · bound, broad check · observed/published 3.207 · registered at event 8, before the first recorded result

Method, tolerance rationale and evidence

Metric. relative L2 error between predicted and exact u(t, x), printed by Burgers.py as Error u

Why this tolerance. The paper's own Table 1 jumps between 6.6e-2 and 6.7e-4 across neighbouring settings, and L-BFGS results differ between CPU and GPU even with the same seed, so the headline check is an order-of-magnitude bound: at most 6.7e-3.

Event 18. Observed 0.00214873, delta +0.001479, consistent. Evidence events [16] (value matched by the legacy rule; not bound to an excerpt). Authors Burgers.py (continuous-time inference) at commit 13ce051, unmodified, with its fixed seeds (numpy and TensorFlow 1234), Python 3.6.15, TensorFlow 1.8.0, scipy 1.1.0 L-BFGS-B, CPU; Error u as printed by the script.

The Burgers relative L2 error is within a factor of 2 of the reported 6.7e-4.

Inconsistent

paper 0.00067observed 0.00214873

Paper 0.00067 relative L2Observed 0.00214873 relative L2Accepted if at most 0.00134

The relative L2 error for this case is 6.7· 10−4

Section 3.1, Fig. 1 caption, p. 8 · quote verified on p. 8 · bound, broad check · observed/published 3.207 · registered at event 9, before the first recorded result

Method, tolerance rationale and evidence

Metric. relative L2 error printed by Burgers.py as Error u

Why this tolerance. A stricter companion check (at most 1.34e-3) to show how close the run gets, given the seed and float sensitivity.

Event 19. Observed 0.00214873, delta +0.001479, inconsistent. Evidence events [16] (value matched by the legacy rule; not bound to an excerpt). Authors Burgers.py (continuous-time inference) at commit 13ce051, unmodified, with its fixed seeds (numpy and TensorFlow 1234), Python 3.6.15, TensorFlow 1.8.0, scipy 1.1.0 L-BFGS-B, CPU; Error u as printed by the script.

Deviations from the paper's protocol

minor
CPU (shared 4 vCPU droplet, 2 threads) instead of the NVIDIA Titan X GPU used in the paper; Python 3.6.15, TensorFlow 1.8.0, numpy 1.14.5, scipy 1.1.0, matplotlib 2.2.2 (2018-era versions; the repository pins none). Training took 974 s against about 60 s reported.
CPU sandbox. L-BFGS trajectories differ between CPU and GPU floating point even with the same seeds, which is why the headline check was registered as an order-of-magnitude bound. event 20

Notes from the attempt

event 21
Code availability: the repository linked in the paper was created on 2018-01-20, about two months after arXiv v1 (2017-11-28), and the Burgers code first appears on 2018-05-01 (commit 13ce051). The code tested therefore postdates the paper version by five months; no earlier code exists.

Assurance

What the harness enforced for this attempt, and what it predates. A legacy attempt is labelled, never rewritten.

Plan sealed: no
Sealed before scored execution: no
Evidence bound to artifact excerpts: no
Values extracted by a sealed rule: no
Execution manifest for cited jobs: no
Semantic re-derivation passed: yes
Predates: a sealed plan before scored execution, evidence bound to an exact artifact excerpt, an execution manifest per job (HEAD, status, diff hash, driver file hashes), the paper text hashed at create, jobs declared scored or unscored, a declared reason for every repeated result, an extraction rule per claim, fixed by the seal
Engine: report built by repro 0.2.1; attempt created by repro 0.1.0

Provenance

Driven by: operator:claude-opus-5-5 (Claude Code session)
Sandbox: 3.5 CPUs, 6.00 GiB memory, no GPU · image repro-sandbox:0.2
Period: 2026-10-18T23:46:39.169Z → 2026-10-19T00:06:41.787Z
Event log: 26 events, hash-chained; head 450dad5c4edbfc2ab0c9dce568dcddd793d19978beba85e4a64f26f6724759da
Verify: python3 openrepro_verify.py arxiv-1711-10561-20261018T234638Z checks the chain and every artifact with the Python standard library alone; repro verify also recomputes every quote match, evidence check and verdict and re-derives this report (engine 0.2.1).
Right of reply: Authors may submit a response, which is published alongside this report unedited.