{
  "schema": "repro.report/v3",
  "engine_version": "0.2.1",
  "attempt_id": "arxiv-1610-01271-20261020T030230Z",
  "legacy": false,
  "paper": {
    "arxiv_id": "1610.01271",
    "title": "Generalized random forests",
    "authors": [
      "Susan Carleton Athey",
      "Julie Tibshirani",
      "Stefan Wager"
    ],
    "year": 2019,
    "doi": "10.1214/18-aos1709",
    "venue": "The Annals of Statistics",
    "openalex_id": "https://openalex.org/W2730059782"
  },
  "code": [
    {
      "commit": "a1b8df1be078078f58ed9ccc50a0b40c245b2cca",
      "full_name": "grf-labs/grf",
      "modified_files": 0,
      "path": "/work/grf",
      "url": "https://github.com/grf-labs/grf",
      "matches_primary": true
    }
  ],
  "code_used": [
    {
      "commit": "a1b8df1be078078f58ed9ccc50a0b40c245b2cca",
      "repo": "grf-labs/grf",
      "code_version": "primary",
      "source": "harness",
      "results": [
        33,
        34,
        35,
        36,
        37,
        38,
        39
      ]
    }
  ],
  "primary_code_source": "protocol",
  "outcome": "reproduced",
  "outcome_label": "Reproduced",
  "qualifiers": [
    "bound_checks_only",
    "other_checks_failed",
    "extra_scored_runs"
  ],
  "qualifier_details": [
    "2 scored jobs started after the seal; not the direct source of any recorded value: grf-settle-sim (event 26)"
  ],
  "evidence_gate_failed": [],
  "assurance": {
    "plan_sealed": true,
    "sealed_before_scored_execution": true,
    "evidence_bound": true,
    "extraction_sealed": true,
    "execution_manifest": true,
    "semantic_rederivation_passed": true,
    "predates": [],
    "engine_version": "0.2.1",
    "attempt_engine_version": "0.2.1",
    "setup_outputs_before_seal": 6,
    "observations_before_seal": {
      "shell": 6,
      "job_finished": 0,
      "job_status": 0,
      "read_file": 0
    },
    "observation_provenance": "captured",
    "plan_hash": "74d6a77e47225f4eef33fc116c07980ad7735070094fe0800e7a391d0d3c9fd5",
    "paper_text_sha256_ok": true
  },
  "plan": {
    "event": 25,
    "setup_outputs_observed": 6,
    "observations_before_seal": {
      "shell": 6,
      "job_finished": 0,
      "job_status": 0,
      "read_file": 0
    },
    "observation_provenance": "captured",
    "reruns": "allowed",
    "rerun_policy": "At most one retry per scored job, only after a documented external interruption (host loss or transient I/O) of that job, with identical inputs, seeds and command (only the job name and its fresh output directory change); both attempts are retained and the first complete invocation is scored.",
    "rerun_rule": "prior_job_failed",
    "run_policy": "Exactly 60 paired replications r=1..60 of the cell conf=yes, heterog=yes, p=20, n=800 (n_test=1000), arm P then arm H for each r, one forest fit at a time (job matrix /work/settle/job-matrix.csv sha256 12e2d0cd...). RNGkind Mersenne-Twister / Inversion / Rejection. Per arm: set.seed(161000+r) then generate the training set; set.seed(162000+r) then generate the test set; set.seed(163000+r) then forest seed = runif(1, 0, .Machine$integer.max) (grf 0.9.5's own seed=NULL rule), passed to causal_forest; identical seeds in both arms, so X and N(0,1) noise are matched and W may differ (checked mechanically, not required for W). Both arms: grf 0.9.5 causal_forest(sample.fraction=0.5, mtry=20, num.trees=2000, num.threads=4, min.node.size=NULL, honesty=TRUE, ci.group.size=2, precompute.nuisance=TRUE, alpha=0.05, lambda=0, downweight.penalty=FALSE), two 500-tree OOB nuisance regression forests, predict without variance estimation. Exactly two scored jobs, with the commands in /work/settle/commands.txt (sha256 934a79d6...): grf-settle-sim (Rscript /work/settle/sim.R, sha256 a2c5344d...) then grf-settle-analysis (Rscript /work/settle/analysis.R, sha256 3400b615...). Environment digest /work/settle/env-digest.txt sha256 78e92889...; plan document /work/settle/plan/1610.01271-settling-plan.md sha256 eec2efa4.... No pilots and no unscored simulation or analysis.",
    "aggregation": "Per replication and arm: mse = mean((prediction - tau)^2) over the 1,000 test points against that arm's own test tau, mse_x10 = 10*mse (the only unit conversion), recomputed by the analysis job from the saved prediction files and checked against replicates.csv; 120 rows with unique (arm, replicate), finite values and 1,000 ordered rows per prediction file, else the analysis fails (evidence incomplete). Estimates: P = mean of the 60 P values (the primary number); H = mean of the 60 H values; H-P = mean of the 60 paired differences. Each: SE = sample SD (n-1) / sqrt(60) over its 60 values; q = qt(1 - 0.05/(2*3), 59) (Bonferroni over the three estimates); interval = estimate +/- q*SE. Decisions, in this order: precision gate q*SE_P <= 0.11858505 (else inconclusive even if the interval contains 0.93); compatible iff the P interval intersects [0.925, 0.935] (registered as P_ci_lower <= 0.935 and P_ci_upper >= 0.925); paired excess positive iff the H-P interval lower end > 0. The applicable plan branch is reported verbatim. Display rounding only after judgement.",
    "stopping_rule": "One pass: the simulation job runs once over all 120 fits under a 3600 s wall cap (timeout), then the analysis job runs once. No replications are added after seeing SEs; no seed changes; never average only survivors; exceeding the cap or the $2 cloud cap makes the comparison evidence-incomplete. Low performance, early stopping, OOM, NaN or optimizer failure are not reasons to rerun.",
    "plan_hash": "74d6a77e47225f4eef33fc116c07980ad7735070094fe0800e7a391d0d3c9fd5",
    "claims": [
      "cgrf-p20-n800-p-ci-lower",
      "cgrf-p20-n800-p-ci-upper",
      "cgrf-p20-n800-p-precision-gate",
      "cgrf-p20-n800-hp-diff-ci-lower",
      "report-only-p-mean",
      "report-only-h-mean",
      "report-only-hp-mean-diff"
    ],
    "claims_registered_after_output": []
  },
  "not_attemptable_reason": null,
  "summary": "Sealed settling test of the disputed Table 1 cell (C. GRF, confounding and heterogeneity, p=20, n=800, published 0.93 MSE x 10), pairing the paper's literal DGP (P) with the 2020 aw3 helper (H) under grf 0.9.5, 60 replications with fixed seeds. P mean 1.014, interval [0.905, 1.124] (q*SE 0.109, inside the 0.1186 precision gate), intersects the rounding interval [0.925, 0.935]: compatible with the published point at the stated precision. H mean 1.125, interval [0.980, 1.270]; paired excess H-P 0.111, interval [-0.090, 0.311], not resolved as positive. Plan branch: the literal-paper cell is compatible with the published point, but this experiment does not establish that the helper caused our earlier gap.",
  "claims": [
    {
      "claim_id": "cgrf-p20-n800-p-ci-lower",
      "comparison": "at_most",
      "extraction": {
        "key": "P_ci_lower",
        "occurrence": "only",
        "transform": "none"
      },
      "headline": true,
      "location": "Table 1, p. 23 (row conf.=yes, heterog.=yes, p=20, n=800; column C. GRF)",
      "metric": "P_ci_lower = P mean - q*SE_P. C. GRF test MSE x 10, cell confounding=yes, heterogeneity=yes, p=20, n=800. Per replication r=1..60: 10*mean((prediction - tau)^2) over 1,000 test points, centered causal_forest of grf 0.9.5 with 2,000 trees. Arm P = literal paper DGP (Sec. 6.2: e=(1+dbeta(X3,2,4))/4, m=2X3-1, tau=z(X1)z(X2), Y=m+(W-0.5)tau+eps, no rescaling); arm H = 2020 aw3 helper (dgps.R at e93a79fe, sigmas=1). Interval = estimate +/- q*SE, SE = sample SD/sqrt(60) over replication values (paired differences for H-P), q = qt(1-0.05/(2*3), 59).",
      "paper_value": 0.93,
      "quote": "yes yes 20 800 5.93 8.68 1.92 0.93",
      "statement": "Plan compatibility rule, part 1: the lower end of the literal-paper-DGP (P) interval for C. GRF MSE x 10 at p=20, n=800 is at most 0.935, the top of the rounding interval of the published 0.93.",
      "tolerance": 0.005,
      "tolerance_rationale": "Plan Sec. 3, sealed before execution: P is compatible with the published 0.93 iff the P interval (P mean +/- q*SE) intersects the paper's rounding interval [0.925, 0.935]; equivalently |P mean - 0.93| <= 0.005 + q*SE. The harness cannot apply an SE-dependent band, so the rule is registered as two fixed bounds on the interval ends, each with the 0.005 rounding half-width: lower end <= 0.935 and upper end >= 0.925. Both together are exactly the plan's rule. Intersection is compatibility, not proof of equivalence. The paper's Monte Carlo SE is unavailable, so this is a local compatibility diagnostic, not a calibrated test.",
      "unit": "MSE x 10",
      "registered_event": 5,
      "quote_verified": true,
      "quote_page": 23,
      "preregistered": true,
      "registered_after_output": false,
      "sealed": true,
      "check_kind": "bound",
      "broad_bound": false,
      "tolerance_ratio": 0.005376344086021505,
      "extraction_registered": true,
      "results": [
        {
          "event": 33,
          "observed_value": 0.9046276596195867,
          "delta": -0.02537234038041336,
          "consistent": true,
          "evidence_events": [
            31,
            28
          ],
          "evidence": {
            "artifact": "00031-job-grf-settle-analysis-stdout.txt",
            "event": 31,
            "excerpt": "P_ci_lower 0.90462765961958669",
            "transform": "none"
          },
          "evidence_binding": "bound",
          "evidence_verified": true,
          "evidence_reason": "ok",
          "evidence_after_seal": true,
          "evidence_from_unscored_job": false,
          "evidence_scored_execution": true,
          "extraction": "registered",
          "job_start_event": 30,
          "tree_modified_during_job": false,
          "manifest_complete": true,
          "retry_reason": null,
          "rerun_rule_checked": false,
          "rederivation_problems": [],
          "method": "Sealed plan (event 25): 60 paired replications of the conf=yes, heterog=yes, p=20, n=800 cell under grf 0.9.5 (scored job grf-settle-sim, event 28), then the sealed analysis (scored job grf-settle-analysis, event 31) recomputing every MSE from the saved prediction files; value read by the registered key.",
          "notes": "",
          "code_version": "primary",
          "code_ref": "",
          "code_heads": [
            {
              "commit": "a1b8df1be078078f58ed9ccc50a0b40c245b2cca",
              "full_name": "grf-labs/grf",
              "modified_files": 0,
              "path": "/work/grf",
              "url": "https://github.com/grf-labs/grf"
            }
          ]
        }
      ],
      "withdrawn_after_seal": false,
      "final": {
        "event": 33,
        "observed_value": 0.9046276596195867,
        "delta": -0.02537234038041336,
        "consistent": true,
        "evidence_events": [
          31,
          28
        ],
        "evidence": {
          "artifact": "00031-job-grf-settle-analysis-stdout.txt",
          "event": 31,
          "excerpt": "P_ci_lower 0.90462765961958669",
          "transform": "none"
        },
        "evidence_binding": "bound",
        "evidence_verified": true,
        "evidence_reason": "ok",
        "evidence_after_seal": true,
        "evidence_from_unscored_job": false,
        "evidence_scored_execution": true,
        "extraction": "registered",
        "job_start_event": 30,
        "tree_modified_during_job": false,
        "manifest_complete": true,
        "retry_reason": null,
        "rerun_rule_checked": false,
        "rederivation_problems": [],
        "method": "Sealed plan (event 25): 60 paired replications of the conf=yes, heterog=yes, p=20, n=800 cell under grf 0.9.5 (scored job grf-settle-sim, event 28), then the sealed analysis (scored job grf-settle-analysis, event 31) recomputing every MSE from the saved prediction files; value read by the registered key.",
        "notes": "",
        "code_version": "primary",
        "code_ref": "",
        "code_heads": [
          {
            "commit": "a1b8df1be078078f58ed9ccc50a0b40c245b2cca",
            "full_name": "grf-labs/grf",
            "modified_files": 0,
            "path": "/work/grf",
            "url": "https://github.com/grf-labs/grf"
          }
        ]
      },
      "earlier_final": null,
      "verdict": "consistent",
      "result_revisions": 0,
      "ratio_to_published": 0.972717913569448,
      "optional_stopping": false,
      "evidence_problems": [],
      "evidence_ok": true
    },
    {
      "claim_id": "cgrf-p20-n800-p-ci-upper",
      "comparison": "at_least",
      "extraction": {
        "key": "P_ci_upper",
        "occurrence": "only",
        "transform": "none"
      },
      "headline": true,
      "location": "Table 1, p. 23 (row conf.=yes, heterog.=yes, p=20, n=800; column C. GRF)",
      "metric": "P_ci_upper = P mean + q*SE_P. C. GRF test MSE x 10, cell confounding=yes, heterogeneity=yes, p=20, n=800. Per replication r=1..60: 10*mean((prediction - tau)^2) over 1,000 test points, centered causal_forest of grf 0.9.5 with 2,000 trees. Arm P = literal paper DGP (Sec. 6.2: e=(1+dbeta(X3,2,4))/4, m=2X3-1, tau=z(X1)z(X2), Y=m+(W-0.5)tau+eps, no rescaling); arm H = 2020 aw3 helper (dgps.R at e93a79fe, sigmas=1). Interval = estimate +/- q*SE, SE = sample SD/sqrt(60) over replication values (paired differences for H-P), q = qt(1-0.05/(2*3), 59).",
      "paper_value": 0.93,
      "quote": "yes yes 20 800 5.93 8.68 1.92 0.93",
      "statement": "Plan compatibility rule, part 2: the upper end of the literal-paper-DGP (P) interval for C. GRF MSE x 10 at p=20, n=800 is at least 0.925, the bottom of the rounding interval of the published 0.93.",
      "tolerance": 0.005,
      "tolerance_rationale": "Plan Sec. 3, sealed before execution: P is compatible with the published 0.93 iff the P interval (P mean +/- q*SE) intersects the paper's rounding interval [0.925, 0.935]; equivalently |P mean - 0.93| <= 0.005 + q*SE. The harness cannot apply an SE-dependent band, so the rule is registered as two fixed bounds on the interval ends, each with the 0.005 rounding half-width: lower end <= 0.935 and upper end >= 0.925. Both together are exactly the plan's rule. Intersection is compatibility, not proof of equivalence. The paper's Monte Carlo SE is unavailable, so this is a local compatibility diagnostic, not a calibrated test.",
      "unit": "MSE x 10",
      "registered_event": 6,
      "quote_verified": true,
      "quote_page": 23,
      "preregistered": true,
      "registered_after_output": false,
      "sealed": true,
      "check_kind": "bound",
      "broad_bound": false,
      "tolerance_ratio": 0.005376344086021505,
      "extraction_registered": true,
      "results": [
        {
          "event": 34,
          "observed_value": 1.123609257851831,
          "delta": 0.19360925785183103,
          "consistent": true,
          "evidence_events": [
            31,
            28
          ],
          "evidence": {
            "artifact": "00031-job-grf-settle-analysis-stdout.txt",
            "event": 31,
            "excerpt": "P_ci_upper 1.1236092578518311",
            "transform": "none"
          },
          "evidence_binding": "bound",
          "evidence_verified": true,
          "evidence_reason": "ok",
          "evidence_after_seal": true,
          "evidence_from_unscored_job": false,
          "evidence_scored_execution": true,
          "extraction": "registered",
          "job_start_event": 30,
          "tree_modified_during_job": false,
          "manifest_complete": true,
          "retry_reason": null,
          "rerun_rule_checked": false,
          "rederivation_problems": [],
          "method": "Sealed plan (event 25): 60 paired replications of the conf=yes, heterog=yes, p=20, n=800 cell under grf 0.9.5 (scored job grf-settle-sim, event 28), then the sealed analysis (scored job grf-settle-analysis, event 31) recomputing every MSE from the saved prediction files; value read by the registered key.",
          "notes": "",
          "code_version": "primary",
          "code_ref": "",
          "code_heads": [
            {
              "commit": "a1b8df1be078078f58ed9ccc50a0b40c245b2cca",
              "full_name": "grf-labs/grf",
              "modified_files": 0,
              "path": "/work/grf",
              "url": "https://github.com/grf-labs/grf"
            }
          ]
        }
      ],
      "withdrawn_after_seal": false,
      "final": {
        "event": 34,
        "observed_value": 1.123609257851831,
        "delta": 0.19360925785183103,
        "consistent": true,
        "evidence_events": [
          31,
          28
        ],
        "evidence": {
          "artifact": "00031-job-grf-settle-analysis-stdout.txt",
          "event": 31,
          "excerpt": "P_ci_upper 1.1236092578518311",
          "transform": "none"
        },
        "evidence_binding": "bound",
        "evidence_verified": true,
        "evidence_reason": "ok",
        "evidence_after_seal": true,
        "evidence_from_unscored_job": false,
        "evidence_scored_execution": true,
        "extraction": "registered",
        "job_start_event": 30,
        "tree_modified_during_job": false,
        "manifest_complete": true,
        "retry_reason": null,
        "rerun_rule_checked": false,
        "rederivation_problems": [],
        "method": "Sealed plan (event 25): 60 paired replications of the conf=yes, heterog=yes, p=20, n=800 cell under grf 0.9.5 (scored job grf-settle-sim, event 28), then the sealed analysis (scored job grf-settle-analysis, event 31) recomputing every MSE from the saved prediction files; value read by the registered key.",
        "notes": "",
        "code_version": "primary",
        "code_ref": "",
        "code_heads": [
          {
            "commit": "a1b8df1be078078f58ed9ccc50a0b40c245b2cca",
            "full_name": "grf-labs/grf",
            "modified_files": 0,
            "path": "/work/grf",
            "url": "https://github.com/grf-labs/grf"
          }
        ]
      },
      "earlier_final": null,
      "verdict": "consistent",
      "result_revisions": 0,
      "ratio_to_published": 1.208181997690141,
      "optional_stopping": false,
      "evidence_problems": [],
      "evidence_ok": true
    },
    {
      "claim_id": "cgrf-p20-n800-p-precision-gate",
      "comparison": "at_most",
      "extraction": {
        "key": "P_halfwidth",
        "occurrence": "only",
        "transform": "none"
      },
      "headline": true,
      "location": "Table 1 caption, p. 23",
      "metric": "P_halfwidth = q*SE_P. C. GRF test MSE x 10, cell confounding=yes, heterogeneity=yes, p=20, n=800. Per replication r=1..60: 10*mean((prediction - tau)^2) over 1,000 test points, centered causal_forest of grf 0.9.5 with 2,000 trees. Arm P = literal paper DGP (Sec. 6.2: e=(1+dbeta(X3,2,4))/4, m=2X3-1, tau=z(X1)z(X2), Y=m+(W-0.5)tau+eps, no rescaling); arm H = 2020 aw3 helper (dgps.R at e93a79fe, sigmas=1). Interval = estimate +/- q*SE, SE = sample SD/sqrt(60) over replication values (paired differences for H-P), q = qt(1-0.05/(2*3), 59).",
      "paper_value": 0.11858505,
      "quote": "results are aggregated over 60 simulation replications with 1,000 test points each",
      "statement": "Plan precision gate: the half-width q*SE_P of the P interval is at most 0.11858505, half the previously observed 1.1671701-0.93 gap. paper_value is the plan's gate threshold, not a published number; the quote is the caption's 60-replication design that the gate is about.",
      "tolerance": 0.0,
      "tolerance_rationale": "Plan Sec. 3, prechosen resolution requirement fixed before execution: if the P half-width exceeds 0.11858505 the run cannot resolve the disputed gap and the cell is inconclusive even if the interval contains 0.93. Registered as a headline bound so that an imprecise run can never yield an unqualified positive outcome. Not a tolerance enlarged after results.",
      "unit": "MSE x 10",
      "registered_event": 7,
      "quote_verified": true,
      "quote_page": 23,
      "preregistered": true,
      "registered_after_output": false,
      "sealed": true,
      "check_kind": "bound",
      "broad_bound": false,
      "tolerance_ratio": 0.0,
      "extraction_registered": true,
      "results": [
        {
          "event": 35,
          "observed_value": 0.10949079911612217,
          "delta": -0.00909425088387783,
          "consistent": true,
          "evidence_events": [
            31,
            28
          ],
          "evidence": {
            "artifact": "00031-job-grf-settle-analysis-stdout.txt",
            "event": 31,
            "excerpt": "P_halfwidth 0.10949079911612217",
            "transform": "none"
          },
          "evidence_binding": "bound",
          "evidence_verified": true,
          "evidence_reason": "ok",
          "evidence_after_seal": true,
          "evidence_from_unscored_job": false,
          "evidence_scored_execution": true,
          "extraction": "registered",
          "job_start_event": 30,
          "tree_modified_during_job": false,
          "manifest_complete": true,
          "retry_reason": null,
          "rerun_rule_checked": false,
          "rederivation_problems": [],
          "method": "Sealed plan (event 25): 60 paired replications of the conf=yes, heterog=yes, p=20, n=800 cell under grf 0.9.5 (scored job grf-settle-sim, event 28), then the sealed analysis (scored job grf-settle-analysis, event 31) recomputing every MSE from the saved prediction files; value read by the registered key.",
          "notes": "",
          "code_version": "primary",
          "code_ref": "",
          "code_heads": [
            {
              "commit": "a1b8df1be078078f58ed9ccc50a0b40c245b2cca",
              "full_name": "grf-labs/grf",
              "modified_files": 0,
              "path": "/work/grf",
              "url": "https://github.com/grf-labs/grf"
            }
          ]
        }
      ],
      "withdrawn_after_seal": false,
      "final": {
        "event": 35,
        "observed_value": 0.10949079911612217,
        "delta": -0.00909425088387783,
        "consistent": true,
        "evidence_events": [
          31,
          28
        ],
        "evidence": {
          "artifact": "00031-job-grf-settle-analysis-stdout.txt",
          "event": 31,
          "excerpt": "P_halfwidth 0.10949079911612217",
          "transform": "none"
        },
        "evidence_binding": "bound",
        "evidence_verified": true,
        "evidence_reason": "ok",
        "evidence_after_seal": true,
        "evidence_from_unscored_job": false,
        "evidence_scored_execution": true,
        "extraction": "registered",
        "job_start_event": 30,
        "tree_modified_during_job": false,
        "manifest_complete": true,
        "retry_reason": null,
        "rerun_rule_checked": false,
        "rederivation_problems": [],
        "method": "Sealed plan (event 25): 60 paired replications of the conf=yes, heterog=yes, p=20, n=800 cell under grf 0.9.5 (scored job grf-settle-sim, event 28), then the sealed analysis (scored job grf-settle-analysis, event 31) recomputing every MSE from the saved prediction files; value read by the registered key.",
        "notes": "",
        "code_version": "primary",
        "code_ref": "",
        "code_heads": [
          {
            "commit": "a1b8df1be078078f58ed9ccc50a0b40c245b2cca",
            "full_name": "grf-labs/grf",
            "modified_files": 0,
            "path": "/work/grf",
            "url": "https://github.com/grf-labs/grf"
          }
        ]
      },
      "earlier_final": null,
      "verdict": "consistent",
      "result_revisions": 0,
      "ratio_to_published": 0.923310308644489,
      "optional_stopping": false,
      "evidence_problems": [],
      "evidence_ok": true
    },
    {
      "claim_id": "cgrf-p20-n800-hp-diff-ci-lower",
      "comparison": "at_least",
      "extraction": {
        "key": "HP_ci_lower",
        "occurrence": "only",
        "transform": "none"
      },
      "headline": false,
      "location": "Table 1, p. 23 (row conf.=yes, heterog.=yes, p=20, n=800; column C. GRF)",
      "metric": "HP_ci_lower = mean(H_r - P_r) - q*SE_diff, SE_diff = sample SD of the 60 paired differences / sqrt(60). C. GRF test MSE x 10, cell confounding=yes, heterogeneity=yes, p=20, n=800. Per replication r=1..60: 10*mean((prediction - tau)^2) over 1,000 test points, centered causal_forest of grf 0.9.5 with 2,000 trees. Arm P = literal paper DGP (Sec. 6.2: e=(1+dbeta(X3,2,4))/4, m=2X3-1, tau=z(X1)z(X2), Y=m+(W-0.5)tau+eps, no rescaling); arm H = 2020 aw3 helper (dgps.R at e93a79fe, sigmas=1). Interval = estimate +/- q*SE, SE = sample SD/sqrt(60) over replication values (paired differences for H-P), q = qt(1-0.05/(2*3), 59).",
      "paper_value": 0.0,
      "quote": "yes yes 20 800 5.93 8.68 1.92 0.93",
      "statement": "Plan diagnostic: the paired helper excess H-P (helper DGP minus literal DGP, same seeds) has an interval whose lower end is above 0. paper_value 0 is the plan's sign threshold, not a published number.",
      "tolerance": 0.0,
      "tolerance_rationale": "Plan Sec. 4: the branch 'P compatible; H-P interval positive' requires the H-P interval to lie above 0. The plan says positive; the harness bound is >= 0, which differs only at exactly 0 and the report states the value. Mandatory diagnostic, not a headline candidate.",
      "unit": "MSE x 10",
      "registered_event": 8,
      "quote_verified": true,
      "quote_page": 23,
      "preregistered": true,
      "registered_after_output": false,
      "sealed": true,
      "check_kind": "bound",
      "broad_bound": false,
      "tolerance_ratio": null,
      "extraction_registered": true,
      "results": [
        {
          "event": 36,
          "observed_value": -0.08954917340128875,
          "delta": -0.08954917340128875,
          "consistent": false,
          "evidence_events": [
            31,
            28
          ],
          "evidence": {
            "artifact": "00031-job-grf-settle-analysis-stdout.txt",
            "event": 31,
            "excerpt": "HP_ci_lower -0.089549173401288748",
            "transform": "none"
          },
          "evidence_binding": "bound",
          "evidence_verified": true,
          "evidence_reason": "ok",
          "evidence_after_seal": true,
          "evidence_from_unscored_job": false,
          "evidence_scored_execution": true,
          "extraction": "registered",
          "job_start_event": 30,
          "tree_modified_during_job": false,
          "manifest_complete": true,
          "retry_reason": null,
          "rerun_rule_checked": false,
          "rederivation_problems": [],
          "method": "Sealed plan (event 25): 60 paired replications of the conf=yes, heterog=yes, p=20, n=800 cell under grf 0.9.5 (scored job grf-settle-sim, event 28), then the sealed analysis (scored job grf-settle-analysis, event 31) recomputing every MSE from the saved prediction files; value read by the registered key.",
          "notes": "",
          "code_version": "primary",
          "code_ref": "",
          "code_heads": [
            {
              "commit": "a1b8df1be078078f58ed9ccc50a0b40c245b2cca",
              "full_name": "grf-labs/grf",
              "modified_files": 0,
              "path": "/work/grf",
              "url": "https://github.com/grf-labs/grf"
            }
          ]
        }
      ],
      "withdrawn_after_seal": false,
      "final": {
        "event": 36,
        "observed_value": -0.08954917340128875,
        "delta": -0.08954917340128875,
        "consistent": false,
        "evidence_events": [
          31,
          28
        ],
        "evidence": {
          "artifact": "00031-job-grf-settle-analysis-stdout.txt",
          "event": 31,
          "excerpt": "HP_ci_lower -0.089549173401288748",
          "transform": "none"
        },
        "evidence_binding": "bound",
        "evidence_verified": true,
        "evidence_reason": "ok",
        "evidence_after_seal": true,
        "evidence_from_unscored_job": false,
        "evidence_scored_execution": true,
        "extraction": "registered",
        "job_start_event": 30,
        "tree_modified_during_job": false,
        "manifest_complete": true,
        "retry_reason": null,
        "rerun_rule_checked": false,
        "rederivation_problems": [],
        "method": "Sealed plan (event 25): 60 paired replications of the conf=yes, heterog=yes, p=20, n=800 cell under grf 0.9.5 (scored job grf-settle-sim, event 28), then the sealed analysis (scored job grf-settle-analysis, event 31) recomputing every MSE from the saved prediction files; value read by the registered key.",
        "notes": "",
        "code_version": "primary",
        "code_ref": "",
        "code_heads": [
          {
            "commit": "a1b8df1be078078f58ed9ccc50a0b40c245b2cca",
            "full_name": "grf-labs/grf",
            "modified_files": 0,
            "path": "/work/grf",
            "url": "https://github.com/grf-labs/grf"
          }
        ]
      },
      "earlier_final": null,
      "verdict": "inconsistent",
      "result_revisions": 0,
      "ratio_to_published": null,
      "optional_stopping": false,
      "evidence_problems": [],
      "evidence_ok": true
    },
    {
      "claim_id": "report-only-p-mean",
      "comparison": "within",
      "extraction": {
        "key": "P_mean_mse_x10",
        "occurrence": "only",
        "transform": "none"
      },
      "headline": false,
      "location": "Table 1, p. 23 (row conf.=yes, heterog.=yes, p=20, n=800; column C. GRF)",
      "metric": "P_mean_mse_x10 = mean of 60 P replication values. C. GRF test MSE x 10, cell confounding=yes, heterogeneity=yes, p=20, n=800. Per replication r=1..60: 10*mean((prediction - tau)^2) over 1,000 test points, centered causal_forest of grf 0.9.5 with 2,000 trees. Arm P = literal paper DGP (Sec. 6.2: e=(1+dbeta(X3,2,4))/4, m=2X3-1, tau=z(X1)z(X2), Y=m+(W-0.5)tau+eps, no rescaling); arm H = 2020 aw3 helper (dgps.R at e93a79fe, sigmas=1). Interval = estimate +/- q*SE, SE = sample SD/sqrt(60) over replication values (paired differences for H-P), q = qt(1-0.05/(2*3), 59).",
      "paper_value": 0.93,
      "quote": "yes yes 20 800 5.93 8.68 1.92 0.93",
      "statement": "Reporting only: the primary number, the mean of the 60 P-arm MSE x 10 values (literal paper DGP), for comparison with the published 0.93. Its compatibility is judged only by the two interval-end claims.",
      "tolerance": 1000.0,
      "tolerance_rationale": "REPORTING ONLY. The plan defines no scalar test for this quantity; the band is deliberately vacuous (1000) so the value is still read by a rule sealed before execution and published, while its 'consistent' verdict means nothing. Judgement uses only the plan rules registered as the other claims.",
      "unit": "MSE x 10",
      "registered_event": 9,
      "quote_verified": true,
      "quote_page": 23,
      "preregistered": true,
      "registered_after_output": false,
      "sealed": true,
      "check_kind": "equivalence",
      "broad_bound": true,
      "tolerance_ratio": 1075.268817204301,
      "extraction_registered": true,
      "results": [
        {
          "event": 37,
          "observed_value": 1.0141184587357088,
          "delta": 0.08411845873570878,
          "consistent": true,
          "evidence_events": [
            31,
            28
          ],
          "evidence": {
            "artifact": "00031-job-grf-settle-analysis-stdout.txt",
            "event": 31,
            "excerpt": "P_mean_mse_x10 1.0141184587357088",
            "transform": "none"
          },
          "evidence_binding": "bound",
          "evidence_verified": true,
          "evidence_reason": "ok",
          "evidence_after_seal": true,
          "evidence_from_unscored_job": false,
          "evidence_scored_execution": true,
          "extraction": "registered",
          "job_start_event": 30,
          "tree_modified_during_job": false,
          "manifest_complete": true,
          "retry_reason": null,
          "rerun_rule_checked": false,
          "rederivation_problems": [],
          "method": "Sealed plan (event 25): 60 paired replications of the conf=yes, heterog=yes, p=20, n=800 cell under grf 0.9.5 (scored job grf-settle-sim, event 28), then the sealed analysis (scored job grf-settle-analysis, event 31) recomputing every MSE from the saved prediction files; value read by the registered key.",
          "notes": "",
          "code_version": "primary",
          "code_ref": "",
          "code_heads": [
            {
              "commit": "a1b8df1be078078f58ed9ccc50a0b40c245b2cca",
              "full_name": "grf-labs/grf",
              "modified_files": 0,
              "path": "/work/grf",
              "url": "https://github.com/grf-labs/grf"
            }
          ]
        }
      ],
      "withdrawn_after_seal": false,
      "final": {
        "event": 37,
        "observed_value": 1.0141184587357088,
        "delta": 0.08411845873570878,
        "consistent": true,
        "evidence_events": [
          31,
          28
        ],
        "evidence": {
          "artifact": "00031-job-grf-settle-analysis-stdout.txt",
          "event": 31,
          "excerpt": "P_mean_mse_x10 1.0141184587357088",
          "transform": "none"
        },
        "evidence_binding": "bound",
        "evidence_verified": true,
        "evidence_reason": "ok",
        "evidence_after_seal": true,
        "evidence_from_unscored_job": false,
        "evidence_scored_execution": true,
        "extraction": "registered",
        "job_start_event": 30,
        "tree_modified_during_job": false,
        "manifest_complete": true,
        "retry_reason": null,
        "rerun_rule_checked": false,
        "rederivation_problems": [],
        "method": "Sealed plan (event 25): 60 paired replications of the conf=yes, heterog=yes, p=20, n=800 cell under grf 0.9.5 (scored job grf-settle-sim, event 28), then the sealed analysis (scored job grf-settle-analysis, event 31) recomputing every MSE from the saved prediction files; value read by the registered key.",
        "notes": "",
        "code_version": "primary",
        "code_ref": "",
        "code_heads": [
          {
            "commit": "a1b8df1be078078f58ed9ccc50a0b40c245b2cca",
            "full_name": "grf-labs/grf",
            "modified_files": 0,
            "path": "/work/grf",
            "url": "https://github.com/grf-labs/grf"
          }
        ]
      },
      "earlier_final": null,
      "verdict": "consistent",
      "result_revisions": 0,
      "ratio_to_published": 1.0904499556297944,
      "optional_stopping": false,
      "evidence_problems": [],
      "evidence_ok": true
    },
    {
      "claim_id": "report-only-h-mean",
      "comparison": "within",
      "extraction": {
        "key": "H_mean_mse_x10",
        "occurrence": "only",
        "transform": "none"
      },
      "headline": false,
      "location": "Table 1, p. 23 (row conf.=yes, heterog.=yes, p=20, n=800; column C. GRF)",
      "metric": "H_mean_mse_x10 = mean of 60 H replication values. C. GRF test MSE x 10, cell confounding=yes, heterogeneity=yes, p=20, n=800. Per replication r=1..60: 10*mean((prediction - tau)^2) over 1,000 test points, centered causal_forest of grf 0.9.5 with 2,000 trees. Arm P = literal paper DGP (Sec. 6.2: e=(1+dbeta(X3,2,4))/4, m=2X3-1, tau=z(X1)z(X2), Y=m+(W-0.5)tau+eps, no rescaling); arm H = 2020 aw3 helper (dgps.R at e93a79fe, sigmas=1). Interval = estimate +/- q*SE, SE = sample SD/sqrt(60) over replication values (paired differences for H-P), q = qt(1-0.05/(2*3), 59).",
      "paper_value": 0.93,
      "quote": "yes yes 20 800 5.93 8.68 1.92 0.93",
      "statement": "Reporting only: mean of the 60 H-arm MSE x 10 values (2020 aw3 helper DGP, same seeds and grf 0.9.5), the control; the predecessor's helper value was 1.1671701.",
      "tolerance": 1000.0,
      "tolerance_rationale": "REPORTING ONLY. The plan defines no scalar test for this quantity; the band is deliberately vacuous (1000) so the value is still read by a rule sealed before execution and published, while its 'consistent' verdict means nothing. Judgement uses only the plan rules registered as the other claims.",
      "unit": "MSE x 10",
      "registered_event": 10,
      "quote_verified": true,
      "quote_page": 23,
      "preregistered": true,
      "registered_after_output": false,
      "sealed": true,
      "check_kind": "equivalence",
      "broad_bound": true,
      "tolerance_ratio": 1075.268817204301,
      "extraction_registered": true,
      "results": [
        {
          "event": 38,
          "observed_value": 1.1247302426955974,
          "delta": 0.19473024269559736,
          "consistent": true,
          "evidence_events": [
            31,
            28
          ],
          "evidence": {
            "artifact": "00031-job-grf-settle-analysis-stdout.txt",
            "event": 31,
            "excerpt": "H_mean_mse_x10 1.1247302426955974",
            "transform": "none"
          },
          "evidence_binding": "bound",
          "evidence_verified": true,
          "evidence_reason": "ok",
          "evidence_after_seal": true,
          "evidence_from_unscored_job": false,
          "evidence_scored_execution": true,
          "extraction": "registered",
          "job_start_event": 30,
          "tree_modified_during_job": false,
          "manifest_complete": true,
          "retry_reason": null,
          "rerun_rule_checked": false,
          "rederivation_problems": [],
          "method": "Sealed plan (event 25): 60 paired replications of the conf=yes, heterog=yes, p=20, n=800 cell under grf 0.9.5 (scored job grf-settle-sim, event 28), then the sealed analysis (scored job grf-settle-analysis, event 31) recomputing every MSE from the saved prediction files; value read by the registered key.",
          "notes": "",
          "code_version": "primary",
          "code_ref": "",
          "code_heads": [
            {
              "commit": "a1b8df1be078078f58ed9ccc50a0b40c245b2cca",
              "full_name": "grf-labs/grf",
              "modified_files": 0,
              "path": "/work/grf",
              "url": "https://github.com/grf-labs/grf"
            }
          ]
        }
      ],
      "withdrawn_after_seal": false,
      "final": {
        "event": 38,
        "observed_value": 1.1247302426955974,
        "delta": 0.19473024269559736,
        "consistent": true,
        "evidence_events": [
          31,
          28
        ],
        "evidence": {
          "artifact": "00031-job-grf-settle-analysis-stdout.txt",
          "event": 31,
          "excerpt": "H_mean_mse_x10 1.1247302426955974",
          "transform": "none"
        },
        "evidence_binding": "bound",
        "evidence_verified": true,
        "evidence_reason": "ok",
        "evidence_after_seal": true,
        "evidence_from_unscored_job": false,
        "evidence_scored_execution": true,
        "extraction": "registered",
        "job_start_event": 30,
        "tree_modified_during_job": false,
        "manifest_complete": true,
        "retry_reason": null,
        "rerun_rule_checked": false,
        "rederivation_problems": [],
        "method": "Sealed plan (event 25): 60 paired replications of the conf=yes, heterog=yes, p=20, n=800 cell under grf 0.9.5 (scored job grf-settle-sim, event 28), then the sealed analysis (scored job grf-settle-analysis, event 31) recomputing every MSE from the saved prediction files; value read by the registered key.",
        "notes": "",
        "code_version": "primary",
        "code_ref": "",
        "code_heads": [
          {
            "commit": "a1b8df1be078078f58ed9ccc50a0b40c245b2cca",
            "full_name": "grf-labs/grf",
            "modified_files": 0,
            "path": "/work/grf",
            "url": "https://github.com/grf-labs/grf"
          }
        ]
      },
      "earlier_final": null,
      "verdict": "consistent",
      "result_revisions": 0,
      "ratio_to_published": 1.2093873577372014,
      "optional_stopping": false,
      "evidence_problems": [],
      "evidence_ok": true
    },
    {
      "claim_id": "report-only-hp-mean-diff",
      "comparison": "within",
      "extraction": {
        "key": "HP_mean_diff_x10",
        "occurrence": "only",
        "transform": "none"
      },
      "headline": false,
      "location": "Table 1, p. 23 (row conf.=yes, heterog.=yes, p=20, n=800; column C. GRF)",
      "metric": "HP_mean_diff_x10 = mean of the 60 paired differences H_r - P_r. C. GRF test MSE x 10, cell confounding=yes, heterogeneity=yes, p=20, n=800. Per replication r=1..60: 10*mean((prediction - tau)^2) over 1,000 test points, centered causal_forest of grf 0.9.5 with 2,000 trees. Arm P = literal paper DGP (Sec. 6.2: e=(1+dbeta(X3,2,4))/4, m=2X3-1, tau=z(X1)z(X2), Y=m+(W-0.5)tau+eps, no rescaling); arm H = 2020 aw3 helper (dgps.R at e93a79fe, sigmas=1). Interval = estimate +/- q*SE, SE = sample SD/sqrt(60) over replication values (paired differences for H-P), q = qt(1-0.05/(2*3), 59).",
      "paper_value": 0.0,
      "quote": "yes yes 20 800 5.93 8.68 1.92 0.93",
      "statement": "Reporting only: mean paired difference H-P of MSE x 10 over the 60 replications. paper_value 0 is a reference, not a published number.",
      "tolerance": 1000.0,
      "tolerance_rationale": "REPORTING ONLY. The plan defines no scalar test for this quantity; the band is deliberately vacuous (1000) so the value is still read by a rule sealed before execution and published, while its 'consistent' verdict means nothing. Judgement uses only the plan rules registered as the other claims.",
      "unit": "MSE x 10",
      "registered_event": 11,
      "quote_verified": true,
      "quote_page": 23,
      "preregistered": true,
      "registered_after_output": false,
      "sealed": true,
      "check_kind": "equivalence",
      "broad_bound": true,
      "tolerance_ratio": null,
      "extraction_registered": true,
      "results": [
        {
          "event": 39,
          "observed_value": 0.11061178395988853,
          "delta": 0.11061178395988853,
          "consistent": true,
          "evidence_events": [
            31,
            28
          ],
          "evidence": {
            "artifact": "00031-job-grf-settle-analysis-stdout.txt",
            "event": 31,
            "excerpt": "HP_mean_diff_x10 0.11061178395988853",
            "transform": "none"
          },
          "evidence_binding": "bound",
          "evidence_verified": true,
          "evidence_reason": "ok",
          "evidence_after_seal": true,
          "evidence_from_unscored_job": false,
          "evidence_scored_execution": true,
          "extraction": "registered",
          "job_start_event": 30,
          "tree_modified_during_job": false,
          "manifest_complete": true,
          "retry_reason": null,
          "rerun_rule_checked": false,
          "rederivation_problems": [],
          "method": "Sealed plan (event 25): 60 paired replications of the conf=yes, heterog=yes, p=20, n=800 cell under grf 0.9.5 (scored job grf-settle-sim, event 28), then the sealed analysis (scored job grf-settle-analysis, event 31) recomputing every MSE from the saved prediction files; value read by the registered key.",
          "notes": "",
          "code_version": "primary",
          "code_ref": "",
          "code_heads": [
            {
              "commit": "a1b8df1be078078f58ed9ccc50a0b40c245b2cca",
              "full_name": "grf-labs/grf",
              "modified_files": 0,
              "path": "/work/grf",
              "url": "https://github.com/grf-labs/grf"
            }
          ]
        }
      ],
      "withdrawn_after_seal": false,
      "final": {
        "event": 39,
        "observed_value": 0.11061178395988853,
        "delta": 0.11061178395988853,
        "consistent": true,
        "evidence_events": [
          31,
          28
        ],
        "evidence": {
          "artifact": "00031-job-grf-settle-analysis-stdout.txt",
          "event": 31,
          "excerpt": "HP_mean_diff_x10 0.11061178395988853",
          "transform": "none"
        },
        "evidence_binding": "bound",
        "evidence_verified": true,
        "evidence_reason": "ok",
        "evidence_after_seal": true,
        "evidence_from_unscored_job": false,
        "evidence_scored_execution": true,
        "extraction": "registered",
        "job_start_event": 30,
        "tree_modified_during_job": false,
        "manifest_complete": true,
        "retry_reason": null,
        "rerun_rule_checked": false,
        "rederivation_problems": [],
        "method": "Sealed plan (event 25): 60 paired replications of the conf=yes, heterog=yes, p=20, n=800 cell under grf 0.9.5 (scored job grf-settle-sim, event 28), then the sealed analysis (scored job grf-settle-analysis, event 31) recomputing every MSE from the saved prediction files; value read by the registered key.",
        "notes": "",
        "code_version": "primary",
        "code_ref": "",
        "code_heads": [
          {
            "commit": "a1b8df1be078078f58ed9ccc50a0b40c245b2cca",
            "full_name": "grf-labs/grf",
            "modified_files": 0,
            "path": "/work/grf",
            "url": "https://github.com/grf-labs/grf"
          }
        ]
      },
      "earlier_final": null,
      "verdict": "consistent",
      "result_revisions": 0,
      "ratio_to_published": null,
      "optional_stopping": false,
      "evidence_problems": [],
      "evidence_ok": true
    }
  ],
  "counts": {
    "claims_scored": 3,
    "consistent": 3,
    "consistent_earlier_code": 0,
    "inconsistent": 0,
    "untested": 0,
    "withdrawn_after_seal": 0,
    "other": {
      "claims": 4,
      "consistent": 3,
      "inconsistent": 1,
      "untested": 0
    }
  },
  "withdrawn": [],
  "protocol": {
    "earlier_code": "not_allowed",
    "primary_code": "grf R package 0.9.5 (github.com/grf-labs/grf, tag v0.9.5 = commit a1b8df1be078078f58ed9ccc50a0b40c245b2cca; built from the CRAN archive tarball grf_0.9.5.tar.gz as in the archived job, with the hashed build-only -include cstddef/cstdint flags) in BOTH arms. Arm H data: generate_causal_data(dgp='aw3', sigma.m=1, sigma.tau=1, sigma.noise=1) from r-package/grf/R/dgps.R at commit e93a79fe5e8ca3ac31f5d8af6d6bd63e78684588 (sha256 8be0a4e28f1b28355616ef6f63e7cf8707c5a0c1de2f92854ab54a0d60ed2ec8), sourced alone; the 2020 forest package is not installed. Arm P data: the paper's Sec. 6.2 DGP written literally. R 4.0.2 (Posit build, deb sha256 7786359e2c471666872d2686a487243b6b4d11bdda3eb8fd3cd629959cd06d3c) with the archived job's dependency set from the CRAN snapshot of 2020-09-15.",
    "rationale": "Settling test of the disputed Table 1 cell from the predecessor attempt arxiv-1610-01271-20261018T023619Z (held as partially reproduced; its 0.9.5 fallback gave 1.1671701 vs 0.93). An independent desk check (office/audits/codex-negative-findings-2026-10-19.md, sha256 5b6e553e...) found that run used the 2020 helper DGP (X1 confounding, rescaling, separate train/test normalization), not the paper's literal DGP. This attempt executes the settling plan ~/Desktop/office/plans/1610.01271-settling-plan.md (sha256 eec2efa48ac27ed5bbaae691b696dded8733d6ad6677479dd83cd7ebf0b73363): a paired comparison of the literal DGP (P) and the helper (H) under the same 0.9.5 release, 60 replications, fixed seeds. 0.9.5 is the already-disclosed historical-release hypothesis, not a proven original binary; no version search is allowed, so earlier code is not allowed."
  },
  "deviations": [
    {
      "event": 40,
      "description": "The sandbox base image was rebuilt on a new box (image id sha256:35f8fd46..., Debian 13, gcc 14.2.0-19), not the archived job's image sha256:36b1e94b..., which no longer exists. R 4.0.2 (deb sha256 7786359e..., verified), the dependency set from the 2020-09-15 snapshot (Rcpp 1.0.5, RcppEigen 0.3.3.7.0, DiceKriging 1.5.6, lmtest, sandwich, zoo; Matrix 1.2-18 shipped with R), the grf 0.9.5 tarball and the build flags are identical by hash; the environment is pinned by content digest in /work/settle/env-digest.txt (sha256 78e92889...) before the seal.",
      "impact": "minor",
      "reason": "The archived image was built from a floating base tag on a destroyed droplet and cannot be recovered byte for byte; the plan's environment identity (R build, dependency/build inventory) was recovered by hash."
    },
    {
      "event": 41,
      "description": "grf 0.9.5 was built from the CRAN archive tarball grf_0.9.5.tar.gz (sha256 46ead518..., packaged 2018-01-09), as in the archived job, not from a checkout of commit a1b8df1b (tag v0.9.5, 2018-01-04). Inspection (event 14): causal_forest.R and input_utilities.R differ only in roxygen comments; the mtry default changes affect only standalone regression/quantile/instrumental forests (causal_forest passes mtry to its nuisance forests explicitly); the C++ differs only by trailing newlines, plus the generated RcppExports.cpp.",
      "impact": "none",
      "reason": "Same binary as the archived 0.9.5 fallback; functionally equal to the pinned commit on the causal_forest path."
    },
    {
      "event": 42,
      "description": "Build-only patch: grf 0.9.5 compiled with -include cstddef -include cstdint (Makevars sha256 b243d82c..., the same flags as archived job 00056). No source change.",
      "impact": "none",
      "reason": "GCC 14 no longer includes these headers transitively; the plan allows separately hashed header/include repairs."
    },
    {
      "event": 43,
      "description": "The plan's compatibility tolerance 0.005 + q*SE (SE-dependent) is registered as two fixed bounds on the interval ends (P_ci_lower <= 0.935, P_ci_upper >= 0.925), plus the precision gate as a third headline bound; the H-P sign test is registered as >= 0 (plan: > 0). The harness extracts and judges these values but does not itself rederive SDs, SEs or q from the 120 rows; that computation is the sealed analysis script's (sha256 3400b615...), whose 120 recomputed rows are printed in the cited artifact so anyone can recompute it. Per the plan this is a qualified statistical analysis, not a harness-rederived verdict on the formula. Three report-only claims carry deliberately vacuous bands and their verdicts mean nothing.",
      "impact": "minor",
      "reason": "The engine supports only fixed tolerances; the bound pair is exactly equivalent to the plan's intersection rule."
    },
    {
      "event": 44,
      "description": "Forest seeding: set.seed(163000+r) then forest seed = runif(1, 0, .Machine$integer.max), i.e. grf 0.9.5's own rule for seed = NULL applied to the reset stream (as in the authors' script, which leaves seed NULL), passed explicitly and logged; not causal_forest(seed = 163000 + r) literally.",
      "impact": "none",
      "reason": "The plan specifies R RNG streams reset per role; this reading was fixed before the seal and is identical in both arms."
    },
    {
      "event": 45,
      "description": "The sealed plan is bound in this attempt's hash-chained log (seal event 25, plan hash 74d6a77e...; plan document sha256 eec2efa4... written at event 17) but was not anchored outside the log before execution; external anchoring happens only when the attempt is committed/released.",
      "impact": "minor",
      "reason": "The engine has no pre-execution external anchoring step."
    },
    {
      "event": 46,
      "description": "Dedicated box in DigitalOcean tor1 (s-4vcpu-8gb-intel, the archived job's size) instead of nyc3.",
      "impact": "none",
      "reason": "No 4 vCPU / 8 GB size had capacity in nyc3 at run time."
    }
  ],
  "notes": [
    {
      "event": 47,
      "text": "Correction of the predecessor attempt arxiv-1610-01271-20261018T023619Z, required by the settling plan regardless of the result: “Our earlier GRF comparison used a later helper whose confounding variable, outcome construction and sample normalization differ from the paper. This also affected our 0.9.5 fallback. We withdraw its interpretation as a test of Table 1 as written. Our statement that all four confounding-only cells matched to two decimals was also incorrect; they were within our selected tolerance.” Applicable branch (precision gate passed: half-width 0.109 <= 0.11858505; P interval [0.905, 1.124] intersects [0.925, 0.935]; H-P interval [-0.090, 0.311] includes 0): “The literal-paper cell is compatible with the published point, but this experiment does not establish that the helper caused our earlier gap.” H's actual interval, published as the plan requires: H mean 1.125, interval [0.980, 1.270] (the predecessor's helper value 1.1671701 lies inside it). P mean 1.014 (interval [0.905, 1.124]). Compatibility means interval intersection with the rounding interval, not proof of equivalence; the paper's Monte Carlo SE is unavailable. This one-cell result is not a 12/12 claim; the historical 3/12 and 11/12 counts describe the old helper comparisons only. 0.9.5 is the disclosed historical-release hypothesis, not the authors' proven build."
    },
    {
      "event": 48,
      "text": "Process notes: (1) A setup-only predecessor attempt arxiv-1610-01271-20261020T025027Z was abandoned before any claim, seal or scored job, because its claims would have been registered after setup output; this attempt registered the protocol and all claims (events 4-11) before any command output. (2) Mechanical pairing check (analysis job): X identical in both arms in all 60 replications (train and test), outcome noise matched to max |diff| 1.8e-15, W agreement about 50% (expected: the arms' propensities depend on different coordinates). (3) Simulation wall time 423 s, well inside the 3600 s cap; no retries, no failed invocations. (4) Values are displayed rounded here only after judgement; the cited artifact holds 17 significant digits."
    }
  ],
  "flags": [],
  "compute": {
    "shell_commands": 6,
    "sandbox_seconds": 197.0,
    "jobs": 2,
    "job_seconds": 423.4,
    "sandbox": {
      "container": "repro-arxiv-1610-01271-20261020t030230z",
      "docker_daemon": {
        "mem_bytes": 8326955008,
        "ncpu": 4
      },
      "effective": {
        "cpus": 3.5,
        "memory_bytes": 6442450944
      },
      "image": "repro-sandbox:0.2",
      "image_id": "sha256:35f8fd46ec064e5824277bfca0a82298d0d326cc89afa18ae68b350c205f9163",
      "limits": {
        "cpus": 3.5,
        "gpu": null,
        "memory": "6g",
        "network": true,
        "shm": "2g"
      },
      "target": "cloud",
      "target_detail": {
        "box": "arxiv-1610-01271-20261020t030230z",
        "droplet": "repro-arxiv-1610-01271-20261020-0304",
        "image": "ubuntu-24-04-x64",
        "price_hourly_usd": 0.08333,
        "provider": "digitalocean",
        "region": "tor1",
        "size": "s-4vcpu-8gb-intel"
      },
      "volume": "repro-arxiv-1610-01271-20261020t030230z-work",
      "warnings": []
    },
    "environment_event": 50
  },
  "drivers": [
    "operator:claude-opus-5-5 (Claude Code subagent, AI-driven)"
  ],
  "model_usage": {},
  "model_cost_usd": null,
  "started_at": "2026-10-20T03:02:30.482Z",
  "ended_at": "2026-10-20T03:23:31.892Z",
  "log": {
    "events": 54,
    "head_hash": "6252e1001c1a681990a613fbd32a2e02f351160eff41659fdcafdf705bf1c3b1"
  },
  "right_of_reply": "Authors may submit a response, which is published alongside this report unedited.",
  "redactions": [
    {
      "event": 53,
      "rule": "operator home directory prefix → ~",
      "reason": "Before release: the operator's local home-directory path, copied from a settling-plan file, was replaced with ~",
      "replacements": 4,
      "pre_redaction_head": "fbcb86abd9ba16348600f4b2887cd874fe7b3bad216c20bbf2d5fbaec138cd89",
      "pre_redaction_events": 53,
      "chain_rehashed_from_event": 17,
      "plan_hash_events": [],
      "seal_bytes_changed": true,
      "results_before_match": true,
      "results_before_status": "operator assertion, checked for internal consistency against the current log",
      "disclosure": "Before release, the operator's local home-directory path was replaced with ~ in 4 places. Every event hash from event 17 on changed with it. The seal event's bytes changed (its chain hash); its plan hash did not. The redaction records the outcome, verdicts and values from before it. That record is an operator assertion, checked for internal consistency against the current log, not proof that the original log was unchanged: the original is not part of the record. It matches what the log derives now."
    }
  ]
}