Semi-Supervised Classification with Graph Convolutional Networks

Protocol

Sealed with the plan: primary code is github.com/tkipf/gcn at commit 193c0992007610b8ff65f3d3f5d6c418b13a343b (2017-02-20, 'Fix for tf 1.0.0'; last commit before arXiv v4 of 2017-02-22), gcn/train.py with its default flags (model gcn, 16 hidden units, dropout 0.5, weight decay 5e-4 on the first layer, Adam learning rate 0.01, at most 200 epochs, --early_stopping 10), dataset citeseer, bundled gcn/data/ind.citeseer.* files and the authors' utils.load_data split; earlier versions of the authors' code are not used. Settling test per office/plans/1609.02907-settling-plan.md section 2, re-run with the precision gate registered as a gate: the pinned commit is the only code version; the plan finds no evidence-based earlier-code rescue arm and forbids a calendar/version search (an earlier revision would need a separately registered hypothesis).

Run policy (sealed at event 19, plan hash f2ac9056e39d597f8002ef6ba9aa7f59a1467c45579fcff350659706cfbf4d9a): Settling plan office/plans/1609.02907-settling-plan.md (SHA-256 ce660079b1301ff588f1d6260f1a2e4e2764d0b82f0c70c6a93150143da40d67, bound at event 13), re-run with the precision gate registered as role 'gate'. Deviation from the plan's seed list (event 10): exactly 100 fits, the fresh headline cohort seeds 201..300 inclusive, because seeds 101..200 and the 21-23 replay already have results in arxiv-1609-02907-20261020T031257Z; the replay is not repeated. Fits of the authors' train.py at 193c0992007610b8ff65f3d3f5d6c418b13a343b on citeseer with default flags (--early_stopping 10, arm fresh_default), through the collector /work/settle/train_settle.py (SHA-256 09042e262a5cd9e29ee7186c1228a12e3b13ba001821c203ef6c884d9cf76d91, byte-identical to the previous attempt's; only SETTLE-marked lines and the read-only block after the authors' test evaluation differ, diff at event 18), one fresh Python process and TF graph/session per fit, NumPy and TF seed = the seed, PYTHONHASHSEED=0, OMP_NUM_THREADS=4, TF intra-op 4 / inter-op 1, one fit at a time. Scored job 'fresh' (bash /work/settle/run_phase.sh fresh; run_phase.sh SHA-256 b958d63c4456789649086080e1f84df166fbc39b5489766f70566110b96ced7a, run_fit.sh SHA-256 26066328368fb729e2227e21506d7a4796b44b162d1dd6d5395ff717a95d68d0): seeds 201..300 in order, all fits run regardless of results; then scored job 'analysis' (/work/settle/analyze.py SHA-256 ff848f2f01573e25ad3f7a7a793c12e8214e8782d0b406f1f37b0298ab63b521, Python 3.11.17, NumPy 1.26.4, SciPy 1.13.1). Inputs: the eight gcn/data/ind.citeseer.* files, manifest digest f6d99979e783fbab2987faf2d2d9a03e719561d6e8a17e11f91b9e8bc3450f91 (equal to the plan's, event 12); training env Python 3.6.15 (conda-forge, micromamba 1.5.10), tensorflow 1.0.1 CPU, numpy 1.12.1, scipy 0.19.0, networkx 1.11, protobuf 3.19.6, six 1.17.0, decorator 5.1.1 (pip freeze SHA-256 60c1655ef28d2c0f5ba3aac236efecb9a817dea91816cdef5440380b3d96674a and conda-meta list SHA-256 523f2973c9359c20a1f195b73261344b773b3dc09de199cf71e82fca7f5e2c3c, both equal to the previous attempt's; TF _pywrap_tensorflow.so SHA-256 d1a1cfb4b5248a0aee64b19e58e71ed7d6106f0bf1fa384abf241ae2bbd76292); analysis env freeze SHA-256 5a81cde5d4f409222788b0a0da88fff0acdb160f45ab8839d9661cff6aa10424; sandbox image sha256:5b40012093bab4b3177f8f0ae5280ae9ce6ed2b5381a7b2cd0a8f25f82c4963e on a dedicated s-4vcpu-8gb-intel droplet in tor1 (container capped at 3.5 CPUs). Each fit writes epochs.csv, test_predictions.csv and final.csv to the fresh per-job directory /work/out/fresh/gcn/fresh_default/seedNNN and echoes its full-precision records, stderr, exit code and file digests into the job output. No pilots, no unscored training, no per-epoch test evaluation, no checkpoint selection. Resource cap: 120 wall-minutes from the first fit's start (environment setup excluded); no fit starts after the cap and a fit is killed at it. Job retry: one retry of the training job is allowed only for a documented external interruption (host loss or transient I/O), with identical inputs, seeds and command under a new job name and fresh output directory; both are retained and the first complete invocation is scored. Low accuracy, early stopping, OOM, NaNs or optimizer failure never justify a rerun or another seed. Aggregation: Canonical per-fit accuracy = 100*correct/test_n percentage points, correct = agreement of argmax final logits and argmax original y_test over the original 1,000-node test mask, test_n must be 1000; it must agree with the authors' TF masked accuracy within 1e-6 (extraction consistency check only; both values kept; a failure makes that row invalid). A required row is invalid if its final.csv is missing, duplicated, has the wrong key or row count, a non-zero fit exit, test_n != 1000, a non-finite value, or fails the consistency check; any invalid row makes the comparison evidence-incomplete (no registered value is printed, no survivor mean). Fresh cohort seeds 201-300: mu = unweighted mean of all 100 accuracies, s = SD (ddof=1), SE = s/10; t interval mu +/- t_99(0.975)*SE; bootstrap: RandomState(160902), idx = randint(0, 100, size=(50000, 100)), resample means, 2.5/97.5 percentiles (linear interpolation); final interval [L, H] = [min of lower ends, max of upper ends]. Claim gcn-citeseer-compat (headline, role claim) observed value = min(max(70.3, L), H) (within 0.05 of 70.3 iff [L, H] overlaps [70.25, 70.35]); gate gcn-citeseer-precision (role gate, non-headline) observed value = max(mu - L, H - mu) (met iff <= 0.731). Both are printed by the analysis job on their own lines as 'gcn_citeseer_compat_point <repr>' and 'gcn_citeseer_precision_halfwidth <repr>' and read by the registered key rules. Branch (plan section 5): gate missed -> 'Precision gate fails' (engine outcome inconclusive); gate met and overlap -> 'Compatible, precision gate met'; gate met, no overlap -> 'Precise but disjoint'; invalid evidence -> 'Missing/invalid evidence'. Descriptive only (never a criterion): the per-seed table, runs below 20 and 60 points with their stopping epochs and predicted-class counts, stop-reason counts. No exclusion, clipping or survivor-only mean anywhere. Stopping rule: One pass: the scored 'fresh' job (100 fits) then one scored 'analysis' job; stop at the 120-minute cap, not when a result passes; no seeds are added if the precision gate fails; any changed scientific setting requires a new plan citing this attempt. Reruns: not allowed.

Claims

Gates

A gate is a registered validity or precision condition on the test itself, not a claim about the paper. The role was designated by the agent before the seal, with the rationale shown; the harness does not check that judgement. If a gate is not satisfied or not tested, the outcome is inconclusive, which is not a finding against the paper. A gate is judged on its primary-code result only, and gates never count as consistent or inconsistent claims.

Deviations from the paper's protocol

• minor

  Seed list deviates from the settling plan: the fresh headline cohort is seeds 201..300 inclusive (100 fits, released default stopping), not the plan's 101..200, and the plan's diagnostic replay of seeds 21-23 (six fits) is not run, so this attempt has 100 fits rather than 106.

  Seeds 101..200 and the 21-23 replay already have known results from arxiv-1609-02907-20261020T031257Z, so they would no longer be a prospective cohort; a fresh, unseen seed range keeps the test prospective. Declared before any command output; no other scientific setting changes. event 10

• minor

  CPU execution on a dedicated 4-vCPU droplet (container capped at 3.5 CPUs) with Python 3.6.15, TensorFlow 1.0.1 CPU, NumPy 1.12.1, SciPy 0.19.0, NetworkX 1.11; TF intra-op 4 / inter-op 1 and OMP_NUM_THREADS=4. The paper ran on GPU; the original kernel, RNG and thread configuration are unknown.

  Environment pinned by the settling plan; the thread configuration is a declared new reproducibility setting, not asserted to be the historical one. event 32

• minor

  The pip freeze (SHA-256 60c1655e...) and the conda-meta package list (SHA-256 523f2973...) equal those of arxiv-1609-02907-20261020T031257Z, but the conda-forge base libraries are the builds served on 2026-10-20 and the first attempt (arxiv-1609-02907-20261018T022718Z) never recorded its conda-level list, so the full original environment cannot be proven identical. The sandbox image was rebuilt for this box (sha256:5b400120..., previous attempt sha256:14183b46...).

  The plan asks for the full previous environment pinned; only its recorded part could be pinned. event 33

• minor

  Full per-node logits (test_predictions.csv) and the per-fit epochs.csv/final.csv files were written on the box and are not retained after it is destroyed. The job output retains every fit's unrounded per-epoch train/validation losses and accuracies, per-node predicted and true class strings, the full final row, stderr, exit code, initialization digest, logits digest and SHA-256 of each file; the analysis output retains the per-seed table and summary.

  The engine stores command output as artifacts, not /work files; printing all logits for 100 fits would add several MB. event 34

• minor

  The plan asks for one externally anchored plan before execution. The sealed plan (protocol, claim and gate registrations, run/aggregation/stopping policy with the SHA-256 of the plan document, drivers, input manifest, environment freezes and image) is hash-chained in this log at the seal event, but was not anchored outside OpenRepro before the scored jobs ran.

  No external anchoring step is available before execution in this engine. event 35

• none

  The verifier rederives the claim and gate values from the printed analysis output, not the bootstrap from the 100 rows, so per the plan section 'Seal and execution contract' this is a qualified statistical analysis.

  The harness reads registered values from job output; it does not re-execute the analysis. event 36

• none

  Instrumentation beyond the plan's list: the collector also records the number of hidden units that are zero on every node at the final weights and the graph op count before and after instrumentation.

  Descriptive diagnostics for the collapse question, unchanged from the previous attempt's collector; they fetch existing tensors only and never enter any criterion. event 37

Notes from the attempt

• event 9

  Scope, roles and verdict mapping, declared before any command output. Re-run of the settling test per office/plans/1609.02907-settling-plan.md with the precision gate registered under the engine's role 'gate' (engine branch claim-roles). The previous attempt arxiv-1609-02907-20261020T031257Z registered the gate as a headline claim, so its failed gate produced the engine word 'Partially reproduced' although the plan calls that result inconclusive; this attempt is a new, independent cohort and does not replace or pool with that one. Only the Citeseer cell of Table 2 (70.3) is scored. Registrations: gcn-citeseer-compat (headline, role claim: the final enclosing interval overlaps [70.25, 70.35]); gcn-citeseer-precision (role gate, non-headline: the interval extends at most 0.731 from the mean). Mapping to plan section 5: gate met and compat consistent = 'Compatible, precision gate met'; gate met and compat inconsistent = 'Precise but disjoint' (a remaining pinned-pipeline difference, not a refutation); gate missed = 'Precision gate fails' (engine outcome inconclusive); any missing/invalid required row = 'Missing/invalid evidence' (no claim values printed). No diagnostic claims or controls are registered: the plan's only diagnostic (the seed 21-23 replay) is not repeated here (already done in arxiv-1609-02907-20261020T031257Z), and the plan calls for no other diagnostic criterion; low-run counts and stopping epochs of the fresh cohort are printed descriptively by the analysis only. Fresh seeds are 201..300 rather than the plan's 101..200, because 101..200 already have known results (arxiv-1609-02907-20261020T031257Z); declared here as a deviation.

• event 29

  Sealed judgement, applied mechanically (analysis job event 25): all 100 required rows valid (every fit exit 0, test_n 1000, canonical vs TF accuracy within 1e-6 for every row; largest difference about 3.5e-7). Fresh cohort seeds 201-300: mean 70.372, SD 4.5446, SE 0.4545 percentage points; t_99 interval [69.4703, 71.2737]; bootstrap interval [69.3790, 70.9300]; enclosing final interval [69.3790, 71.2737]. It overlaps [70.25, 70.35] (claim gcn-citeseer-compat consistent), but its maximum distance from the mean is 0.9930 > 0.731 (gate gcn-citeseer-precision missed), so the engine outcome is inconclusive, matching the plan. Plan section 4/5 branch, selected mechanically: 'Precision gate fails': 'The fresh cohort does not estimate the mean precisely enough to settle the disputed gap. All low-accuracy runs remain included.' Seeds were not added.

• event 30

  Fresh cohort detail (descriptive, not a criterion): 1 of 100 runs is low (seed 241: 26.0%, stopped by the released rule after 15 epochs, predicting class 0 for 597 of 1,000 test nodes); no run is below 20%. The other 99 lie between 68.6% and 72.4% (their mean, 70.82, is description only, never an estimate). 91 runs reached the 200-epoch maximum and 9 stopped early (seed 241 at 15 epochs; the other eight at 180-200 epochs). No run had non-finite logits or a dead hidden unit at the final weights. As in arxiv-1609-02907-20261020T031257Z (seeds 101-200: mean 70.256, one low run, gate distance 1.237), a single early-stopped low run widens the bootstrap lower end past the gate; the two cohorts are independent and are not pooled.

• event 31

  Operational note: the fresh job ran 04:03:27-04:21:53 UTC (1106 s), but its job.finished event (22) was logged only at 13:00 UTC, because the operator's polling call (job_status with wait_sec) crashed with IndexError in repro/tools.py _job_status (state[name][0] on an empty sandbox exec result, apparently a transient loss of the connection to the box while the host was idle) and the next poll came about 8.6 hours later. The job output and the log are unaffected (event 22 holds the full output; analysis event 25 read the files the job wrote). The box stayed up for about 9 hours; the cost is in sandbox.down.

Assurance

What the harness enforced for this attempt, and what it predates. A legacy attempt is labelled, never rewritten.

Plan sealed

yes

Sealed before scored execution

yes

Evidence bound to artifact excerpts

yes

Values extracted by a sealed rule

yes

Execution manifest for cited jobs

yes

Semantic re-derivation passed

yes

Observed before the seal

2 shell; provenance captured

Engine

report built by repro 0.2.1; attempt created by repro 0.2.1

Redaction before release

Before release, the operator's local home-directory path was replaced with ~ in 4 places. Every event hash from event 13 on changed with it. The seal event's bytes changed (its chain hash); its plan hash did not. The redaction records the outcome, verdicts and values from before it. That record is an operator assertion, checked for internal consistency against the current log, not proof that the original log was unchanged: the original is not part of the record. It matches what the log derives now.

Rule: operator home directory prefix → ~. Reason: Before release: the operator's local home-directory path, copied from a settling-plan file, was replaced with ~ Recorded at event 43; the head the operator states for the log before the redaction is 42ee07add40c3612e75c05bd347cf87e816c86989358658dd7368a241b9ba2a1 (not checkable from this log; a copy anchored elsewhere or the operator's kept original can check it).

Checked from the record: the redacted log's hash chain and artifact digests; re-derivation of every judgement from it; the redaction event's internal consistency (its position, replacement rows, recomputed digests and plan hashes, a single redaction or a second that names the first); and that the outcome, qualifiers, tallies, assurance flags, verdicts and observed values it records as before equal what the log derives now. Not checked: any other text (metric and protocol wording, units, paths), what files and commands meant when they ran, or anything about the log before the redaction, which is not in the record.

Provenance

Driven by

operator:claude-opus-5-5 (Claude Code subagent, AI-driven)

Sandbox

3.5 CPUs, 6.00 GiB memory, no GPU · image repro-sandbox:0.2

Period

2026-10-20T03:57:06.976Z → 2026-10-20T13:02:25.163Z

Event log

44 events, hash-chained; head 1ef063a357fbc913ab4a383e0b30ce717a5ce0a6051a021d0fcc90d517955917

Verify

python3 openrepro_verify.py arxiv-1609-02907-20261020T035706Z checks the chain and every artifact with the Python standard library alone; repro verify also recomputes every quote match, evidence check and verdict and re-derives this report (engine 0.2.1) (the engine that does this is not yet public; the standalone verifier at /verify/ runs the chain and artifact checks).

Right of reply

Authors may submit a response, which is published alongside this report unedited.