Replication record · arXiv:1810.08473

From Louvain to Leiden: guaranteeing well-connected communities

V. A. Traag, L. Waltman, Nees Jan van Eck · 2019 · doi:10.1038/s41598-019-41695-z · arXiv

Reproduced

Using the authors Java implementation at the release tag matching the paper (networkanalysis 1.0.0) on the SNAP DBLP and Amazon networks, with the paper protocol of 10 random starts of 10 iterations each, all tested Table II values reproduced within 0.0005: DBLP Louvain 0.8267 vs 0.8262 and Leiden 0.8384 vs 0.8387; Amazon Louvain 0.9298 vs 0.9301 and Leiden 0.9340 vs 0.9341. Leiden found higher modularity than Louvain on both networks, as claimed. The other four networks in Table II were out of scope (download blocked, proprietary, or too large for the sandbox), and the badly connected community percentages in Fig. 4 were not tested.

Plan. Registered before results were recorded; not sealed (legacy).

Qualifiers. At least one passing check accepts a band wider than half the published value. The plan was never sealed (legacy attempt): claims were frozen only by their first result. At least one result was matched by the legacy rule rather than bound to an excerpt. At least one scored value was read from an excerpt the agent chose after seeing the output, not by a rule fixed before it. Dblp-leiden-gain: observed 0.938x the published value; passes a registered floor of 0x the published value. Amazon-leiden-gain: observed 1.04x the published value; passes a registered floor of 0x the published value.

Scored (headline): 4 of 4 consistent
Other checks: 2 of 2 consistent
Deviations: 2 (0 material)
Integrity flags: 0
Logged compute: 0.05 h over 7 commands and 1 jobs
Commits the recorded results name: 2d385631: 4 results, named in the result's method text
Work tree when the attempt ended: CWTSLeiden/networkanalysis@2d385631ccba in /work/networkanalysis

Claims

On DBLP, the maximal modularity found by the Leiden algorithm over 10 replications of 10 iterations is 0.8387.headline

Consistent

paper 0.8387observed 0.838377

Paper 0.8387 modularityObserved 0.838377 modularityAccepted if within ±0.002

For each network, Table II reports the maximal modularity obtained using the Louvain and the Leiden algorithm.

Table II, p. 6 (text p. 9) · quote verified on p. 9 · equivalence check · observed/published 0.9996 · registered at event 10, before the first recorded result

Method, tolerance rationale and evidence

Metric. max modularity (gamma = 1) over 10 random starts x 10 iterations, Leiden, SNAP com-DBLP

Why this tolerance. Each value is the best of 10 randomized replications of 10 iterations, reported to 4 decimals; the maximum of 10 random starts moves by roughly 1e-3 between independent sets of runs on networks of this size, so 0.002 allows for that and for rounding.

Event 29. Observed 0.838377, delta -0.000323, consistent. Evidence events [22] (value matched by the legacy rule; not bound to an excerpt). networkanalysis tag 1.0.0 (commit 2d385631) built with the authors release.sh javac/jar commands; RunNetworkClustering -q modularity -r 1 -s 10 -i 10 --seed 20261018 on the SNAP network; maximum over the 10 random starts as printed by the tool, confirmed by an independent numpy modularity computation on the output clustering. (Leiden)

On Amazon, the maximal modularity found by the Leiden algorithm is 0.9341.headline

Consistent

paper 0.9341observed 0.934022

Paper 0.9341 modularityObserved 0.934022 modularityAccepted if within ±0.002

For each network, Table II reports the maximal modularity obtained using the Louvain and the Leiden algorithm.

Table II, p. 6 (text p. 9) · quote verified on p. 9 · equivalence check · observed/published 0.9999 · registered at event 11, before the first recorded result

Method, tolerance rationale and evidence

Metric. max modularity (gamma = 1) over 10 random starts x 10 iterations, Leiden, SNAP com-Amazon

Event 30. Observed 0.934022, delta -7.8e-05, consistent. Evidence events [22] (value matched by the legacy rule; not bound to an excerpt). networkanalysis tag 1.0.0 (commit 2d385631) built with the authors release.sh javac/jar commands; RunNetworkClustering -q modularity -r 1 -s 10 -i 10 --seed 20261018 on the SNAP network; maximum over the 10 random starts as printed by the tool, confirmed by an independent numpy modularity computation on the output clustering. (Leiden)

On DBLP, the maximal modularity found by the Louvain algorithm is 0.8262.

Consistent

paper 0.8262observed 0.826653

Paper 0.8262 modularityObserved 0.826653 modularityAccepted if within ±0.002

For each network, Table II reports the maximal modularity obtained using the Louvain and the Leiden algorithm.

Table II, p. 6 (text p. 9) · quote verified on p. 9 · equivalence check · observed/published 1.001 · registered at event 12, before the first recorded result

Method, tolerance rationale and evidence

Metric. max modularity (gamma = 1) over 10 random starts x 10 iterations, Louvain, SNAP com-DBLP

Event 31. Observed 0.826653, delta +0.000453, consistent. Evidence events [22] (value matched by the legacy rule; not bound to an excerpt). networkanalysis tag 1.0.0 (commit 2d385631) built with the authors release.sh javac/jar commands; RunNetworkClustering -q modularity -r 1 -s 10 -i 10 --seed 20261018 on the SNAP network; maximum over the 10 random starts as printed by the tool, confirmed by an independent numpy modularity computation on the output clustering. (Louvain)

On Amazon, the maximal modularity found by the Louvain algorithm is 0.9301.

Consistent

paper 0.9301observed 0.929846

Paper 0.9301 modularityObserved 0.929846 modularityAccepted if within ±0.002

For each network, Table II reports the maximal modularity obtained using the Louvain and the Leiden algorithm.

Table II, p. 6 (text p. 9) · quote verified on p. 9 · equivalence check · observed/published 0.9997 · registered at event 13, before the first recorded result

Method, tolerance rationale and evidence

Metric. max modularity (gamma = 1) over 10 random starts x 10 iterations, Louvain, SNAP com-Amazon

Event 32. Observed 0.929846, delta -0.000254, consistent. Evidence events [22] (value matched by the legacy rule; not bound to an excerpt). networkanalysis tag 1.0.0 (commit 2d385631) built with the authors release.sh javac/jar commands; RunNetworkClustering -q modularity -r 1 -s 10 -i 10 --seed 20261018 on the SNAP network; maximum over the 10 random starts as printed by the tool, confirmed by an independent numpy modularity computation on the output clustering. (Louvain)

On DBLP, Leiden finds higher maximal modularity than Louvain (paper: +0.0125).headline

Consistent

paper 0.0125observed 0.011724

Paper 0.0125 modularityObserved 0.011724 modularityAccepted if at least 0

For each network, Table II reports the maximal modularity obtained using the Louvain and the Leiden algorithm.

Table II, p. 6 (text p. 9) · quote verified on p. 9 · bound, broad check · observed/published 0.9379 · registered at event 14, before the first recorded result

Method, tolerance rationale and evidence

Metric. Leiden max modularity minus Louvain max modularity on DBLP

Why this tolerance. The paper's comparative claim is that Leiden does at least as well as Louvain; the criterion is a non-negative difference.

Event 33. Observed 0.011724, delta -0.000776, consistent. Evidence events [28] (value matched by the legacy rule; not bound to an excerpt). Difference of the two DBLP maxima from the job output (event 22), computed by gains.py.

On Amazon, Leiden finds higher maximal modularity than Louvain (paper: +0.0040).headline

Consistent

paper 0.004observed 0.004176

Paper 0.004 modularityObserved 0.004176 modularityAccepted if at least 0

For each network, Table II reports the maximal modularity obtained using the Louvain and the Leiden algorithm.

Table II, p. 6 (text p. 9) · quote verified on p. 9 · bound, broad check · observed/published 1.044 · registered at event 15, before the first recorded result

Method, tolerance rationale and evidence

Metric. Leiden max modularity minus Louvain max modularity on Amazon

Why this tolerance. As for DBLP: the criterion is a non-negative difference.

Event 34. Observed 0.004176, delta +0.000176, consistent. Evidence events [28] (value matched by the legacy rule; not bound to an excerpt). Difference of the two Amazon maxima from the job output (event 22), computed by gains.py.

Deviations from the paper's protocol

none
Built and run with OpenJDK 21 instead of a 2018-era JDK.
Current Debian package. The code uses java.util.Random, whose sequence is specified by the Java language and unchanged across versions. event 35
minor
SNAP node ids were renumbered to 0..n-1 in sorted order, and runs used a fixed seed (20261018) rather than an unspecified random one.
The tool requires zero-indexed ids; the paper does not say how ids were mapped or which seeds were used. Both choices affect only which random starts are drawn, not the method. event 36

Notes from the attempt

event 16
Scope, declared before any data is downloaded or any run made: only DBLP and Amazon from Table II are tested. IMDB (sparse.tamu.edu) returned HTTP 403 when checked during candidate selection, Web of Science is proprietary, and LiveJournal (4M nodes) and Web UK (39M nodes) exceed this sandbox (6 GB). The Fig. 4 percentages of badly connected communities are not tested because the repository has no script for that analysis.
event 37
Network sizes match Table II (DBLP 317,080 nodes, mean degree 6.6; Amazon 334,863 nodes). The current SNAP Amazon file gives mean degree 5.53 (925,872 edges), which rounds to 5.5, while Table II lists 5.6. The node count matches exactly, so this is likely rounding or a slightly different edge count in the file version used for the paper; it does not change the comparison.

Assurance

What the harness enforced for this attempt, and what it predates. A legacy attempt is labelled, never rewritten.

Plan sealed: no
Sealed before scored execution: no
Evidence bound to artifact excerpts: no
Values extracted by a sealed rule: no
Execution manifest for cited jobs: no
Semantic re-derivation passed: yes
Predates: a sealed plan before scored execution, evidence bound to an exact artifact excerpt, an execution manifest per job (HEAD, status, diff hash, driver file hashes), the paper text hashed at create, jobs declared scored or unscored, a declared reason for every repeated result, an extraction rule per claim, fixed by the seal
Engine: report built by repro 0.2.1; attempt created by repro 0.1.0

Provenance

Driven by: operator:claude-opus-5-5 (Claude Code session, no human in the loop)
Sandbox: 3.5 CPUs, 6.00 GiB memory, no GPU · image repro-sandbox:0.2
Period: 2026-10-18T01:39:46.321Z → 2026-10-18T01:48:45.899Z
Event log: 42 events, hash-chained; head 578432e9062b72e8e2576db5eeed0bf5ac96e6f5b649204a2e77964f446edc6c
Verify: python3 openrepro_verify.py arxiv-1810-08473-20261018T013946Z checks the chain and every artifact with the Python standard library alone; repro verify also recomputes every quote match, evidence check and verdict and re-derives this report (engine 0.2.1).
Right of reply: Authors may submit a response, which is published alongside this report unedited.