You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This is a community-detection tuning issue (Louvain producing 275 small clusters) rather than a correctness bug — reproducing/"fixing" it well needs a target: what would good output look like for your repo (rough cluster count, or resolution behavior)? Options I can implement: (a) a resolution parameter so you can tune granularity, (b) a min-cluster-size merge pass that folds singletons/tiny clusters into their nearest neighbor, (c) cap the reported clusters and summarize the long tail. (b)+(c) give the cleanest default. Could you share the repo size (files/modules) and what cluster count you'd consider reasonable? Keeping open pending that so I can calibrate + add a deterministic test. 🙏
Thanks. This is a tuning/quality issue rather than a correctness bug: Louvain community detection produces many small clusters (275 in your case), which is noisy for navigation. Improving it is a parameter/algorithm-tuning task — e.g. a resolution parameter to favor fewer/larger communities, a minimum-cluster-size merge pass, or switching to a modularity threshold that suppresses singletons. Tracking it as a clustering-quality enhancement. If you can share the rough graph size (nodes/edges) and what cluster granularity would actually be useful for your navigation, that'll help me pick sensible defaults. Thanks!
Researched the current science as requested. Conclusion: the scientifically-proven fix is to replace Louvain with the Leiden algorithm.
Leiden — Traag, Waltman & van Eck, "From Louvain to Leiden: guaranteeing well-connected communities", Scientific Reports 9, 5233 (2019), arXiv:1810.08473. It is the established, widely-cited successor to Louvain and directly targets your symptom: Louvain can leave up to ~25% of communities badly connected and ~16% internally disconnected, which is exactly how "275 noisy clusters" arises. Leiden adds a refinement phase that guarantees every community is connected, converges to a partition where all subsets are locally optimally assigned, runs faster than Louvain, and finds better-modularity partitions.
A resolution parameter (γ) is the standard knob to control granularity (lower γ → fewer, larger communities), and Leiden supports it natively (RB/CPM quality functions).
Follow-up work (e.g. arXiv:2402.11454) notes Leiden can still leave rare internally-disconnected communities; a cheap post-pass that splits any disconnected community into its connected components closes that gap.
This is in scope and the right fix — not out of scope. Plan: implement Leiden (local move → refinement → aggregation) to replace the current Louvain pass, expose a resolution parameter to tune cluster granularity, and add a connected-components post-pass as a safety net. That's a focused algorithm change with its own validation (modularity ≥ Louvain on a fixed graph; zero disconnected communities), which I'll do as a dedicated piece rather than rush alongside unrelated work.
Keeping this open as the tracked Leiden implementation. Thanks for pushing for the scientifically-grounded answer.
Thanks for filing this, @fmgoodman 🙏 — the 275-cluster noise was the visible symptom of a deeper problem.
Root cause: the detector ran a single local-moving pass with no aggregation and no refinement. Single-level Louvain leaves most nodes in tiny communities and can even produce internally-disconnected communities — hence the fragmentation.
What I shipped (955b87d): rather than the Rust-specific Cargo.toml-seeding heuristic from the issue (which would only help Rust workspaces), I replaced the algorithm with Leiden (Traag, Waltman & van Eck 2019, arXiv:1810.08473) — the scientifically-proven successor to Louvain that works for every language:
Local moving → refinement → aggregation, repeated until the partition can't be coarsened further (true multi-level, which is what collapses the noise).
Refinement re-derives each community bottom-up from singletons, merging only along edges — this guarantees every reported community is internally connected, which is exactly the defect that produced the noisy split.
Resolution parameter (cbm_leiden(..., resolution, ...)) to tune granularity; cbm_louvain stays as a thin resolution=1.0 wrapper for API compatibility.
Verification: new tests cover multi-level collapse (a 32-node clique chain resolves to a handful of connected communities instead of dozens), resolution control, and a connectivity checker that proves the refinement guarantee. Full suite green (3665 tests), clean under AddressSanitizer/UBSan, zero lint warnings. Shipping in the next release.
Looking forward to 0.7.1 @ 0.7.0 now. Thanks so much for this. I've been using this tool extensively! I appreciate all your hard work and the community bit was no small change!
Activity
DeusData commentedon May 31, 2026
This is a community-detection tuning issue (Louvain producing 275 small clusters) rather than a correctness bug — reproducing/"fixing" it well needs a target: what would good output look like for your repo (rough cluster count, or resolution behavior)? Options I can implement: (a) a resolution parameter so you can tune granularity, (b) a min-cluster-size merge pass that folds singletons/tiny clusters into their nearest neighbor, (c) cap the reported clusters and summarize the long tail. (b)+(c) give the cleanest default. Could you share the repo size (files/modules) and what cluster count you'd consider reasonable? Keeping open pending that so I can calibrate + add a deterministic test. 🙏
DeusData commentedon May 31, 2026
Thanks. This is a tuning/quality issue rather than a correctness bug: Louvain community detection produces many small clusters (275 in your case), which is noisy for navigation. Improving it is a parameter/algorithm-tuning task — e.g. a resolution parameter to favor fewer/larger communities, a minimum-cluster-size merge pass, or switching to a modularity threshold that suppresses singletons. Tracking it as a clustering-quality enhancement. If you can share the rough graph size (nodes/edges) and what cluster granularity would actually be useful for your navigation, that'll help me pick sensible defaults. Thanks!
DeusData commentedon May 31, 2026
Researched the current science as requested. Conclusion: the scientifically-proven fix is to replace Louvain with the Leiden algorithm.
This is in scope and the right fix — not out of scope. Plan: implement Leiden (local move → refinement → aggregation) to replace the current Louvain pass, expose a resolution parameter to tune cluster granularity, and add a connected-components post-pass as a safety net. That's a focused algorithm change with its own validation (modularity ≥ Louvain on a fixed graph; zero disconnected communities), which I'll do as a dedicated piece rather than rush alongside unrelated work.
Keeping this open as the tracked Leiden implementation. Thanks for pushing for the scientifically-grounded answer.
DeusData commentedon May 31, 2026
Thanks for filing this, @fmgoodman 🙏 — the 275-cluster noise was the visible symptom of a deeper problem.
Root cause: the detector ran a single local-moving pass with no aggregation and no refinement. Single-level Louvain leaves most nodes in tiny communities and can even produce internally-disconnected communities — hence the fragmentation.
What I shipped (955b87d): rather than the Rust-specific
Cargo.toml-seeding heuristic from the issue (which would only help Rust workspaces), I replaced the algorithm with Leiden (Traag, Waltman & van Eck 2019, arXiv:1810.08473) — the scientifically-proven successor to Louvain that works for every language:cbm_leiden(..., resolution, ...)) to tune granularity;cbm_louvainstays as a thinresolution=1.0wrapper for API compatibility.Verification: new tests cover multi-level collapse (a 32-node clique chain resolves to a handful of connected communities instead of dozens), resolution control, and a connectivity checker that proves the refinement guarantee. Full suite green (3665 tests), clean under AddressSanitizer/UBSan, zero lint warnings. Shipping in the next release.
fmgoodman commentedon May 31, 2026
thank you! Which release will this hit?
feat(arch): replace single-level Louvain with multi-level Leiden comm…
DeusData commentedon Jun 1, 2026
@fmgoodman likely 0.7.1
fmgoodman commentedon Jun 1, 2026
Looking forward to 0.7.1 @ 0.7.0 now. Thanks so much for this. I've been using this tool extensively! I appreciate all your hard work and the community bit was no small change!