Skip to content

feat(resolver): strict shared cross-file resolver with per-language static rules - #2886

Open
DeusData wants to merge 206 commits into
mainfrom
feat/strict-shared-resolver
Open

DeusData wants to merge 206 commits into
mainfrom
feat/strict-shared-resolver

Conversation

@DeusData

@DeusData DeusData commented Nov 2, 2026 •

Copy link
Copy Markdown
Owner

What this does

IMPORTS, CALLS, CALL_REFERENCE, USAGE, READS, WRITES and USES_TYPE edges are now published only when a language's own static rules prove the target. cbm_shared_resolver_project deletes those edges and re-inserts only admitted proofs. Unique names, suffix matches, confidence numbers and lsp_ prefixes no longer count as proof. A false edge is a bug; a missing edge is debt; the resolver abstains rather than guess. DEPLOYS and INHERITS behaviour is unchanged. CALLS means the declaration static rules select, not runtime dispatch.

Language rules added:

Languages Rules
Go Package scope, embedding and method sets, type aliases, bare results, literals, conversions
Rust Cargo crates, module trees, use trees, re-exports, globs, trait and inherent method selection, autoderef
Python Scopes, C3 member order, decorators that keep identity, closed star imports, search paths
JS, TS, Vue, Svelte ES modules, CommonJS, export surfaces, lexical and type scopes, declared receivers
C, C++ Translation-unit visibility, include resolution, prototypes, attribute macros, members, namespaces
Java, Kotlin, C# Lexical scopes, imports, inherited members, arity and known-type overload selection
PHP, Elixir, Erlang, Perl, Ruby, Zig, Haskell, OCaml, Dart, Swift, Scala Per-occurrence source adapters in internal/cbm/resolve/adapter_*.c

Resolver evidence is stored as integer rows behind read-only views. The semantic index version moves to 13, so existing indexes rebuild once.

Read this first: what gets worse

1. Raw edge counts fall in most repositories. Baseline counts include edges bound by name alone, so a lower count is not simply a loss, but real recall is lost too: USAGE on Elixir, Erlang and Ruby goes to zero because those adapters decide calls only. Full table below.

2. One changed file now costs a full rebuild. Design stage 5 (incremental optimization) is not implemented. cbm_store_resolver_requires_full forces a full index on any change, because a change can invalidate another file's negative proof. An unchanged tree is still a cheap no-op. Measured, seconds, under load:

Repo Full, base → new One file changed, base → new Unchanged, base → new
expressjs/express 1.2 → 3.3 1.2 → 3.2 1.2 → 2.0
meilisearch 3.8 → 10.1 2.2 → 10.8 1.0 → 0.9
redis 6.0 → 16.1 2.7 → 15.5 0.9 → 0.6
django 9.3 → 65.4 9.3 → 91.7 1.2 → 1.2

3. Indexing is slower and databases are larger, typically 2 to 3 times on both, and up to 9 times the time on kilocode. Tables below.

4. This changes the global shape. The maintainer's standing rule keeps suppressors per language. This deletes and re-proves globally. That was a deliberate decision for this change, with the recall loss disclosed here.

Quality

Frozen oracle

291 positive and 42 negative hand-checked sites across 24 pinned repositories, frozen before the language rules were written. The oracle is not fully blind: an earlier Go review exposed part of it. Scored on the repositories both legs indexed, reported as aggregates only.

Binary 4a514ea9; the only production change after it is one assignment.

Leg, 23 repositories Found Wrong target Miss Indeterminate Precision Recall on indexed sites Negative sites held
Baseline main a6d0d6c 102 27 130 20 79.1% 36.6% 30 of 40, 8 violated
This branch 174 1 99 5 99.4% 62.4% 38 of 40, 2 violated
Relation Sites Found, base → new Wrong, base → new
IMPORTS 81 47 → 57 10 → 1
CALLS 98 47 → 75 11 → 0
USAGE 41 7 → 21 4 → 0
USES_TYPE 59 0 → 15 0
CALL_REFERENCE 3 0 → 2 0
WRITES 7 1 → 4 2 → 0
READS 2 0 0

The site is the unit here, so this measures exact occurrences, which the raw counts below cannot. Linux is not in either row; see Evidence limits. I did not open the one wrong site or the two violated negatives, so the oracle stays usable for the next measurement. They should be looked at.

Found per repository, base → new. Four repositories got worse or stayed at zero, and they are real gaps: Alamofire 1 → 0, bloc 5 → 2, llvm 7 → 6, neovim 1 → 0. Swift and Dart misses are mostly unproven imports.

Kilo 11 → 14 of 14 TypeScript 7 → 13 of 13 django 4 → 14 of 15 kubernetes 4 → 12 of 14
elasticsearch 5 → 11 of 13 redis 10 → 11 of 12 express 8 → 11 of 14 meilisearch 3 → 9 of 12
plug 4 → 9 of 12 koel 7 → 9 of 15 Exposed 5 → 8 of 14 mojo 2 → 8 of 11
cowboy 0 → 8 of 10 dune 0 → 6 of 10 PostgREST 3 → 5 of 12 play 1 → 3 of 10
sinatra 2 → 3 of 9 zls 10 → 10 of 12 dotnet/runtime 2 → 2 of 13

Real repositories, baseline main a6d0d6c vs this branch

Repo CALLS USAGE IMPORTS USES_TYPE Index s DB MB
Alamofire (Swift) 5,453 → 1,586 9,534 → 715 52,810 → 0 0 → 33 2.2 → 6.7 70 → 89
Exposed (Kotlin) 27,625 → 20,357 41,493 → 19,195 137,124 → 5,404 0 6.6 → 23.2 236 → 510
PostgREST (Haskell) 1,436 → 1,017 3,341 → 503 496 → 78 0 1.1 → 1.7 18 → 33
django (Python) 63,691 → 48,172 40,215 → 30,064 15,562 → 15,447 0 → 5 6.5 → 44.5 374 → 922
plug (Elixir) 971 → 861 472 → 0 42 → 81 0 0.9 → 3.1 7 → 13
express (JS) 600 → 198 312 → 167 177 → 160 0 0.9 → 2.2 6 → 25
bloc (Dart) 2,068 → 1,926 4,129 → 1,361 967 → 860 0 → 29 1.4 → 3.7 41 → 75
koel (PHP) 10,546 → 5,076 15,720 → 6,944 7,843 → 6,106 0 → 1,508 1.9 → 6.8 74 → 142
meilisearch (Rust) 25,112 → 16,579 37,804 → 23,346 3,314 → 6,814 0 → 7,863 2.7 → 7.1 122 → 198
mojo (Perl) 1,967 → 1,425 84 → 27 4 → 435 0 1.1 → 1.9 15 → 29
neovim (C) 55,227 → 29,608 95,509 → 53,792 8,450 → 6,660 0 8.1 → 23.1 247 → 647
cowboy (Erlang) 4,771 → 5,691 4,975 → 0 204 → 55 0 1.2 → 1.7 32 → 43
dune (OCaml) 15,524 → 5,239 19,282 → 2,048 91 → 14 0 3.2 → 4.5 94 → 112
play (Java, Scala) 24,686 → 11,069 30,444 → 10,428 4,018 → 2,121 0 → 4,224 3.8 → 8.6 152 → 237
redis (C) 34,995 → 28,732 49,206 → 42,224 2,302 → 1,715 0 → 4 4.0 → 12.7 181 → 398
sinatra (Ruby) 2,106 → 555 359 → 0 126 → 108 0 0.9 → 1.1 11 → 14
zls (Zig) 4,898 → 2,401 8,728 → 2,783 259 → 229 0 → 8 0.9 → 1.7 22 → 36

Huge repositories

Repo CALLS USAGE IMPORTS USES_TYPE Index s DB GB
kubernetes 221,251 → 257,502 576,608 → 534,811 57,334 → 57,197 0 → 210,425 43 → 140 2.2 → 4.2
elasticsearch 1,457,475 → 1,308,131 1,585,096 → 1,049,503 438,469 → 321,329 0 → 404,949 180 → 301 5.3 → 9.6
dotnet/runtime 1,035,361 → 392,440 2,368,274 → 709,951 124,504 → 117,615 0 → 40,463 200 → 511 6.3 → 10.8
llvm-project 969,997 → 432,362 1,669,132 → 586,127 166,332 → 128,093 0 → 789 242 → 508 6.9 → 11.5
TypeScript 80,732 → 47,952 372,439 → 16,004 26,166 → 19,156 0 → 20,724 62 → 151 1.1 → 3.6
kilocode 110,797 → 74,589 240,939 → 51,675 59,050 → 31,936 0 → 23,491 14 → 126 0.7 → 2.0

Times are under varying load, so read them as orders of magnitude.

Precision sampling

Each language lane read emitted edges from real repositories and adjudicated them against source, for example kubernetes 208 read, 0 wrong; django 229 and 76 read, 0 wrong; express 165 read, 0 wrong. Those samples predate the final merge. The oracle above is the check on the final binary. See its results above.

Verification

All results are on f6556b7b unless a row says otherwise. The one later commit, d26e6463, stops writing an unread property on C file nodes. It was verified with the affected suites (1,425 passed) and leaves edge counts on redis and neovim identical; hosted CI runs the whole matrix on it.

Gate Result
macOS ASan + UBSan, whole suite 10,893 passed, 0 failed, 12 skipped, 0 sanitizer reports
Linux arm64 GCC ASan + LeakSanitizer 10,731 passed, 0 failed, 11 skipped, 220 suites
Linux arm64 TSan 1,569 passed, 8 skipped, no reports, at e821b3e0. The only later commit touches tests.
Windows native ARM64, every suite, sequential, one process 10,705 passed, 1 failed, 99 skipped. The failure, mem_map_attributes_a_known_allocation, measures committed-memory growth; 200 suites into one process that memory is reused. The mem suite alone passes 73 of 73.
Forced spill, CBM_MEM_SPILL=1, 62 resolver suites, ASan per suite 3,310 passed, 0 failed, at 415d9af2. Later production edits are one local rename and one assignment.
Linux lint: cppcheck 2.20.0, clang-format 20, NOLINT, memory core Passed
Security audit, scripts/security-audit.sh Passed
Linux amd64 Not run locally. Hosted CI covers it, and any red there gets fixed before merge.
Windows guards, win.sh guards Not run locally. Hosted CI covers it.

On this VM the parallel Windows runner oversubscribes: heavy suites hit the 900 s cap, and it then stops before finishing. pipeline and test_impact_git, the two it capped most recently, pass together when run alone (503 passed, about 8 minutes). The sequential run above is the established Windows leg here. test_impact_tree failed once in that parallel run, with its log lost to cleanup; it passed in the sequential run.

RED on revert was run on the final tree for the spill lifetime fix (the same stack as the crash), the four Go rules (8 tests fail), and PHP, the source adapters and Python members together (22 tests fail). RED runs for the other rules happened during development and were not repeated on the final tree.

Bugs found only by running everything together

Each language lane was green alone. These appeared after integration, and each has a test:

  • A Python class header read a sibling declared below it, giving a false INHERITS.
  • Two nested same-named TypeScript consts bound through the Go variant rule, giving a false CALLS.
  • Over the memory budget, the Java, Kotlin and C# declaration index read qualified names from a freed result. TypeScript's merged label did the same. This is use-after-free in production on a large repository under pressure.
  • The HTTP default audit was quadratic in top-level statements and ran serially. TypeScript took over 28 minutes; it now takes about 150 s.
  • The parallel route skipped the Java, Kotlin and C# walk on files with no calls, and lost their field and type rows.
  • Calls inside a C# operator were attributed to the file. Operators now have nodes.
  • Go cache[k] = v emitted WRITES for cache.
  • A generated header with 70,000 includes gave one File node a 9 MB property nothing read, and a database 13 times the baseline's. Found by the hosted diagnostic lane, whose strict_string_checks made the FTS backfill quadratic on it.
  • A C# doc scope read the repaired parse view's tree, so a reference valid in another #if branch read as missing.

Decisions a reviewer may want to overrule

  • Go annotation tokens are USES_TYPE only, like TypeScript. On kubernetes pkg/kubelet this removes at most 3,596 of 18,570 Go USAGE edges (19%); USES_TYPE keeps those pairs. To flip: remove the two is_go_annotation_type calls in extract_usages.c and adjust es_usage_crossfile_go.
  • Kotlin ::x inside if, when, ?: or a to pair is USAGE; direct forms are CALL_REFERENCE. This matches the Java ternary and the extractor's documented rule.
  • A constructor call with no constructor node targets the class for Java, Kotlin, C#, Scala and Swift. For Swift Widget(name:) this is the weakest edge here: init is a real declaration with no node, so calls inside an init body likely go to the file too.
  • Kotlin nested class QNs changed to include the outer class, and unnamed companions now have nodes. A nested class sharing a name with a member function now collides: on Exposed, 1 doc target lost and 66 gained.
  • Java, Kotlin and C# judgement calls: an unknown-type argument keeps the only candidate of that arity; a type-shaped name is assumed not to be a field of an unindexed supertype; external tables are closed.
  • C# reference assemblies abstain. A ref/ type beside its implementation shares a QN, and nothing says which one a caller compiles against. This is likely the largest C# recall loss on dotnet/runtime.

Known limits

  • USAGE on a type name is inconsistent across languages: JS and TS enforce value-only, others emit it on signatures.
  • Field initializer edges have a File source in Java and a Module source in Kotlin and C#.
  • Go arr[i] = x reads as USAGE, a missing write rather than a false edge.
  • C# conversion operators, property accessors and indexers still have no node.
  • Haskell writes one occurrence row per branch view; the graph edge is unaffected.
  • Over-budget indexing aborts by design. Running a huge index beside an in-process test runner shrinks the budget enough to fail it.

Removed tests

29 test names on main no longer exist on this branch. A ledger recorded at an earlier tree classifies them: 5 covered elsewhere, 16 that dropped a genuine positive edge, 8 that dropped none, plus 2 partial holes and 3 weakened tests. Most are name-fallback tests in tests/test_registry.c. I have not re-checked how many of the 16 the language rules have since restored.

Evidence limits

  • The sealed Linux tree has 13 case-collapsed paths on APFS, all with identical content, and one local modification, arch/alpha/boot/bootp.c.
  • torvalds/linux has not been indexed with this branch's final binary. Its database is estimated near 23 GB and the laptop did not have room to do it safely alongside everything else; an earlier attempt was stopped by the swap guard. The largest completed run is llvm-project.
  • The formal controller run prepared for this change was not executed, because it needs a session able to run git against its build roots. The census and oracle above are the evidence.
  • Some commit messages are inaccurate: d38554e is labelled WIP but is final; 4b4b77a carries older text; 456bdbb says about 16 and the true number is 11; ad60078 names the wrong rename commit, corrected in 9a5f62e.

Open questions

  • Extend #field QNs to Rust?
  • Should enum variants get nodes?
  • Should framework knowledge such as Django's X.objects.y be modelled?
  • Should design stage 5 land before this ships, given the one-file rebuild cost?

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…ared-evidence-resolver-recovered

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Resolves the nine textual conflicts with the reviewed reconciliation: semantic index version 12 on top of main's migrations, digest stream v3 because both sides independently claimed v2, incremental metadata-current requires main completeness and the strict resolver generation receipt, and exact call joins by extraction index, full span and origin. Unresolved-call diagnostics reconcile only after final proof sealing and successful projection.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Adds a class-filtered, thread-local, one-shot realloc failure hook compiled only with CBM_MEM_TEST_API, and a regression that fails the primary edge array's first growth. The old code registered the secondary and dedup entries and returned a positive id while the array the dump reads stayed empty, so a retry appeared to succeed and the edge was lost and leaked.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…grow

cbm_gbuf_insert_edge ignored a failed cbm_da_push, registered the secondary and dedup indexes, and returned a positive id. Strict projection treats that as published even though the dump reads only the primary array, and diagnostic reconciliation could then hide the miss. Compare the count around the append and on failure free the edge and return 0 before any index registration. Existing strict projection already treats 0 as failure.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
… the merge

The reviewed reconciliation of tests/test_pipeline.c kept main's call sites of cross_file_call_has_strategy but dropped its definition, and test_resolver_integration.c used a bare yyjson.h that no include path provides. Neither candidate had been compiled.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Threads main's Go rel_path (#1910) through the proof-source and import-snapshot variants, keeps main's ADR sidecar so the rebuild no longer re-applies ADRs, and keeps the feature's removal of the import name fallback and JVM tail join, inside which main had added a generated-member filter. Main took semantic index version 12 for Markdown Section names, so the resolver generation becomes version 13.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Shape of redis src/defrag.c:1234: an #ifdef splitting a long else-if chain
puts the first syntax error more than 16 levels under the root. Today that
fails the whole file before its include table exists, and the next declared
binding dereferences it, so the indexing worker dies with SIGSEGV.

RED: SIGSEGV in c_proof_direct_class.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The strict resolver's syntax audit shared CBM_LSP_MAX_LOOKUP_DEPTH (16), a limit
meant for registry chains. It tripped on any node 17 levels down, even a clean
one under an error-bearing ancestor, and an #ifdef inside an else-if chain gets
there in ordinary C. That set binding_failed for the whole file.

c_proof_prepare then returned before allocating proof_includes, with include_count
already set, and processing carried on: c_proof_type_visible dereferenced NULL.
On redis the worker crashed 10 times; each crash quarantined files that happened to
be in flight on other threads (src/server.c, src/module.c, src/config.c), so the
failure status and the set of missing files changed from run to run.

Walk the audit with a tree cursor instead of recursing, so nesting depth is not a
property the answer can depend on. The result is what proof_syntax_limit already
documents: sites before the first error keep their proofs, sites at or after it
do not. The fail-closed behaviour is unchanged for every site past the error, and
the walk is linear in the error-bearing spine, so the depth and visit guards had
nothing left to bound.

c_proof_type_visible also refuses when binding_failed is set, because a failed
prepare can still leave the table unset on allocation failure.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Each route is one way such bytes reach a strict JSON writer before publication:
a call argument (redis deps/tre/tests/retest.c, neovim's legacy specs), a callee
spelling, and an import path. The parallel leg also checks the argument text
survives on the published edge as U+FFFD.

RED: cbm_pipeline_run returns CBM_PIPELINE_ABORT_PRESERVE_DB or
CBM_PIPELINE_PERSIST_FAILED, depending on the route.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
yyjson's strict writer refuses a whole document over one undecodable byte. Source
files can be in any encoding, and the strict resolver puts source text in JSON on
its way to publication, treating a NULL write as fatal:

  record_properties    call arguments and callee    -> aborted_previous_preserved
  fact_json            proof lookup keys and scopes -> persist_failed
  pass_pkgmap          import paths and names       -> aborted_previous_preserved
  lsp_surface          definitions and imports      -> every file's surface row dropped

Neovim and redis took the first route: a latin-1 string argument in a test file
stopped the whole index. Main writes these same bytes into edge properties raw, so it
published, with invalid UTF-8 inside that JSON.

Add cbm_json_write_text: valid documents are written unchanged at no extra cost,
and only a document the writer refused has its strings repaired, replacing undecodable
bytes with U+FFFD. That covers every field of a document, rather than each field that
happens to carry source text today. The exact-bytes columns (resolver_occurrences
spelling, resolver_lookups key) keep the original bytes; the MCP layer normalizes
those at its own encoding boundary.

The pending unresolved-call key is compared against text read back from that JSON,
so it applies the same replacement to both sides.

The UTF-8 sequence check moves from mcp.c to foundation/str_util so there is one copy.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Each pool below holds a whole project's proofs and refused publication once its
fixed budget ran out:

  shared resolver  64M  django needs 133M, kubernetes 2.28G
  graph buffer     4G   kubernetes needs 5.03G
  infra adapters   4M   kubernetes' 6456 YAML files exhaust it

The shared test reaches its budget with one small Go file and a long project name,
because every proof hashes that name; the same file under a short name is the
control. The graph and adapter tests spend the former budget up front instead of
building a project that size. The graph pool is private, so that needs a test seam.

RED: cbm_shared_resolver_build returns NULL for the long name only,
cbm_gbuf_record_resolution fails, and the adapter pool refuses a leaf.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Three pools hold a whole project's proofs under fixed caps, and running out of one
refuses the whole publication with no diagnostic:

  shared resolver  4M nodes, 64M work           cbm_shared_resolver_build
  graph buffer     16M nodes, 4G work           cbm_gbuf_new
  infra adapters   65536 nodes, 4M work         cbm_shared_resolver_adapter_init

proof_hash counts every hashed byte as work, so reaching a cap only means a large
enough repository. django reached the shared cap at 78580 proof nodes, and because
every proof hashes the project name, how soon a cap is reached also depends on the
length of the checkout path.

Measured, graph pool / shared pool work:

  redis       (967K occurrences)  524M / 1.2M
  neovim     (1.47M occurrences)  934M / 0.24M
  django      (672K occurrences) 1.36G / 133M    over the shared cap
  kubernetes (6.43M occurrences) 5.03G / 2.28G   over both, and the adapter cap

These pools hold one node per distinct project fact and each operation is linear in
its own text, so the totals follow the input and are not guarding against a runaway.
cbm_resolve_proof_pool_init_project sets no cap, so only the memory core can refuse.
With this kubernetes publishes, 355464 nodes against main's 355619.

Per-file pools keep their limits: they degrade the file's outcomes to LIMITED rather
than stop publication.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
An occurrence names its project, file, scope and two 64-character content
identities, and a fact names them again inside a JSON payload. Written out
row by row that is about 3 KB per occurrence: redis publishes a 2.9 GB
database for a 180 MB graph.

The cost test pins bytes per occurrence by page count, never wall time, and
that the dictionary does not grow with the row count. The round-trip test pins
what compaction may not change: every column reads back byte for byte,
including property shapes the compact form does not recognise.

Corruption tests now target what is stored: the logical names become views, so
dropping one of those or any compact table must read as unavailable, and
removing or changing compact rows must invalidate the receipt.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…nary

The evidence tables were 10-30x the graph they describe, and three secondary
indexes on repeated strings took random inserts in one transaction, so a
kubernetes publish used 46 GB of disk and most of redis's 22 seconds.

Rows now hold integer ids and each string lives once in resolver_text. A fact's
digest is 32 raw bytes and its payload is its columns; an occurrence's
properties are its own columns plus an interned remainder without the line
number, so calls that differ only by position share one record. Lookups are
derived from fact rows. No secondary index remains: rows arrive in key order,
every b-tree grows at its right edge and each page is written once.

resolver_occurrences, resolver_facts, resolver_supports, resolver_lookups and
resolver_occurrence_evidence keep their names, columns and values as views, so
readers and the graph digest see the same logical content.

A file's observed type states were a per-file indexed query; they are now noted
as facts are written and hashed from a sorted slice. The walk's 16M node cap is
gone: it would have refused to publish any repository past that size.

Proof on redis, base 9c946d5 -> this commit: database 2,864 -> 321 MB, wall
21.9 -> 10.7 s, publish writes 18.7 -> 4.0 s. Edges, nodes and all seven logical
resolver tables hash identically, ordinals aside: those follow worker scheduling
in both binaries.

The write logs resolver.evidence.progress every 1M rows, because the worker's
log is the supervisor's no-progress heartbeat.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
TypeScript sat over 11 minutes in semantic_class_facts, sampled at
ts_node_named_child: fetching a child by index walks from the first child, so
enumerating n children costs n*n/2 steps.

A new counter charges that real cost at the loop, and the complexity suite gates
its ratio when one file's top level doubles. It lands at ~4 here; the fix follows.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
… paths

Python and ECMAScript each own separate paths through the binding index, so conflicts are unions. BINDING_DECORATED moves to the next free bit; Python lookups pass JS_SPACE_ANY; too_early keeps the ECMAScript closure rule, and the Python-guarded deferred site may still excuse it. Commit 456bdbb's message says about 16 wrong edges; the measured figure is 11.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…n and flow

The project model replaces the old import-context walker, keeping the JVM lane's scoping of a context failure to Rust and C-family. Rust's identity rule runs before the merged-variant rule for Rust targets. Call, usage, write and type dispatch each gain a Rust arm beside Go's.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
… C++ members

C-family takes the attribute-macro parse, everything else the C# repairing parse. The Rust project model already replaced the walker in this lane's pkgmap hunk, so only build_include_path_context survives. c_variants joins the identity rule; the read-as-value-reference lookup is guarded to C-family because other lanes were measured without it.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…hout one

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A configured define's body keeps its meaning in the expanded parse, so -DAPI=Junk does not read class Junk Foo as class Foo. An ambiguous TypeScript proof is two distinct functions extraction named flat, not one merged identity. The _Thread_local pointer form now parses, so its pin says so. The module index and filtered defs allocate through the memory core. Format production files the merges left unformatted.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…egration

Rows the integrated resolver now proves become required. Fixes: a Python class statement reads its bases where it stands, so a later sibling is no base; a Kotlin callable reference in an if branch stays a usage; cppcheck findings on the integrated tree, including the OO walkers keeping a stack address in the walk. Swift package entries now count the two membership facts.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…iteral rows

Rows written before the language rules existed. Each was re-derived from its
fixture source:

- Java, Kotlin, C#, Scala, Swift: a constructor call with no constructor node
  binds the class, marked as a class call. Rows expecting a Method or nothing are
  now required rows on the class declaration.
- Field initializers run in no callable node. The file-level node is their scope,
  so the LRP oracle accepts a 0/0 range for a row naming that node, as rq_target
  already does.
- Go struct literal: the type name is both USAGE and USES_TYPE at both of its
  sites, and a literal key names a field.
- Recovered receiver and fluent rows in those scenarios become required.
- probe_kotlin_data_class and probe_cs_class_methods pinned both rows missing,
  though both are required and found.

mkc_c2_csharp_operator_plus and the new mkc_c2_csharp_operator_overloads stay red
here: a call inside a C# operator is attributed to the file. The next commit fixes
that in production.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
operator_declaration was no function kind, so `operator+` had no node and every
call and usage in its body was attributed to the file. The resolver then published
`Vec CALLS Vec2.Vec2` from a body that belongs to a named member.

The operator is now a Method named by its token, spelled like C++'s `operator+`,
through the one resolver the def side and the call scope share. Overloads merge
into one node with variants, as other C# methods do.

RED: mkc_c2_csharp_operator_plus reported scope=Vec for the constructor call. The
binary operator site `a + b` still abstains, and conversion operators, property
accessors and indexers still have no node of their own.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…that now resolve

Each row was checked against the fixture source and the language's own rules.

Conditional variants (Go build tags, Swift, Erlang, Elixir, Haskell): one node
holds every branch's span, so a body call is that node's call under its authored
variant, and a caller reaches that one node in every configuration. Targets
anchor on the node's stored span. Haskell later branches stay a known gap: the
source adapter walks the raw parse, which holds only the first branch of a #if.

C# same-name classes: all seven member rows promote, the constructor call under
`using Contoso.Threading` gets its own required row, and the edge checks now pin
the bound target with the decoy still refused. `new Thing()` on a partial type is
a known gap: no one node is that type.

The parallel harness now builds the declaration index production builds, so Java,
Kotlin and C# static selection runs on that route too.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
… no calls

The parallel route decides per file whether cross-file analysis runs, and a file with no call
left to resolve was skipped. For Java, Kotlin and C# that also skipped the strict walk, which is
the only source of their field and type rows, so a class that only reads and writes its own
fields lost every USAGE, WRITES and type row in the parallel route. The sequential route has no
such filter, so the two routes disagreed.

C and C++ already had this exception. Extend it to the strict-walk languages.

RED: writes_java, writes_csharp, cp_reads_writes_java_static_field and
cp_reads_writes_cs_static_field found the write in the sequential route and lost it in the
parallel one.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…SES_TYPE alone

`cache[key] = v` stores an element through the operand's map or slice and rebinds nothing, but it
published WRITES for cache and nothing at all for key. Both are reads now, so they are USAGE. An
array variable is written in place by this form as well; its element type is not known here, so
it reads as USAGE too, which is a missing write and not a false one.

A Go parameter, result, var or conversion type also published USAGE on the same token that
type_refs already carries as USES_TYPE, which es_usage_crossfile_go forbids and the TypeScript
adapter already refuses. One token now has one role. A composite literal builds a value of its
type, a receiver has no USES_TYPE occurrence, and USES_TYPE reads a qualified type's whole
spelling rather than its member token, so those three stay USAGE.

RED: cp_reads_writes_go_global (forbid Set WRITES cache) and es_usage_crossfile_go (forbid
annotation-is-not-value-usage).

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…the t1 suites

Each row was checked against its fixture source.

Promoted to required, the target is what the language selects: same-class C# await call, Kotlin
suspend call, Kotlin sealed `is` branches with their forbids kept, Java enum methods, the static
interface target with its forbid kept, Kotlin operator plus, the imported Celery instance's
send_task with the foreign decoy kept, RestTemplate literal routes.

Corrected:
- writes_go named a `cache.` package part that a Go root file does not have, so neither the row
  nor its decoy forbid could match anything. Both now bind.
- Python and Kotlin constructor rows name the class node. A Kotlin primary constructor has no node
  of its own and Python's class call selects the class, as test_pipeline requires.
- FunC and Meson import probes required 0/0 spans. The carrier now records the literal's exact
  occurrence, and the probes require that occurrence.
- es_php_include_occurrence_spans_and_sealing_guards pinned a reason string renamed in aeb5971.

Reads are USAGE: Go, Java and C# rows list it as required. Their READS rows stay named gaps because
nothing emits READS. Java's count++ stays a gap: its assignment kinds omit update_expression.
Celery's ASYNC_CALLS rows stay gaps with an updated reason: CALLS is proved, the service
classifier does not see that proof.

Adds es_go_annotation_types_are_not_value_usage and es_go_index_assignment_reads_operand_and_index
for the Go role rules. No forbid was removed.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A matrix row on a class target brings that class's other roles into the harness
audit. `make(): Widget` and `anon(): User` name the class they also construct, so
the signature's USAGE, and USES_TYPE in Scala and Swift, become required rows at
that exact token.

mkc_c2_csharp_operator_overloads pins the operator scope for overloads stored as
variants, for a spaced `operator -` and for the word form `operator true`.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Kotlin records no USES_TYPE occurrence for a result type, so USAGE names the class there. A
parameter annotation emits neither, so ad60078's parameter rows required an edge production does
not make. Dropped those two rows.

Also corrects ad60078's message: the PHP reason string was renamed in b821d15, not aeb5971.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…ep an unreadable type

Kotlin nested types are named under their outer class and an unnamed companion
has nodes, so the pins move to those names, each confirmed against a real index.
One of them, a zero count on gk.K4.Wrapped.m, passed vacuously under a stale
suffix.

A nested class and a member function of one name now share a qualified name, and
so do a member function and a property. The later declaration owns the node, so
[Holder.Part] and a doc on the earlier twin are graph gaps. No edge goes to the
wrong twin. Both are pinned as known gaps.

The C# parse view reads `ref partial struct`, so those fixtures write it with a
comment between the words, which the grammar still cannot read and the view does
not repair. The plain spelling gets its own positive.

RED until the two doc-link fixes that follow: the slot of an unnamed companion, and
the C# scope of a file whose parse view hid a branch of a conditional.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
An unnamed companion now has a node, but the scope looked a definition up only by
a name token, which it does not have. [Type.Companion] stayed a graph gap with the
node right there. RED: doc_mentions_jvm_kotlin_scope_blob,
doc_mentions_jvm_kotlin_companion_inherits.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The parse view keeps one branch of each #if, which is enough for the graph's nodes.
The scope read that same tree, so a namespace or a type another branch names
differently looked placed and complete. A reference valid in the other build was
then reported as doc rot, where it used to be a graph gap.

When the view hid a branch, keep the file's own parse for the scope. A view that
only blanked `ref` hides nothing and stays in use. RED:
doc_mentions_cs_scope_parse_errors, doc_mentions_resolver_parse_errors.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…are required

With the harness building the index production builds, the reference argument,
the same-package access and the constructor call all bind, and each is what the
rules select: Java and Kotlin share com.example, and no constructor is declared,
so the class stands for the implicit one, as test_resolver_oo.c pins.

The Kotlin alias gap is named after its real cause. The file parses completely, so
the parse-gap name and text were stale. The alias call stays unbound once the file
also declares a class with a member.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…lement writes and annotation tokens

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…ness runs static selection

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Only a direct argument, parenthesized value, standalone reference or initializer hands over one callable. Branches and pairs do not, which is what this test required before it was flipped and what a Java ternary does. A Go result annotation is USES_TYPE alone, like TypeScript, so its USAGE row goes.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…n one pass

Fetching a child by index walks from the first child, so the audit was quadratic in the statement count, and it runs serially for every ECMAScript file. TypeScript's baselines held indexing that repository for over 28 minutes; it now takes 182 s. The C# reference-assembly fixture binds neither type now that the parse view keeps both namespaces.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The same three comparisons read as one helper, and cppcheck no longer infers a constant from the indent loop. Behaviour is unchanged.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Over the memory budget each result is parked on disk and read back only for a moment. The Java, Kotlin and C# declaration index kept qualified names from the freed result, and a resolve worker read them. TypeScript's merged label did the same. Both are found by ASan with CBM_MEM_SPILL=1; every resolver suite now runs clean that way.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Windows headers define near and far as empty macros, so the production build failed there.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…sing-braces check

GCC treats the conditional compound literal inside the context's designated initializer as missing braces, and the Linux build fails on it. A zeroed node is already null, so only a repaired parse assigns one.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
js_fixture returned its arena by value while the index kept a pointer to it, which only worked where the copy was elided. The parallel harness never freed the Rust project and ECMAScript surfaces. Templates sized exactly to their text overflow on Windows, where cbm_mkdtemp rewrites /tmp/ to the longer temp root in place.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Nothing reads it, and a generated header with tens of thousands of includes made one File node 9 MB and its database 13 times the baseline's. Under strict_string_checks that string also made the FTS backfill's LIKE quadratic, so the diagnostic CI lane timed out at 900 s. The test now pins the node's size.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant