Skip to content

UTB parity 6/8: release pins, trust base store (read + compare-and-swap), memory store and the cache session - #5

Merged
ristik merged 3 commits into
mainfrom
dev2/java-6-session
Nov 8, 2026
Merged

ristik merged 3 commits into
mainfrom
dev2/java-6-session

Conversation

@ristik

@ristik ristik commented Nov 8, 2026

Copy link
Copy Markdown
Owner

Part of the Java UTB parity stack (owner §51, plan PR 6). Stacked on #4 (dev2/java-5-fetch, approved at 1d74173), which is stacked on dev3's #1-#3. Mirrors JS ReleasePin.ts, TrustBaseStore.ts, TrustBaseSession.ts and Rust trust_base_cache.rs, tests/{utb_cache,utb_cache_max}.rs. Rust is the semantic reference. Nothing here touches unicitynetwork/*.

What it adds (package org.unicitylabs.sdk.api.bft.trustbase)

  • Pins: ReleasePin (network, genesis and pinned epoch/EntryID, complete embedded bundle, bundle sha256) with verify(limits) (digest, strict chain from the pinned genesis, both ids, both epochs; anything wrong is the fatal ReleasePinInvalidException), ReleasePin.fromBundle(bytes) for a local/private network, VerifiedReleasePin.isExtendedBy for overrides. ReleasePins.ROWS (the built-in table) is empty here; PR 8 generates it from the approved assets. Sessions take an explicit table (TrustBaseSessionOptions.withTable) for tests.
  • Store: TrustBaseStore (read(key, maxBytes) -> absent or {bytes, revision}, atomic compareAndSwap(key, expectedRevision|null, bytes) -> WRITTEN/CHANGED, both CompletableFuture), TrustBaseStoreException (OVERSIZE, BUSY, PERMISSION, QUOTA, IO), MemoryTrustBaseStore (copies in and out), TrustBaseCacheKey (utb/v1/<network>/<genesisEntryId>/<namespace>, revision = lowercase hex SHA-256).
  • Session: TrustBaseSession.openPublic(network, override, store, options), openPrivate(namespace, pin, ...) (namespace 1-64 bytes, no /, not public), openLocal(bundle, ...) (fixed namespace local). Open verifies the pin, rereads the cache strictly from the pinned genesis, merges only a compatible extension (a shorter cache is extended from the embedded history, a longer one keeps the embedded witnesses), reports a damaged/foreign/oversize cache as a CacheDiagnostic and falls back to the embedded history; it never writes. append/appendAsync, flush (clean sessions neither read nor write; otherwise reread and revalidate, import a compatible suffix, write by compare-and-swap, at most 4 writes; a damaged cache is replaced only against its exact observed revision and reported once; an over-budget or checkpoint-diverging valid cache is never overwritten), refresh(sources, options) (update through the session, then flush, also after a partial failure), history() (read-only TrustBaseHistoryView, fails with HistoryConflictedException after a fork), exportBundle(), getConflict().
  • Conflict: a different entry for a held epoch is first verified against the held predecessor; an invalid one is a plain refusal, one at or before the checkpoint is TrustBasePinMismatchException (no latch), an authenticated fork after it throws TrustBaseConflictException and latches the session (sticky, session-local, including the view). The latch is not claimed to survive a restart.
  • Concurrency: all mutation runs one task at a time on the executor passed in TrustBaseSessionOptions.of(executor) (a bounded executor of the application; Runnable::run runs on the caller). Store futures are awaited on that executor, never on the caller's thread. Reads of the view take a consistent snapshot at any time. The core owns no global executor and does no file I/O.

Decisions to review

Evidence

  • src/test/resources/vectors/utb-cache/v1/vectors.json is byte-identical to the Rust file (sha256 a071fef1f3d5c34a2918e8d9f6e50a083aa87bc1261740f5e4ecedb7b8790342, pinned by a test): all 15 pins, 40 open cases, 23 flush cases, 12 offer cases, key and revision rows pass through the Java session.
  • TrustBaseSessionTest (14 tests): one bundle then no write, store suffix imported, stale revision reread/merged, contention (4 writes then CacheContendedException, session stays dirty and verified), failed save, read failure cannot authorize a replacement, damaged cache reported once and replaced only at its revision, fork latches reads/appends/flush/refresh, refresh through the session saves the result and saves the prefix after a failure, public override rules, namespace rules, local bundle, concurrent appends/flushes/reads on a 4-thread pool.
  • Mutants killed (one per rule): clean-flush shortcut removed; successor verification before classifying a conflict removed; checkpoint-overlap check removed; header-network check removed; 4 attempts -> 3; latch removed; dirty flag not set on append; blocked cache overwritten; memory store CAS ignoring the revision. (One survived at first because the test compared against the constant it mutates; the test now asserts 4.)
  • Real JDK 11 (JAVA_HOME Temurin 11.0.33): all trustbase tests, animalsnifferMain (Android API 31) and checkstyle clean for the new main files. No new dependency; no file code in the core.

… serialized session (open, merge, flush, refresh, sticky conflict), the shared cache vectors
@ristik

ristik commented Nov 8, 2026

Copy link
Copy Markdown
Owner Author

java#5 (PR6/8) at 679da53: changes requested. One small blocker, a lost-update race on dirty in flushNow. The rest holds.

Blocker: a synchronous append can run while a flush is in progress. append takes synchronized (this.history) on the caller's thread (the fetch loop's target path, for example), while flushNow runs on the serial executor without that lock. In the "store already holds this history" branch:

  1. flushNow takes ours = history.getTip() and reads the store, whose tip equals ours.
  2. Another thread's append adds the next entry and sets dirty = true.
  3. flushNow sets this.dirty = false and returns saved(false).

The new entry is now unsaved, but the session is clean, and later flushes take the "clean flush does not touch the store" shortcut. The design's meaning of saved ("nothing of ours is unsaved") breaks. The WRITTEN branch already guards against this (dirty = !Arrays.equals(tip now, ours)); the "store already holds it" branch needs the same guard. A test with a store whose read lets you append from another thread before it returns would pin it. This can't happen in JS (single-threaded) or Rust (&mut), so it's Java-specific.

Smaller (not blocking): append locks on this.history, which is a reassignable (volatile) field. It's only reassigned in load before the session is published, so it's correct today, but a final lock object would be robust.

Checked:

  • Vectors: src/test/resources/vectors/utb-cache/v1/vectors.json is byte-identical to sdk-rust main's (SHA-256 a071fef1…, the 128 KiB revision).
  • Tests: locally on JDK 11, test --tests '*trustbase*' compileJava animalsnifferMain passes, 93 tests and 0 failures. CI was still running.
  • No cached bytes are trusted without verification: loadCache checks the byte, entry-count and per-entry budgets first. It then decodes, checks the network and genesis, and rebuilds the history from the pinned genesis through the verifying append. The overlap with the checkpoint has to match. Nothing in the store is taken on trust.
  • Rejection classes match the design and Rust and JS: OVERSIZE and PIN_MISMATCH are blocked and never overwritten. MALFORMED, WRONG_NETWORK and WRONG_GENESIS are replaced against the exact observed revision and reported. A longer compatible cache extends the embedded history, so the embedded witnesses stay, and a shorter one marks the session dirty.
  • Flush: a clean flush doesn't read the store (decision (a)). Otherwise it rereads and strictly revalidates, imports the suffix past the checkpoint (a fork latches), makes at most 4 CAS writes, and finally reports CacheContendedException.
  • Conflict latch: session-local. HistoryConflictedException on later use, and export still works on a conflicted session.

… history dirty (one guard for both branches, under a final lock); two tests, one mutant
@ristik

ristik commented Nov 8, 2026

Copy link
Copy Markdown
Owner Author

java#5 at 024a436: APPROVE, for the owner's list. The fix is right: settle(saved) sets dirty to whether the current tip differs from the saved one, under a final lock that append also takes. That's used for both the store-already-holds branch and the WRITTEN branch, and the lock no longer sits on the volatile history field. Locally on JDK 11, the trust base tests pass (95), and compileJava and animalsnifferMain are clean. Putting back the unconditional dirty = false in settle fails TrustBaseSessionTest. CI build is green. My earlier checks stand: strict re-verification of cached bytes, the rejection classes, CAS with 4 attempts, the session latch, and vectors a071fef1.

…istoryView is dev3's; the session view adds getGenesisPin
@ristik

ristik commented Nov 8, 2026

Copy link
Copy Markdown
Owner Author

java#5 at cc7d1f1: APPROVE carried. It merges #4 (fb814af), with one add/add conflict in TrustBaseHistoryView, resolved to dev3's version. Against dev2's version, the interface is the same set of methods plus getGenesisPin(); the doc wording of epochEndRound differed, but it means the same thing (the start round of the held next entry). The session's view delegates every method to live(), the new getGenesisPin included, so behaviour is unchanged. CI is green and the full build passes at #6's head.

@ristik
ristik changed the base branch from dev2/java-5-fetch to main November 8, 2026 03:10
@ristik
ristik merged commit 5cba14d into main Nov 8, 2026
1 check passed
@ristik
ristik deleted the dev2/java-6-session branch November 14, 2026 17:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant