Skip to content

live_sync: follow LIB, single batch path, AH-lag guard - #97

Open
ety001 wants to merge 3 commits into
nextfrom
fix/livesync-lib-follow
Open

ety001 wants to merge 3 commits into
nextfrom
fix/livesync-lib-follow

Conversation

@ety001

@ety001 ety001 commented Oct 19, 2026

Copy link
Copy Markdown
Member

Why

Two failure modes observed on the full-replay test database, both from following too close to the head:

  1. 19-day ops gap (blocks 109,995,083–110,553,713): in single-block follow mode, get_ops_in_block on seconds-old blocks hits the public node's account_history index before it has recorded the block — returns empty ops, no error — and followBlock wrote the block header anyway. Every block processed in follow mode lost its operations; blocks+transactions kept landing, masking the gap.
  2. Orphan exposure: following HeadBlockNumber means non-final blocks are ingested (see the duplicate-creation-op incident on the same database).

What

Per the review directive, the follow loop is now:

  • Target last irreversible block (LastIrreversibleBlockNum) — every ingested block is final and AH-indexed by construction.
  • One batch path for all distances (followBlock deleted): gap > follow_threshold (default 100, LIVE_SYNC_FOLLOW_THRESHOLD) → chunkSize batches (default now 100, LIVE_SYNC_CHUNK_SIZE); smaller gap → single call; caught up → sleep 3s and poll.
  • AH-lag guard: a block returning zero ops fails the chunk for retry (every block carries at least producer_reward, so empty = not yet indexed).

Verified

  • go build, go vet, go test -count=1 ./internal/... ./cmd/... all pass.
  • Docs synced (AI-driver 01/03, LiveSyncConfig comments).
  • The fix is being validated live on the test database's ongoing gap backfill (chunk path unchanged for catch-up).

…ess page into account page

Dashboard:
- Stop stretching the three info cards to equal height (items-start) and
  collapse Global Properties to the first 10 entries behind a toggle,
  removing the large blank areas under Network Performance / Reward Pool
- Tighten paddings, font sizes and row spacing across the dashboard cards
  and the activity stream rows

Activity stream:
- Render a readable, link-rich summary per operation (transfer from/to +
  amount, vote voter/target/weight, comment post link, witness votes,
  delegations, reward claims, limit orders, custom_json follow/reblog,
  ...) parsed from the full op payload the WebSocket already pushes
- Block numbers link to /blocks/:id, accounts to /accounts/:name, posts
  to /posts/:author/:permlink; block witness links to the account page

Witness detail merge:
- Fold the standalone /witnesses/:id page into /accounts/:name: witness
  accounts get a Witness badge plus a Witness card (votes, missed blocks,
  last confirmed block, versions, props); /witnesses/:id now redirects
- Point all witness links (witness list, account witness votes, sidebar
  favorites, dashboard feed) at the account page
The overview cards and the Supply/Network info cards duplicated content
already shown on the Dashboard (accounts, supply, head block, ...). Drop
them along with the now-unused global-stats/blockchain-props queries, so
Dashboard answers 'how is the chain right now' and /stats answers 'how
has the chain trended'.
… AH-lag guard

Reworks the follow loop per the review directive and the 19-day ops-gap
incident:

- Target the last irreversible block (LastIrreversibleBlockNum), never the
  head: every ingested block is final (no orphan pollution) and its
  account_history entry exists by construction.
- Remove the single-block followBlock mode entirely. One batching rule for
  all distances: gap > follow_threshold (default 100, LIVE_SYNC_FOLLOW_
  THRESHOLD) → chunkSize batches (default now 100, LIVE_SYNC_CHUNK_SIZE);
  smaller gap → one call. Caught up → sleep 3s and poll again.
- syncChunk fails and retries any block that returns zero ops. Every block
  carries at least producer_reward, so an empty ops list means the node's
  account_history index has not recorded the block yet; writing the header
  anyway was the 19-day ops gap (blocks+transactions landed, operations
  never did).

Docs synced (AI-driver 01/03, LiveSyncConfig comments).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant