feat(app): wire the duty tracker into the core workflow

[varex83agent]

Stacked on #569 (which is itself stacked on #567) — base is fix/transaction-ssz-bare-byte-list. This PR's diff is the 4 files below.

Problem

The tracker is fully implemented in pluto-core — trait, service, analysis, reason/step mapping, reporters, metrics, inclusion core, ~4.4k lines with tests — but nothing ever instantiated it. grep for MetricsDutyReporter or tracker:: across the workspace, excluding the tracker module itself, returns nothing.

Two consequences:

1. MetricsDutyReporter::new() never ran, so its zero-initialisation never happened, and report() never ran, so no label set was ever touched. Every duty metric is a LabeledFamily, which emits nothing until touched — so pluto exposed 1 of charon's 11 core_tracker_* metrics: inclusion_delay, the only unlabeled gauge.
2. All Duty failed / Not all peers participated logging was silent. Over one run charon logged 250 tracker lines; pluto logged 0.

So a dashboard panel computing success_duties_total / expect_duties_total returned no series at all — not zero, nothing.

What this wires

• TrackerService with two deadliners derived from the shared duty deadline, offset by INCL_MISSED_LAG + INCL_CHECK_LAG slots (deleter a further minute), so a duty is analysed only once its inclusion verdict can have arrived. Parity: app.go newTracker.
• calculate_tracker_delay — port of calculateTrackerDelay: at most 10s, never fewer than 2 slots, suppressing startup noise from a VC still coming up.
• Networked InclusionChecker driving the existing I/O-free InclusionCore — the follow-up the module's own TODO called for. One check per due slot at head minus INCL_CHECK_LAG; 404 means "no block proposed", not an error (charon's is404Error distinction); then trims submissions older than INCL_MISSED_LAG.
• All ten events across the nine core-workflow stitch points, mirroring core/tracking.go: call through to the real component, then report that step's error, returning it unchanged so control flow is untouched. inclusion.submitted runs before the broadcast and regardless of its outcome — peers may still succeed.
• INCL_MISSED_LAG exported and INCL_CHECK_LAG added, both needed for the analyser offset.

Step errors are shared, not stringified

The tracker needs an owned StepError (Arc<dyn Error>) while the stitch point must still return its own error, and those error types are not Clone. SharedStepError moves the error into an Arc and exposes it through source().

This is not cosmetic: analysis::is_eth2_api_error walks the source() chain looking for an EthBeaconNodeApiClientError. Flattening a step error to a string would silently downgrade every beacon-node failure to an unknown reason code.

Results

Measured on a kurtosis mixed 2-charon/2-pluto cluster (lighthouse VCs, Fulu).

Metric coverage: 1 → 9 of 11 core_tracker_* metrics. The remaining two (participation_total, inclusion_missed_total) are labeled families that appear on first occurrence.

Per-duty success/expect, charon vs pluto:

duty	charon	pluto
attester	48/71 (67.6%)	48/71 (67.6%)
aggregator	42/42	42/42
prepare_aggregator	42/42	42/42
prepare_sync_contribution	49/49	49/49
sync_contribution	49/49	49/49
sync_message	49/49	49/49
randao	10/10	10/10
proposer	8/16	10/10

Failure reasons match too — attester no_local_vc_signature = 23 on both. Duty failed logging now works, in charon's shape:

WARN pluto_core::tracker::reporters: Duty failed step=validator_api
  reason=signed duty not submitted by local validator client
  reason_code=no_local_vc_signature duty=3/attester

Inclusion checking is confirmed live — 23 Broadcasted block included on-chain reports with correct slots, pubkeys and broadcast delays, against charon's 25, with zero beacon-node errors.

Known gaps

Proposer row diverges. Charon additionally reports 6 proposer_zero_randaos and 2 not_included_onchain failures that pluto does not, because charon tracked 16 proposer duty instances against pluto's 10. Pluto's fetcher awaits the aggregated randao rather than erroring when it is absent (fetch_proposer_data → query_agg_sig_db), so a zero-randao proposer duty emits no event and is never analysed. That is fetcher behaviour, not tracker wiring — worth a follow-up.

Attestation-inclusion path. Feature::AttestationInclusion is Alpha and min_status defaults to Stable, so it is off by default in pluto exactly as in charon. Enabling it needs getBlockAttestationsV2 + getEpochCommittees and a port of conjugateAggregationBits. Left as a follow-up to keep this reviewable; the default proposer-inclusion path is complete. Worth noting for whoever picks it up: charon flattens CommitteeAggregations back into a single concatenated bitlist before building the block map (inclusion.go:796-808), so pluto's existing Block type already has the right shape — the conjugation is purely driver-side and needs no core type changes.

Checks

• cargo test --workspace --all-features: all pass (4 integration::* fail in this environment only — they need a Docker socket for testcontainers)
• cargo clippy --workspace --all-targets --all-features -- -D warnings: clean
• cargo +nightly fmt --all --check: clean
• cargo deny check: advisories ok, bans ok, licenses ok, sources ok

🤖 Generated with Claude Code

[iamquang95]

With AttestationInclusion enabled, submitted() stores attester/aggregator duties, but run() always calls proposer-only check_block(), which panics on those entries. This poisons the mutex and blocks later beacon broadcasts.

[varex83agent]

Good catch. The networked InclusionChecker only implements the proposer path (the attestation-inclusion driver is the documented follow-up), so I've masked the alpha, off-by-default AttestationInclusion feature off across the whole tracker subsystem in wire_core_workflow (new tracker_feature_set helper). With it disabled, incl_supported returns proposer-only, so the core never stores attester/aggregator submissions and check_block can never hit the unreachable!. Masking it for both the TrackerService and the checker keeps the analyser consistent too (no stalled ChainInclusion steps), and a warn! fires at startup if someone enables it. Added unit tests for the masking. Fixed in 5822bb9.

[iamquang95]

supervise it in run_lifecycle

[varex83agent]

Done — the run loop is now spawned into the run_lifecycle JoinSet (via a new inclusion_checker field on WiredComponents) instead of a detached tokio::spawn, so its exit triggers node shutdown like the scheduler/parsigdb tasks. Fixed in 5822bb9.

[emlautarom1]

+1 to @iamquang95 comments. Rest looks good to me. I'd reduce the comments/inline docs (overly verbose with little to no value)

[varex83agent]

Addressed the review in 5822bb9:

• @iamquang95 — attestation-inclusion panic: masked the alpha, off-by-default AttestationInclusion feature off for the whole tracker subsystem (new tracker_feature_set helper in wire.rs) until the attestation-inclusion driver lands. The core now never stores attester/aggregator submissions, so check_block's unreachable! is unreachable in the wired path; the analyser stays consistent and a warn! fires if the feature is enabled. Added unit tests.
• @iamquang95 — supervise the checker: the InclusionChecker::run loop is now spawned/supervised in run_lifecycle's JoinSet (via a new inclusion_checker field on WiredComponents) rather than a detached tokio::spawn.
• @emlautarom1 — verbose docs: trimmed the overly verbose inline comments/docs in the new tracker wiring and inclusion.rs.

fmt/clippy -D warnings/cargo test (app + core) all clean.