Skip to content

Recurring schedules: attribute occurrences to their schedule in metrics, and carry the occurrence time #4445

Description

@jeremydmiller

Split out of #4437 (GAP-4). Scoped to occurrence attribution only; the manual trigger and durable last-run state are separate issues.

What already exists

More than #4437 assumed. The schedule name is already stamped on every occurrence and already reaches tracing:

  • Stamped: RecurringMessageAgent.publishOccurrenceAsyncoptions.Headers[RecurringMessage.HeaderKey] = message.Name (src/Wolverine/Runtime/Recurring/RecurringMessage.cs:16 defines the key as recurring-schedule).
  • Consumed: src/Wolverine/Envelope.Internals.cs:476-479 lifts it onto the activity as wolverine.schedule.name (src/Wolverine/Runtime/WolverineTracing.cs:183).
  • It is not a reserved key (EnvelopeSerializer.cs:20-45), so it round-trips every transport and is present on the handler span on whichever node handles the occurrence.

It is also durably present in every dead-letter row — the loose headers ride inside the serialized body blob (DatabasePersistence.cs:234,245), and a caller can read DeadLetterEnvelope.Envelope.Headers["recurring-schedule"] per row today, including over HTTP.

What is actually missing

1. The occurrence time is not on the envelope as a first-class value. It reaches the envelope only as DeliveryOptions.ScheduledTime (RecurringMessageAgent.cs:391), and the only tag derived from it is the boolean wolverine.message.scheduled (Envelope.Internals.cs:468-471). The occurrence instant is embedded in the dedup id as "{Name}:{occurrenceUtc:O}" (RecurringMessage.cs:55-58), but recovering it means string-parsing a key that exists for a different purpose. A recurring-occurrence header plus a matching tag would make "which scheduled run was this" directly readable.

2. Metrics carry no schedule attribution. Envelope.ToMetricsHeaders() (Envelope.Internals.cs:239-253) emits only message type, destination and tenant, so the success / failure / effective-time / DLQ counters in WolverineRuntime.Tracking.cs:145-204 cannot be sliced by schedule. SetMetricsTag (Envelope.Internals.cs:261) is the existing seam.

3. Dead letters are not queryable by schedule. The header is in the body blob but there is no column and no predicate — DeadLetterEnvelopeQuery (DeadLetterEnvelopeQuery.cs:21-43) filters on message type, exception type/message, received-at, replayable, time range and ids. So "which schedule produced this dead letter" is answerable one row at a time; "show me all dead letters from schedule X" is not answerable server-side.

Items 1 and 2 are small and additive. Item 3 implies either a promoted column or an index over the body, and may be worth deferring or splitting again.

🤖 Filed with Claude Code from the #4437 analysis

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions