Skip to content

Use look-ahead scheduling to run independent committor intents #1805

Description

@snawaz

AppliesTo : dev / 87ee668

Goal

Use look-ahead scheduling to find independent committor intents beyond earlier blocked work and run them on available execution slots. Avoid a fixed look-ahead depth cutoff and repeated scans of the entire pending list.

This is a low-priority, later refactor. It is blocked by #1804: both master PRs must be implemented, and dev must first be rebased onto master containing those changes. Re-evaluate this design against the rebased code before implementation. The AppliesTo commit records the current planning baseline.

Why

#1804 deliberately uses approach 1: a bounded admission reserve that reuses the existing scheduler and limits release-branch risk. Its hard cutoff can still hide independent work behind a sufficiently long conflicting prefix.

Approach 2 uses look-ahead: earlier blocked intents remain queued while independent intents farther back can execute. Dependencies on earlier waiting intents still determine which work is eligible.

We can implement look-ahead with incremental dependency tracking and a ready queue: inspect new work once, then reconsider older blocked work only when a relevant dependency changes.

On the current dev baseline, eligibility selection scans the account queues. Its backlog stores IDs backed by outbox accounts, so master's payload-retention assumptions must not be copied into this design.

Scope

Approach 2: look-ahead algorithm

  1. Process new pending work in arrival order, including recovered work. Register each intent once in an ID lookup and the FIFO queues for its committed accounts.
  2. Track how many required account queues still have an earlier intent ahead of it. Zero unresolved account dependencies makes the intent ready.
  3. Maintain a ready queue. Available execution slots take ready work without searching unrelated blocked intents.
  4. On successful completion, remove the intent from its account queues and update only the affected successors. Enqueue each newly eligible intent once.
  5. Preserve dependencies while an intent retries. Keep ready work waiting safely when all 50 execution slots are occupied, and dispatch it promptly when permits become available.
  6. Integrate failure and cancellation handling with every index and the ready queue. Preserve dev's terminal-failure poisoning, dependent-intent invalidation, result-reporting, and outbox recovery behavior.

For example, while A executes, A+B must wait and a later B must wait behind A+B. Independent C can become ready immediately, regardless of the number of unrelated blocked intents already indexed.

There is no fixed look-ahead-depth window for indexed pending work. Admission and completion processing must still run in bounded batches and yield fairly. Define explicit memory and metadata budgets and safe behavior at those budgets; unlimited scheduling visibility is not permission for unlimited RAM growth. Preserve durable outbox ownership and accepted work.

Potential additional performance improvements

Evaluate these against the post-rebase baseline and include them only when measurements justify them:

  • Direct ID/state lookup for duplicate detection instead of scanning account-queue fronts.
  • Cache immutable committed-account keys and maintain reverse ownership/count information to avoid repeated key extraction and global validation scans without weakening correctness checks.
  • Shorten lock hold times, reuse temporary allocations, and batch completion updates or notifications.
  • Keep dependency metadata compact and load large payloads from outbox accounts when needed; measure the memory-versus-I/O tradeoff.
  • Provide fair dispatch among eligible intents and prevent arrival processing or retry bursts from starving completions and ready work.
  • Report actual permit utilization separately from retrying tasks, plus ready/blocked/backlog counts, waiting age, and scheduling latency using bounded metric labels.

Acceptance criteria

  • Independent work deep behind a conflicting prefix becomes ready without repeatedly rescanning older unchanged entries.
  • New arrivals are indexed once and normal completion processing touches affected dependencies rather than rebuilding readiness from all account queues.
  • Preserve per-account FIFO ordering, transitive dependencies, duplicate handling, retries, terminal-failure cascades, and recovery ordering.
  • No intent is dispatched twice, lost, or left in stale ready/dependency indexes; poisoned successors do not execute.
  • Available execution slots consume ready work promptly, with concurrency at most 50 and no busy polling or missed wakeups.
  • Compare dispatch latency, scheduling CPU, allocations, lock contention, and payload I/O with the rebased baseline under hot-account, broad-account, mixed-bundle, retry/failure, and recovery workloads.

Activity

  1. self-assigned this
    on Oct 14, 2026
  2. added theissue type on Oct 14, 2026
  3. changed the title [-]Schedule pending committor intents incrementally from account dependencies[/-] [+]Use look-ahead scheduling to run independent committor intents[/+] on Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions