Skip to content

No CLI command for the recording-stats and signal-defaults sweeps #1194

Closed

Description

@neuromechanist

Two backfill sweeps have admin API endpoints but no CLI command, so a manual backfill requires hand-rolling an authenticated HTTP request.

  • POST /admin/datasets/recording-stats-sweep?limit=N (RECORDING_STATS_SWEEP_MAX 200)
  • POST /admin/datasets/signal-defaults-sweep?limit=N (SIGNAL_DEFAULTS_SWEEP_MAX 30)

Both are defined in backend/src/routes/admin/datasets-lifecycle.ts and both have cron wrappers that run daily at 03:00 UTC, so routine operation is covered. The gap only shows up when someone needs to drain a backfill on demand, which is exactly what happened after the #1182 cutover: the two sweeps that migrations 0070 and 0072 unblocked had to be driven by curl because nemar admin has no command for either.

For contrast, nemar admin already exposes the comparable sweeps: data-integrity-sweep, hed-sweep, and reindex. The inconsistency is the surprising part rather than the missing capability.

Worth adding alongside: both endpoints accept a reset mode (RECORDING_STATS_SWEEP_RESET_SQL), which is likewise only reachable over raw HTTP today.

A useful shape would mirror the existing sweep commands, including the progress reporting that makes a drain legible - each call returns processed, remaining, and an errors array, so a --drain style loop can report as it goes and stop when remaining reaches 0.

Activity

  1. neuromechanist commented on Sep 3, 2026

    @neuromechanist
    ContributorAuthor

    Fixed in #1223 (dev commit fdfd3a0); ships with the next dev to main release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions