Skip to content

Import failure: on004917 (ds004917) #78

Open

Description

@neuromechanist

Dataset: on004917 | OpenNeuro source: ds004917
Missing objects: 104/410 (at declared size, per nemar admin import verify)
Failure class: upstream-403

What happened: part of the epic #967 (nemarOrg/nemar-cli) OpenNeuro 0-byte-content recovery. This dataset did not reach a complete state. The hardened finalize gate correctly did NOT publish incomplete data — the dataset retains its real (partial) state; nothing empty was published.

Triage action: Specific files 403 upstream. UNRECOVERABLE via copy; leave incomplete or withdraw.

Refs: epic nemarOrg/nemar-cli#967; recovery run followed .context/recover-runbook.md. data_complete=0 is the honest catalog state until resolved.

Activity

  1. added
    import-failureOpenNeuro import/recovery failed for a dataset
    upstream-403Specific files return 403 from OpenNeuro (unrecoverable)
    on Jul 22, 2026
  2. nemar-bot commented on Sep 12, 2026

    @nemar-bot

    Relabelled (2026-09-12T03:02:09.982Z): cause changed: upstream-403 -> needs-triage.

    No cause could be determined from the reported error. Open the workflow run to classify it by hand.

  3. added
    needs-triageNo cause could be determined from the reported error; classify by hand from the run
    and removed
    upstream-403Specific files return 403 from OpenNeuro (unrecoverable)
    on Sep 12, 2026
  4. neuromechanist commented on Sep 16, 2026

    @neuromechanist
    MemberAuthor

    Recovery concluded; on notice to 15 October 2026 (#1396, ADR 0064)

    Re-measured against the bucket on 2026-09-16, on the basis ADR 0064 defines
    (distinct annexed keys from git annex find, never tree entries and never
    total_files, because metadata is not annexed and arrives from GitHub whether
    or not a recording survived):

    on004917: 104 of 410 data keys have no object. 74.6% of its data is available.

    That is below the 90% a listed dataset must reach, so this dataset is on
    notice, not withdrawn
    . Nothing was tombstoned: ADR 0064 opens a notice period
    where the gap is an upstream export defect, because a defect the source can
    clear in a day should not cost a dataset its DOI. OpenNeuro was sent the
    per-dataset measurements on 2026-09-15 with a withdrawal date of
    15 October 2026. If the content is fetchable by then the gap closes; if not,
    the dataset is withdrawn (repository private, DOIs tombstoned) until it is.

    Recovery was re-attempted and it is upstream's gap, not ours. Five defects in
    our own tooling were fixed first, each of which had made content look
    unrecoverable when it was not: a double-encoded copy source (%2520 for a
    space), no fallback when a recorded version was refused, a dry run that reported
    would-recover without probing, a .log.rmet retraction (-) read as part of
    the version id, and a base64-encoded pin read as no pin at all. Only after those
    were fixed was every key tried against both routes OpenNeuro publishes. Six of
    the eleven datasets previously withdrawn on an unmeasured upstream_403 turned
    out to be fetchable the whole time, which is why this measurement was redone
    rather than trusted.

    The false presence claims were already retracted. The location log
    advertised keys the bucket cannot back, which tells every clone to fetch bytes
    NEMAR does not hold, and that is wrong today whatever happens in October. 230
    keys across on003574, on004475, on004917, on005571 and on005279 were
    withdrawn with nemar admin fleet key-registration --retract-false-claims, and
    verified afterwards by re-reading each location log from a fresh clone of origin
    and replaying it to a final state. The zero-byte objects themselves are left in
    place: inert once nothing claims them, though still visible to a size audit.

    Verification basis: presence at declared size, so a zero-byte object left by
    a failed copy counts as missing rather than present (#967). Size-level, not
    checksum-level.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    import-failureOpenNeuro import/recovery failed for a datasetneeds-triageNo cause could be determined from the reported error; classify by hand from the run

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions