Skip to content

Fix DatasetInfosDict metadata overwrite - #8696

Open
r-abaryan wants to merge 1 commit into
huggingface:mainfrom
r-abaryan:fix/dataset-infos-overwrite
Open

r-abaryan wants to merge 1 commit into
huggingface:mainfrom
r-abaryan:fix/dataset-infos-overwrite

Conversation

@r-abaryan

Copy link
Copy Markdown

Fix DatasetInfosDict metadata overwrite

Summary

DatasetInfosDict.write_to_directory(..., overwrite=True) raises
AttributeError: 'dict' object has no attribute 'to_dataset_card_data' for
nonempty input. The writer initializes its collection as a plain dict; if a
legacy dataset_infos.json exists, it can update that file before crashing and
leave the README metadata stale.

Initialize a DatasetInfosDict and replace the README's existing
dataset_info field when overwriting. An empty overwrite now clears old
configuration metadata while preserving other card metadata and description.

Validation

  • Added regression coverage for fresh directories, README and legacy JSON
    layouts, new/existing/empty configurations, merge and overwrite behavior,
    multiple configurations, and preservation of unrelated card content. The
    new tests exposed 11 failures before the fix.
  • tests/test_info.py tests/test_metadata_util.py: 50 passed on both Python
    3.10.21 and 3.14.7.
  • Windows Python 3.14 unit suite: 3,318 passed, 232 skipped, with three tests
    requiring unavailable symlink privileges excluded.
  • Repository-wide Ruff lint and formatting checks passed.

The broader Windows run excluding CI Hub tests completed with 3,399 passed,
258 skipped, 4 integration failures, and 3 symlink setup errors. Those failures
did not enter the changed writer; they involved missing torchcodec, a
subprocess using base Python, an old cache hash expectation, and Windows
symlink privileges. The CI Hub module stalled on a cleanup request without a
timeout, so a clean full-suite result is not available locally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant