Skip to content

Don't import aiohttp when importing datasets - #8680

Open
kuldeeepy wants to merge 1 commit into
huggingface:mainfrom
kuldeeepy:fix/lazy-aiohttp-import
Open

kuldeeepy wants to merge 1 commit into
huggingface:mainfrom
kuldeeepy:fix/lazy-aiohttp-import

Conversation

@kuldeeepy

Copy link
Copy Markdown

import datasets imports aiohttp just to reference aiohttp.ClientError in the two retry except clauses in file_utils.py. That's about 50 ms of a ~470 ms import (python -X importtime -c "import datasets"), and nothing else imports aiohttp at that point.

This looks the error type up when the except clause runs, and only if aiohttp is already in sys.modules: if it was never imported, it can't have raised. This follows the "x" in sys.modules checks used elsewhere in the library.

Tests: test_import_datasets_does_not_import_aiohttp fails on main and passes here; test_read_with_retries_retries_aiohttp_client_error checks aiohttp errors are still retried. tests/test_file_utils.py passes and make quality is clean.

AI assistance (Claude) was used in preparing this change; I reviewed it and ran the tests locally.

@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants