Avoid optional requests import in TF converter - #28825
Justin Chu (justinchuby) merged 1 commit into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR makes the TensorFlow-to-PyTorch converter module cheaper to import by deferring the optional requests dependency until the specific download code paths execute. This supports the transformer conversion test suite that imports the module during test collection to exercise safe_extract_archive without requiring requests.
Changes:
- Removed the top-level
requestsimport to avoid requiring the dependency at module import time. - Added local
import requests # noqa: PLC0415statements insidedownload_compressed_fileand the non-archive download path indownload_tf_checkpoint.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
I do not see real issues in original code. Do we need this change? |
|
Copilot, Is this code change needed? |
Strictly speaking, no — the original top-level The only thing this change buys is making the module importable when If the CI test environment already has |
There was a failure in the CI pipeline when I created a PR, the reason was one of the import libraries('requests') failed. When I analyzed the issue with AI, it suggested to move the import statements to the functions where it was getting used, there by avoiding this particular import until it was actually needed. |
|
Is importing requests expensive though? |
|
Justin Chu (@justinchuby) — you're right, it isn't. The actual reason is dependency availability, not speed. Reproduced locally in a venv without Moving the two So this decouples the archive-safety tests from an undeclared optional dependency — it isn't a perf optimization. If you'd prefer to keep the top-level import, the equivalent fix is to add |
Description
Move the
requestsimports inconvert_tf_models_to_pytorch.pyinto the two download paths that actually use them (download_compressed_fileand the non-archive path indownload_tf_checkpoint). This lets the archive-extraction tests import the module without requiringrequeststo be installed.Motivation
The transformer conversion tests import the converter module at pytest collection time (
from convert_tf_models_to_pytorch import safe_extract_archive) purely to exercisesafe_extract_archive. However,requestsis not a declared dependency of the transformers tests —onnxruntime/python/tools/transformers/requirements.txtlistsonnx,numpy,transformers,torch, etc., but notrequests(it is normally present only transitively viatransformers).When
requestsis absent, the module-levelimport requestsraisesModuleNotFoundErrorat collection time, which fails thesafe_extract_archivetraversal/symlink tests even though those tests never touch the download paths that userequests. Deferring the import into the download functions decouples the archive-safety tests from this undeclared optional dependency.Note: this is about dependency availability, not import speed — importing
requestsis cheap (its own module code is ~1 ms; the full transitive tree is a one-time ~0.25 s cold, mostly stdlib). Addingrequeststo the transformers test requirements would be an equivalent fix; the lazy import is just the smaller, more localized change.Testing
requests: importing the module raisesModuleNotFoundError: No module named 'requests'. After moving the imports,safe_extract_archiveimports and runs withoutrequests.safe_extract_archivetar/zip traversal checks passed with.\.venv\Scripts\python.exe..\.venv\Scripts\python.exe -m pytest onnxruntime\test\python\transformers\test_convert_tf_models_to_pytorch.py -qwas attempted locally but collection requirestorch, which is not installed in this venv.