Skip to content

Stop flit from downloading and verifying trove classifiers on every provider - #73612

Merged
ashb merged 1 commit into
mainfrom
stop-downloading-trove-classiviers
Sep 23, 2026
Merged

ashb merged 1 commit into
mainfrom
stop-downloading-trove-classiviers

Conversation

@ashb

@ashb ashb commented Sep 23, 2026

Copy link
Copy Markdown
Member

Pypi was returning some 503's for this today causing CI to fail. To all
intents and purposes, the PyPI trove classifiers don't change in a way
that affects us, so making a (number) of network requests to check them
is unnecessary.

Example of a CI job output where this error appeared for ~15 of the providers we prepared:

Fetching list of valid trove classifiers               I-flit.validate
 Traceback (most recent call last):
   File "/opt/hostedtoolcache/Python/3.10.21/x64/lib/python3.10/runpy.py", line 196, in _run_module_as_main
     return _run_code(code, main_globals, None,
   File "/opt/hostedtoolcache/Python/3.10.21/x64/lib/python3.10/runpy.py", line 86, in _run_code
     exec(code, run_globals)
   File "/home/runner/work/airflow/airflow/dev/breeze/.venv/lib/python3.10/site-packages/flit/__main__.py", line 3, in <module>
     main()
   File "/home/runner/work/airflow/airflow/dev/breeze/.venv/lib/python3.10/site-packages/flit/__init__.py", line 175, in main
     main(args.ini_file, formats=set(args.format or []),
   File "/home/runner/work/airflow/airflow/dev/breeze/.venv/lib/python3.10/site-packages/flit/build.py", line 44, in main
     read_flit_config(ini_file)
   File "/home/runner/work/airflow/airflow/dev/breeze/.venv/lib/python3.10/site-packages/flit/config.py", line 13, in read_flit_config
     if validate_config(res):
   File "/home/runner/work/airflow/airflow/dev/breeze/.venv/lib/python3.10/site-packages/flit/validate.py", line 281, in validate_config
     validate_classifiers(i.metadata.get('classifiers')),
   File "/home/runner/work/airflow/airflow/dev/breeze/.venv/lib/python3.10/site-packages/flit/validate.py", line 124, in validate_classifiers
     valid_classifiers = _download_and_cache_classifiers()
   File "/home/runner/work/airflow/airflow/dev/breeze/.venv/lib/python3.10/site-packages/flit/validate.py", line 57, in _download_and_cache_classifiers
     resp.raise_for_status()
   File "/home/runner/work/airflow/airflow/dev/breeze/.venv/lib/python3.10/site-packages/requests/models.py", line 1167, in raise_for_status
     raise HTTPError(http_error_msg, response=self)
 requests.exceptions.HTTPError: 503 Server Error: Backend is unhealthy for url: https://pypi.org/pypi?%3Aaction=list_classifiers
 The command returned an error Command '['/home/runner/work/airflow/airflow/dev/breeze/.venv/bin/python', '-m', 'flit', 'build', '--use-vcs']' returned non-zero exit status 1.

…rovider

Pypi was returning some 503's for this today causing CI to fail. To all
intents and purposes, the PyPI trove classifiers don't change in a way
that affects us, so making a (number) of network requests to check them
is unncessary.

@amoghrajesh amoghrajesh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense

@Andrushika Andrushika left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, hitting the same problem

@ashb
ashb merged commit 32df179 into main Sep 23, 2026
155 checks passed
@ashb
ashb deleted the stop-downloading-trove-classiviers branch September 23, 2026 14:01
@github-actions

Copy link
Copy Markdown
Contributor

Backport successfully created: v3-3-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
✅ v3-3-test PR Link

potiuk pushed a commit that referenced this pull request Sep 23, 2026
…s on every provider (#73612) (#73619)

PyPI was returning some 503's for this today causing CI to fail. To all
intents and purposes, the PyPI trove classifiers don't change in a way
that affects us, so making a (number) of network requests to check them
is unnecessary.
(cherry picked from commit 32df179)

Co-authored-by: Ash Berlin-Taylor <ash@astronomer.io>
@potiuk potiuk mentioned this pull request Sep 23, 2026
1 task done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants