Skip to content

Status of testing of Apache Airflow 3.0.2rc2 #51381

Closed
Closed
Status of testing of Apache Airflow 3.0.2rc2#51381
Labels
kind:metaHigh-level information important to the communitytesting statusStatus of testing releases

Description

@kaxil
kaxil
opened on Jun 3, 2025

Body

We are kindly requesting that contributors to Apache Airflow RC 3.0.2rc1 help test the RC.

Please let us know by commenting if the issue is addressed in the latest RC.

Thanks to all who contributed to the release (probably not a complete list!):
@sunank200 @zachliu @praneeth552 @amoghrajesh @ayush3singh @AzisK @QBidolet @zach-overflow @highlyavailable @jason810496 @Lee-W @kaxil @vatsrahul1001 @aritra24 @uranusjr @oboki @potiuk @vincbeck @dominic-lcw @guan404ming @tirkarthi @jscheffl @rtrindvg @scalewithlee @jedcunningham @ephraimbuddy @quentin-sommer @XD-DENG @shubhamraj-git @dheerajturaga @parksunah @abhishekbhakat @bbovenzi @RoyLee1224 @pierrejeambrun @dstandish @choo121600 @gopidesupavan @eladkal

Committer

  • I acknowledge that I am a maintainer/committer of the Apache Airflow project.

Activity

  1. kaxil
    added
    kind:metaHigh-level information important to the community
    on Jun 3, 2025

  2. jscheffl commented on Jun 3, 2025

    @jscheffl
    jscheffl
    on Jun 3, 2025
    Contributor

    All good, mainly CI fixes that back-ported. But still a lot of fixes into this release. Wow.

  3. onprema commented on Jun 3, 2025

    @onprema
    onprema
    on Jun 3, 2025
    Contributor

    Are there plans to push the 3.0.2rc1 version to docker hub?

  4. jason810496 commented on Jun 4, 2025

    @jason810496
    jason810496
    on Jun 4, 2025
    Member

    Checked:

    Note: #50833 was reverted by #51145.

  5. kaxil commented on Jun 4, 2025

    @kaxil
    kaxil
    on Jun 4, 2025
    MemberAuthor

    Are there plans to push the 3.0.2rc1 version to docker hub?

    They are available, check https://hub.docker.com/r/apache/airflow/tags/?page=1&name=3.0.2

  6. eladkal
    added
    testing statusStatus of testing releases
    on Jun 5, 2025

  7. praneeth552 commented on Jun 5, 2025

    @praneeth552
    praneeth552
    on Jun 5, 2025
    Contributor

    Hi @kaxil and team,

    I’ve tested the changes from Add Apache Airflow setup instructions for Apple Silicon (#50179) on Airflow 3.0.2rc1 with Python 3.11.13 on my Apple Silicon Mac.

    Running airflow standalone worked flawlessly, and I was able to log in with the default credentials.

    The setup instructions work well on Apple Silicon.

    Thanks for the great work!
    — Praneeth

  8. Lee-W commented on Jun 5, 2025

    @Lee-W
    Lee-W
    on Jun 5, 2025
    Member

    tested #50773, #50950

    I think we might have missed #51295, not sure whether it's expected, but the original case seems to be well handled by #50950. The only other case that might not be handled yet is this rare mysql case https://github.com/apache/airflow/pull/51295/files#diff-680cf6d70e96761db3869142642ba1df32e5c2c5c8d1b9ab65d0200dca4718daL471

    also I saw the following dag import error when testing

    Timestamp: 2025-06-05, 18:05:02

Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/.../.venv/lib/python3.13/site-packages/airflow/example_dags/standard/example_external_task_parent_deferrable.py", line 60, in <module>
    from tests_common.test_utils.watcher import watcher
ModuleNotFoundError: No module named 'tests_common'

  9. onprema commented on Jun 5, 2025

    @onprema
    onprema
    on Jun 5, 2025
    Contributor

    Tested #50304 -- No issues ✅

  10. guan404ming commented on Jun 5, 2025

    @guan404ming
    guan404ming
    on Jun 5, 2025
    Member

    Tested all my changes and LGTM

  11. aritra24 commented on Jun 5, 2025

    @aritra24
    aritra24
    on Jun 5, 2025
    Collaborator

    Validated my change

  12. Lee-W commented on Jun 6, 2025

    @Lee-W
    Lee-W
    on Jun 6, 2025
    Member

    I found one critical bug that will break most asset-related features. example_assets.py won't work correctly now and here is the fixing PR #51464

  13. kaxil
    changed the title [-]Status of testing of Apache Airflow 3.0.2rc1[/-] [+]Status of testing of Apache Airflow 3.0.2rc2[/+] on Jun 6, 2025

  14. kaxil commented on Jun 6, 2025

    @kaxil
    kaxil
    on Jun 6, 2025
    MemberAuthor

    Changes from 3.0.2rc2

  15. dheerajturaga commented on Jun 6, 2025

    @dheerajturaga
    dheerajturaga
    on Jun 6, 2025
    Member

    @kaxil, Im trying to install 3.0.2rc2 with pip install --pre apache-airflow==3.0.2rc2 but it fails on libcst wheel

      Building wheel for libcst (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for libcst (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [443 lines of output]
      /tmp/pip-build-env-ft4bazon/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py:289: UserWarning: Unknown distribution option: 'test_suite'
        warnings.warn(msg)
      WARNING setuptools_scm.pyproject_reading toml section missing 'pyproject.toml does not contain a tool.setuptools_scm section'
      Traceback (most recent call last):
        File "/tmp/pip-build-env-ft4bazon/overlay/lib/python3.11/site-packages/setuptools_scm/_integration/pyproject_reading.py", line 36, in read_pyproject
          section = defn.get("tool", {})[tool_name]
                    ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^

  16. potiuk commented on Jun 6, 2025

    @potiuk
    potiuk
    on Jun 6, 2025
    Member

    @kaxil, Im trying to install 3.0.2rc2 with pip install --pre apache-airflow==3.0.2rc2 but it fails on libcst wheel

    This is not an issue with airflow.

    You have an issue with installing libcst package on your system. You likely have a very old pip version that does not build the packages you install in isolated environment (< 22.* or < 23.* - those two numbers mean year of release of pip. Also it seems you might have very obscure system where .whl packages produced by libcst are missing - they have many of them: https://pypi.org/project/libcst/#files - the error you see happens when:

    a) pip cannot find a binary .whl matching your system (this should be very rare case looking at the number of different binary packages they release

    b) when [i[ fails during building of the .whl from the .tgz as a backup

    Also it can happen if pip is trying to find matching versions of packages it should installl - sometimes it tries to backrack and download and build old versions of packages. If you look here: https://github.com/apache/airflow/blob/constraints-3.0.2rc2/constraints-3.11.txt the version of libcst that is "golden" - i.e. whan that we know works and we recommend to install airfow with is latest 1.8.0

    https://github.com/apache/airflow/blob/constraints-3.0.2rc2/constraints-3.11.txt

    Generally - we recommend people to use constraints to install airflow - for that very reason - see https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html

    So:

    1. upgrade your pip
    2. use constraints
    3. change the system you are trying to install airflow on to match some common architectures

  17. dheerajturaga commented on Jun 6, 2025

    @dheerajturaga
    dheerajturaga
    on Jun 6, 2025
    Member

    @potiuk , yes you are right, I have a non standard setup ( on a older sles12 machine) I was able to get around this by downgrading libcst to 1.7.0

    I verified #49380 and is good

  18. potiuk commented on Jun 7, 2025

    @potiuk
    potiuk
    on Jun 7, 2025
    Member

    I checked that all my changes (mostly in CI area) are presented. I tested Airflow with example dags (biggest change that had real impact on released package) . All looks good.

  19. RoyLee1224 commented on Jun 8, 2025

    @RoyLee1224
    RoyLee1224
    on Jun 8, 2025
    Contributor

    Tested my change, no issues.

  20. amoghrajesh commented on Jun 9, 2025

    @amoghrajesh
    amoghrajesh
    on Jun 9, 2025
    Contributor

    Wow, long list of changes, mostly in the CI area though.

    Installed the RC with breeze and verified changes

    CI:
    #50291
    #50721
    #50731
    #50974
    #50996
    #51126
    #51342

    Actual changes:
    #50384 -- works as expected
    #50596 -- works as expected
    #50602 -- works as expected
    #50697 -- works as expected
    #50808 -- works fine
    #50809 -- ui renders fine
    #50889 -- works fine
    #50927 -- works fine
    #50953 -- works fine
    #50998 -- works fine
    #51008 -- back compat restored
    #51291 -- works fine

    Doc changes:
    #49987
    #50293
    #50819
    #50954
    #50967
    #51186

  21. Lee-W commented on Jun 9, 2025

    @Lee-W
    Lee-W
    on Jun 9, 2025
    Member

    Verified

  22. quentin-sommer commented on Jun 9, 2025

    @quentin-sommer
    quentin-sommer
    on Jun 9, 2025
    Contributor

    #50949 LGTM ✅

    Here is a reproduction gist if you want to check it. It compares the current release with the RC
    https://gist.github.com/quentin-sommer/e884c904d4ff5f2552bc162ac54fd9b0

  23. kaxil
    closed this as completedon Jun 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind:metaHigh-level information important to the communitytesting statusStatus of testing releases

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions