Skip to content

Add tests for SQLAlchemy 2 #48953

Description

@potiuk

Body

Since we now do not have FAB in preinstalled providers we can attempt to test Airflow with SQLAlchemy 2 - that however requires a separate run similar to "latest botocore" - where we remove all dependencies that block upgrade to sqlalchemy 2 and run all the tests.

Committer

  • I acknowledge that I am a maintainer/committer of the Apache Airflow project.

Activity

added
kind:metaHigh-level information important to the community
on Apr 8, 2025
added
area:CIAirflow's tests and continious integration
on Apr 8, 2025

Dev-iL commented on Jun 25, 2025

@Dev-iL
Collaborator

I'd like to give this a try.

Dev-iL commented on Jun 25, 2025

@Dev-iL
Collaborator

I tried finding the dependencies that block the upgrade. When I did

airflow$ uv pip install --upgrade "sqlalchemy[asyncio]<2.1" --no-deps --dry-run

As well as uv pip check, after actually installing, I don't get any conflicts (if ignoring providers). Are there really no conflicts or is the above check incorrect?

The below providers complain:

  • apache-airflow-providers-fab (if uninstalling flask-appbuilder)
  • apache-airflow-providers-microsoft-azure (if uninstalling flask-appbuilder)
  • apache-airflow-providers-databricks (unless we also upgrade databricks-sqlalchemy)

potiuk commented on Jun 25, 2025

@potiuk
MemberAuthor

The easy way to check is:

  1. set sqlalchemy to >=2 in pyproject.toml
  2. run uv sync --all-packages

You will see the conflicts then,

Dev-iL commented on Jun 25, 2025

@Dev-iL
Collaborator

@potiuk Thanks for the tip! Looks like it's really just fab at the end of the day, and setting the version to 5.0.0rc1 makes those issues go away.

potiuk commented on Jun 25, 2025

@potiuk
MemberAuthor

@potiuk Thanks for the tip! Looks like it's really just fab at the end of the day, and setting the version to 5.0.0rc1 makes those issues go away.

Yes. That would by my guess as well. Though - you should rather be looking at https://pypi.org/project/Flask-AppBuilder/5.0.0a8/ not rc (the original rc from 2024 is already abandoned and a8 is now the one that has the latest in-dev version).

Also you might want to know that we have a "green" PR that basically fixes all the problems caused by fab migration - here #50960 - we rebase it periodically and work with FAB creator to be ready to migrate to FAB 5. So you might want to rebase your changes / PR for sqlalachemy on top of the FAB PR and then you will be able to upgrade sqlalchemy and run all the tests (and fix them there).

Dev-iL commented on Jun 25, 2025

@Dev-iL
Collaborator

Ah, I was comparing only M+D and didn't notice that Y was different. I just rebased and adjusted the code accordingly.

Regardless, package-compatibility-wise, even with rc1 it's better. a8 seems to modify some additional dependencies but doesn't affect compatibility:

 + apispec==6.8.2
 + babel==2.17.0
 + colorama==0.4.6
 + flask-appbuilder==5.0.0a8
 + flask-babel==2.0.0
 + flask-jwt-extended==4.7.1
 + flask-limiter==3.12
 - flask-session==0.5.0
 + flask-session==0.8.0
 + flask-sqlalchemy==3.0.5
 + limits==5.4.0
 + marshmallow-sqlalchemy==1.4.2
 + ordered-set==4.1.0
 + prison==0.2.1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:CIAirflow's tests and continious integrationkind:metaHigh-level information important to the community

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions