Repository navigation
Fix ORM models failing to configure in processes that skip database setup - #74642
Merged
Merged
Conversation
…etup Models name each other in string relationships, which only resolve once every model module is imported. The before_configured hook that imports them all was registered in configure_orm(), so a process that skips it (virtualenv-mode subprocesses, a task re-executed for run_as_user, non-DB tests) or configures mappers before it runs (local settings) only worked if it had already imported the right model modules. If it had not, building a model failed, for example with Team.dag_bundles unable to find DagBundleModel. Register the hook in airflow.models.base, before any model class is defined, and drop the test-only copy from the pytest plugin. Remove the orm_event_handlers test: it guarded the line that moved, and now passes whatever the code does.
vatsrahul1001
approved these changes
Oct 13, 2026
phanikumv
approved these changes
Oct 14, 2026
Contributor
Backport successfully created: v3-4-testNote: As of Merging PRs targeted for Airflow 3.X In matter of doubt please ask in #release-management Slack channel.
|
vatsrahul1001
added a commit
that referenced
this pull request
Oct 18, 2026
…p database setup (#74642) (#74971) Models name each other in string relationships, which only resolve once every model module is imported. The before_configured hook that imports them all was registered in configure_orm(), so a process that skips it (virtualenv-mode subprocesses, a task re-executed for run_as_user, non-DB tests) or configures mappers before it runs (local settings) only worked if it had already imported the right model modules. If it had not, building a model failed, for example with Team.dag_bundles unable to find DagBundleModel. Register the hook in airflow.models.base, before any model class is defined, and drop the test-only copy from the pytest plugin. Remove the orm_event_handlers test: it guarded the line that moved, and now passes whatever the code does. (cherry picked from commit d79c0b1) Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com> Co-authored-by: Rahul Vats <43964496+vatsrahul1001@users.noreply.github.com>
vatsrahul1001
added a commit
that referenced
this pull request
Oct 20, 2026
…p database setup (#74642) (#74971) Models name each other in string relationships, which only resolve once every model module is imported. The before_configured hook that imports them all was registered in configure_orm(), so a process that skips it (virtualenv-mode subprocesses, a task re-executed for run_as_user, non-DB tests) or configures mappers before it runs (local settings) only worked if it had already imported the right model modules. If it had not, building a model failed, for example with Team.dag_bundles unable to find DagBundleModel. Register the hook in airflow.models.base, before any model class is defined, and drop the test-only copy from the pytest plugin. Remove the orm_event_handlers test: it guarded the line that moved, and now passes whatever the code does. (cherry picked from commit d79c0b1) Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com> Co-authored-by: Rahul Vats <43964496+vatsrahul1001@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Models name each other in string relationships, such as
Team.dag_bundles = relationship("DagBundleModel", ...), and SQLAlchemy can only resolve a name once its module has been imported. Airflow covers this with abefore_configuredhook that imports every model before mappers are configured, but the hook was registered insideconfigure_orm().settings.initialize()skipsconfigure_orm()in virtualenv-mode subprocesses (PYTHON_OPERATORS_VIRTUAL_ENV_MODE), in a task process re-executed forrun_as_user(_AIRFLOW__REEXECUTED_PROCESS) and in non-DB unit tests, andairflow_local_settingsis imported before it runs. Those processes only work if they happen to have imported the right model modules already. When they have not, building the first model fails:Non-DB tests started hitting this when #74496 stopped
import airflowfrom loading the task runner, which had been importingdagbundlealong the way. That PR added a test-only copy of the hook to the pytest plugin.This registers the hook in
airflow/models/base.py, next to the registry, so it is in place before the first model class is defined, in every process. The copy in the pytest plugin goes away. The new tests build a model in a process that skipsconfigure_orm(), and configure mappers fromairflow_local_settingswhile Airflow is being imported. Both fail on main.Why not fix
Team.dag_bundlesinstead. 37 other string relationships also name a model whose module the declaring module does not import. On main before #74496, in a process without the hook, importing any one of 59 modules (the model modules plus entry points such as the API app and the Dag processor) and then configuring mappers failed for 28 of them. Fixing each relationship in place runs into circular imports between the model modules. Importing every model before configuration covers all of them, including relationships added later.SQLAlchemy 2.0 only has
before_configuredon theMapperclass (2.1 adds a per-registry event). So, as before, the hook fires for whichever registry configures first, FAB's included, andonce=Truekeeps it to one call per process. The cost is importing every model the first time a process configures mappers, which processes that runconfigure_orm()already pay.test_orm_event_handlers.pyis removed. It checked that theevent.listenline raised no SQLAlchemy deprecation warning, and with that line gone fromsetup_event_handlersit passed whatever the code did.