ImportError: cannot import name '_ENABLE_AIP_52' from 'airflow.settings'

[dominik-werner-casra]

Apache Airflow version

2.6.0

What happened

After updating to Airflow 2.6.0, none of the DAGs can be imported anymore.

What you think should happen instead

The DAGs should be imported like in version 2.5.1.

How to reproduce

The airflow webserver needs to be running. Then navigate to the /home and the errors import errors appear at the top.

Operating System

Ubuntu 22.04.1 LTS

Versions of Apache Airflow Providers

apache-airflow-providers-common-sql==1.3.3
apache-airflow-providers-ftp==3.3.0
apache-airflow-providers-http==4.1.1
apache-airflow-providers-imap==3.1.1
apache-airflow-providers-postgres==5.4.0
apache-airflow-providers-sqlite==3.3.1

Deployment

Other

Deployment details

Airflow is installed via pip on the Ubuntu server system. No containers (nor orchestrators) involved.

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg[bot]]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[eladkal]

mmm _ENABLE_AIP_52 was added in #30510 which is targeted for 2.7.0
#30509 that uses _ENABLE_AIP_52 was cherry picked for 2.6.0

cc @ephraimbuddy @potiuk not sure how this happened but I guess we need 2.6.1 rather quickly?

[eladkal]

@dominik-werner-casra did you modify settings.py in some way?

[dominik-werner-casra]

No, all configuration is done with the airflow.cfg file

[potiuk]

I think you have a broken installation. It almost looks like part of airflow files in your installation come from one airflow version and part comes from another. Can you please take a look at the installation process you have and any custom steps you do there, because I think this problem you have might only happen if somewhere in your installation process you manually replace the settings.py file that comes with airlfow with one that is your custom settings.py - which of course you absoluttely should not do. You have local_setings.py that you can replace but the "settings.py" that comes with airflow should never be replaced.

[potiuk]

Conveting it to a discussion. We both wiht @jedcunningham agree there is no way you get this error unless you do something really bad to airflow during installation. We would love to hear what is what you are doing @dominik-werner-casra

[dominik-werner-casra]

Thank you for providing some insight with our issue. After looking into the code I agree that this is strange and should not happen. However, we did not change "settings.py" but just ran pip3 install --upgrade apache-airflow. Also strange is that after this happened I put a print statement in the settings.py file but it never appeared in the output. Having a print statement in main.py on the other hand worked as expected. Then I tried to remove all *.pyc files but still the error persists.

[dominik-werner-casra]

Ok... I have been using python on a regular basis for 7 years now, but this has to be the weirdest issue I ever came across:

• So I have uninstalled airflow including all provider packages
• I did this for the global packages and for the user packages
• Then I went into "/usr/local/lib/python3.10/dist-packages" and removed every directory containing the word "airflow"
• I did the same in ~/.local/lib/python3.10/site-packages"
• Then I installed a Python virtual environment python3 -m venv ~/airflow_venv and activated it source ~/airflow_venv/bin/activate
• I reinstalled airflow pip3 install apache-airflow[postgres,dask,ldap]==2.6.0 --constraint contraints.txt (constraints file: https://raw.githubusercontent.com/apache/airflow/constraints-2.6.0/constraints-3.10.txt)
• Running which airflow tells me "/home/airflow/airflow_venv/bin/airflow" -> this is what I would expect
• Then starting the web server airflow webserver
• Navigating to the page only to discover the same error
• Repeating the whole thing but this time using python3 ~/airflow_venv/lib/python3.10/site-packages/airflow/__main__.py webserver
• Error persists

The really strange thing is this: the error should be impossible

File "/usr/local/lib/python3.10/dist-packages/airflow/decorators/setup_teardown.py", line 25, in <module>
    from airflow.settings import _ENABLE_AIP_52
ImportError: cannot import name '_ENABLE_AIP_52' from 'airflow.settings' (/usr/local/lib/python3.10/dist-packages/airflow/settings.py)

the file "/usr/local/lib/python3.10/dist-packages/airflow/decorators/setup_teardown.py" does not exist on the system!!!

Does anyone have an idea what is happening here? Is there some mechanism involved that converts python code to executables for increased performance? I have never seen something so strange with any python package before.

[potiuk]

Yes. This is what we also concluded (that it should work).

1. Are you sure you have only one scheduler (standalone dag_file_processor maybe ?) that runs? Maybe on another machine sharing the same database ?
2. Has it been restarted after the upgrade?
3. Do you happen to have .pyc files or __pycache__ somewhere in your system that have not been deleted? I believe you can configure (hint: PYTHONBYTECODEBASE env variable ?)

[dominik-werner-casra]

I followed your pointers:

1. No scheduler is running (I even get a warning on the page)

2. I restarted the server
3. I've removed all *.pyc files system-wide

However, your first point made me thinking: is it possible that the error messages from the DAG imports are not current but come from the old scheduler? I have done all experiments with the scheduler turned off but maybe I should try to get it working first.

Running the scheduler (using dask), I get the following error message (continuously until I terminate it):

/home/airflow/airflow_venv/lib/python3.10/site-packages/airflow/jobs/scheduler_job_runner.py:979 DeprecationWarning: The worker_pods_pending_timeout_check_interval option in [kubernetes_executor] has been moved to the task_queued_timeout_check_interval option in [scheduler] - the old setting has been used, but please update your config.
Process ForkProcess-1:
Traceback (most recent call last):
  File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/airflow/airflow_venv/lib/python3.10/site-packages/airflow/dag_processing/manager.py", line 259, in _run_processor_manager
    processor_manager.start()
  File "/home/airflow/airflow_venv/lib/python3.10/site-packages/airflow/dag_processing/manager.py", line 493, in start
    return self._run_parsing_loop()
  File "/home/airflow/airflow_venv/lib/python3.10/site-packages/airflow/dag_processing/manager.py", line 639, in _run_parsing_loop
    self.start_new_processes()
  File "/home/airflow/airflow_venv/lib/python3.10/site-packages/airflow/dag_processing/manager.py", line 1092, in start_new_processes
    processor.start()
  File "/home/airflow/airflow_venv/lib/python3.10/site-packages/airflow/dag_processing/processor.py", line 196, in start
    for module in iter_airflow_imports(self.file_path):
  File "/home/airflow/airflow_venv/lib/python3.10/site-packages/airflow/utils/file.py", line 389, in iter_airflow_imports
    parsed = ast.parse(Path(file_path).read_bytes())
  File "/usr/lib/python3.10/ast.py", line 50, in parse
    return compile(source, filename, mode, flags,
ValueError: source code string cannot contain null bytes

[dominik-werner-casra]

Ok I found it: we use some office files ".pptx" and ".xlsx" in some of our DAGs as templates to generate reports. It seems that those files cannot be parsed into an ast-node. This was never an issue before with 2.5.1 so I am not sure if anything changed with regard to this.

[ephraimbuddy]

@dominik-werner-casra For now, set AIRFLOW__SCHEDULER__PARSING_PRE_IMPORT_MODULES=False. The log above has been fixed and will be released in 2.6.1

[dominik-werner-casra]

Nice, this seems to work. Thank you!

[potiuk]

Ah. Yeah. That's a nice catch and - great vigilance @ephraimbuddy !

[potiuk]

That was a good idea eventually to add the escape hatch :)