Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 20 additions & 5 deletions airflow-core/docs/administration-and-deployment/dag-importers.rst
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,10 @@ Airflow ships with two importers:
Each :doc:`language SDK <../authoring-and-scheduling/language-sdks/index>` with a configured coordinator
also registers an importer for its bundle format.

Airflow also has a YAML Dag importer at **airflow.sdk.importers.yaml_importer.YamlDagImporter**. Unlike the
two importers above, it is not enabled by default. Register it as shown under `Configuring importers`_.
See :ref:`yaml-dags` for the document format.

How Airflow uses importers
--------------------------

Expand Down Expand Up @@ -73,18 +77,29 @@ Register importers in :ref:`config:dag_processor__dag_importer_configs`. Each en
``classpath`` of the importer and, optionally, the ``extensions`` it handles and the ``kwargs`` to create it
with:

.. code-block:: ini

[dag_processor]
dag_importer_configs = [
{"classpath": "airflow.sdk.importers.yaml_importer.YamlDagImporter"}
]

Additional configurations may be added if the importer accepts them. The YAML
importer, for example, allows customizing the file extensions it scans (by
default, ``.yaml`` and ``.yml``):

.. code-block:: ini

[dag_processor]
dag_importer_configs = [
{
"classpath": "my_company.importers.YamlDagImporter",
"extensions": [".yaml", ".yml"]
"classpath": "airflow.sdk.importers.yaml_importer.YamlDagImporter",
Comment thread
uranusjr marked this conversation as resolved.
Resolved
"extensions": [".yaml", ".yml", ".json"]
}
]

Without ``extensions``, the importer handles the extensions its class declares. An entry with only a
classpath can be given as a string, for example ``["my_company.importers.YamlDagImporter"]``.
If no additional arguments are needed, an entry with only a classpath can also
be given as a string, for example ``["my_company.importers.TomlDagImporter"]``.

To register an importer for one bundle only, add an ``importers`` key, in the same format, to that bundle's
entry in :ref:`config:dag_processor__dag_bundle_config_list`:
Expand All @@ -97,7 +112,7 @@ entry in :ref:`config:dag_processor__dag_bundle_config_list`:
"name": "pipelines",
"classpath": "airflow.dag_processing.bundles.local.LocalDagBundle",
"kwargs": {"path": "/opt/airflow/pipelines"},
"importers": ["my_company.importers.YamlDagImporter"]
"importers": ["my_company.importers.TomlDagImporter"]
}
]

Expand Down
1 change: 1 addition & 0 deletions airflow-core/docs/authoring-and-scheduling/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ It's recommended that you first review the pages in :doc:`core concepts </core-c
assets
dag-result
Non-Python Task SDKs <language-sdks/index>
Writing Dags in YAML <yaml-dags>

.. _scheduling-section:

Expand Down
Loading