Skip to content

Support multiple Docker hosts in docker_url attribute. - #38466

Merged
Taragolis merged 1 commit into
apache:mainfrom
oboki:feature-for-multi-host-connection
Apr 4, 2024
Merged

Taragolis merged 1 commit into
apache:mainfrom
oboki:feature-for-multi-host-connection

Conversation

@oboki

@oboki oboki commented Mar 25, 2024

Copy link
Copy Markdown
Contributor

This PR improves DockerOperator to support multiple Docker hosts, enabling high availability.

Changes include:

  • Allowing the docker_url parameter to accept a list of str.
  • Iterating over multiple host URLs to create and attempt connection.

@Taragolis Taragolis left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also need to fix static checks and add some tests for this feature

Comment thread airflow/providers/docker/operators/docker.py Outdated
Comment thread airflow/providers/docker/operators/docker.py Outdated
@oboki

oboki commented Mar 25, 2024

Copy link
Copy Markdown
Contributor Author

@Taragolis Thank you for the detailed review. 👍

Comment thread airflow/providers/docker/operators/docker.py Outdated
@oboki
oboki force-pushed the feature-for-multi-host-connection branch 2 times, most recently from 5f187ed to ba1227f Compare March 26, 2024 16:05
@oboki
oboki requested a review from Taragolis March 28, 2024 14:33

@Taragolis Taragolis left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also need to add new tests cases for hook, and update docstrings

Comment thread airflow/providers/docker/hooks/docker.py Outdated
Comment thread airflow/providers/docker/hooks/docker.py Outdated
Comment thread airflow/providers/docker/operators/docker.py Outdated
Comment thread airflow/providers/docker/hooks/docker.py Outdated
Comment thread airflow/providers/docker/operators/docker.py Outdated
Comment thread airflow/providers/docker/hooks/docker.py Outdated
@oboki
oboki force-pushed the feature-for-multi-host-connection branch 2 times, most recently from 271d0d6 to f225232 Compare March 29, 2024 19:46
* Allowing `docker_url` in `DockerOperator` and `base_url` in `DockerHook` to accept a list of str.
* Iterating over multiple host URLs to create and attempt connection.
@oboki
oboki force-pushed the feature-for-multi-host-connection branch from f225232 to 12ec5bc Compare March 29, 2024 20:00
@oboki
oboki requested a review from Taragolis March 29, 2024 20:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants