Skip to content

Support azure-datalake-store 1.x in AzureDataLakeHook - #73819

Merged
eladkal merged 3 commits into
apache:mainfrom
haseebmalik18:azure-datalake-store-1x
Oct 2, 2026
Merged

eladkal merged 3 commits into
apache:mainfrom
haseebmalik18:azure-datalake-store-1x

Conversation

@haseebmalik18

@haseebmalik18 haseebmalik18 commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

azure-datalake-store 1.x removed lib.auth, so the tenant path in AzureDataLakeHook.get_conn() fails. The managed identity path breaks too, since 1.x calls get_token() on the credential and AzureIdentityCredentialAdapter doesn't have it.

The hook now passes an azure-identity credential straight to AzureDLFileSystem: ClientSecretCredential when a tenant is set, the default Azure credential otherwise. Same approach as AzureDataLakeStorageV2Hook.

Per review, the floor is now azure-datalake-store>=1.0.1 and the 0.0.x handling is gone. adlfs releases before 2026.5.0 pin azure-datalake-store<0.1, so adlfs is bumped to >=2026.5.0 as well.

Removed the <1 cap and the dependency floor exclusion.

closes: #73795

@aaron-y-chen aaron-y-chen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. It meets the requirements of the original issue and also prevents a breaking change.

Comment thread providers/microsoft/azure/docs/index.rst Outdated
@eladkal
eladkal merged commit be53013 into apache:main Oct 2, 2026
589 of 591 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support azure-datalake-store 1.x in the Microsoft Azure provider

3 participants