Skip to content

HDFS-17980. Propagate Rename.TO_TRASH in authorization context for external enforcers. - #8741

Open
sadanand48 wants to merge 3 commits into
apache:trunkfrom
sadanand48:HDFS-17980
Open

sadanand48 wants to merge 3 commits into
apache:trunkfrom
sadanand48:HDFS-17980

Conversation

@sadanand48

@sadanand48 sadanand48 commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Description of PR

When HDFS Trash is enabled, rm is executed as an internal rename to .Trash using Options.Rename.TO_TRASH.
Today, external authorization plugins (for example Ranger via checkPermissionWithContext) receive operationName=rename, but do not get an explicit trash-intent signal. As a result, rm (to trash) and mv appear identical from authorization/audit context.

This JIRA proposes adding and propagating a dedicated context flag (for example renameToTrash) in INodeAttributeProvider.AuthorizationContext so external enforcers can distinguish:

rm with trash enabled -> operationName=rename, renameToTrash=true
mv -> operationName=rename, renameToTrash=false
rm skipTrash > operationName=delete
Scope (HDFS side)

Add renameToTrash field to AuthorizationContext and its builder/accessors.
Propagate the flag through FSPermissionChecker when building context for checkPermissionWithContext.
In FSNamesystem.renameTo(..., Options.Rename... options), set the flag based on presence of Options.Rename.TO_TRASH, and clear it in finally to avoid ThreadLocal leakage.

How was this patch tested?

Unit tests

For code changes:

  • Does the title of this PR start with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: Have the integration tests been executed and the endpoint
    declared according to the connector-specific documentation? Note: Automated CI
    testing doesn't cover all cases so manual testing with cloud storage is still
    required.
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

AI Tooling

If an AI tool was used:

@hadoop-yetus

Copy link
Copy Markdown

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 7m 28s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 27m 28s trunk passed
+1 💚 compile 1m 0s trunk passed with JDK Ubuntu-21.0.12+8-1-24.04-Ubuntu
+1 💚 compile 1m 4s trunk passed with JDK Ubuntu-17.0.20+8-1-24.04-Ubuntu
+1 💚 checkstyle 1m 6s trunk passed
+1 💚 mvnsite 1m 9s trunk passed
+1 💚 javadoc 0m 54s trunk passed with JDK Ubuntu-21.0.12+8-1-24.04-Ubuntu
+1 💚 javadoc 0m 55s trunk passed with JDK Ubuntu-17.0.20+8-1-24.04-Ubuntu
+1 💚 spotbugs 2m 15s trunk passed
+1 💚 shadedclient 17m 14s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 47s the patch passed
+1 💚 compile 0m 41s the patch passed with JDK Ubuntu-21.0.12+8-1-24.04-Ubuntu
+1 💚 javac 0m 41s the patch passed
+1 💚 compile 0m 46s the patch passed with JDK Ubuntu-17.0.20+8-1-24.04-Ubuntu
+1 💚 javac 0m 46s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 0m 44s /results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 141 unchanged - 0 fixed = 142 total (was 141)
+1 💚 mvnsite 0m 47s the patch passed
+1 💚 javadoc 0m 34s the patch passed with JDK Ubuntu-21.0.12+8-1-24.04-Ubuntu
+1 💚 javadoc 0m 36s the patch passed with JDK Ubuntu-17.0.20+8-1-24.04-Ubuntu
+1 💚 spotbugs 2m 6s the patch passed
+1 💚 shadedclient 16m 58s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 183m 33s hadoop-hdfs in the patch passed.
+1 💚 asflicense 0m 31s The patch does not generate ASF License warnings.
268m 9s
Subsystem Report/Notes
Docker ClientAPI=1.56 ServerAPI=1.56 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8741/1/artifact/out/Dockerfile
GITHUB PR #8741
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 06e04040eadc 5.15.0-190-generic #200-Ubuntu SMP Fri Aug 7 15:06:04 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 6855a35
Default Java Ubuntu-17.0.20+8-1-24.04-Ubuntu
Multi-JDK versions /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.12+8-1-24.04-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.20+8-1-24.04-Ubuntu
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8741/1/testReport/
Max. process+thread count 4074 (vs. ulimit of 10000)
modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8741/1/console
versions git=2.43.0 maven=3.9.15 spotbugs=4.9.7
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

@jojochuang jojochuang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good just some nits

operationType.set(opType);
}

public static void setRenameToTrash(boolean value) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please add javadoc to explain what this flag mean.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

Comment on lines +3391 to +3392
} finally {
FSPermissionChecker.setRenameToTrash(false);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this one's redundant.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

@Hexiaoqiao

Copy link
Copy Markdown
Contributor

Today, external authorization plugins (for example Ranger via checkPermissionWithContext) receive operationName=rename, but do not get an explicit trash-intent signal. As a result, rm (to trash) and mv appear identical from authorization/audit context.

Sorry, didn't get this point. For the external authorization plugins, is it enough to use operation name and src/dest path? How will Ranger use this rename options. Thanks.

@sadanand48

Copy link
Copy Markdown
Contributor Author

Sorry, didn't get this point. For the external authorization plugins, is it enough to use operation name and src/dest path? How will Ranger use this rename options. Thanks.

This is mainly for compliance tracking in Ranger Audits as when trash is enabled rm and mv operations look identical .

@hadoop-yetus

Copy link
Copy Markdown

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 23s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 28m 4s trunk passed
+1 💚 compile 0m 59s trunk passed with JDK Ubuntu-21.0.12+8-1-24.04-Ubuntu
+1 💚 compile 1m 2s trunk passed with JDK Ubuntu-17.0.20+8-1-24.04-Ubuntu
+1 💚 checkstyle 1m 6s trunk passed
+1 💚 mvnsite 1m 6s trunk passed
+1 💚 javadoc 0m 51s trunk passed with JDK Ubuntu-21.0.12+8-1-24.04-Ubuntu
+1 💚 javadoc 0m 56s trunk passed with JDK Ubuntu-17.0.20+8-1-24.04-Ubuntu
+1 💚 spotbugs 2m 21s trunk passed
+1 💚 shadedclient 17m 22s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 46s the patch passed
+1 💚 compile 0m 40s the patch passed with JDK Ubuntu-21.0.12+8-1-24.04-Ubuntu
+1 💚 javac 0m 40s the patch passed
+1 💚 compile 0m 43s the patch passed with JDK Ubuntu-17.0.20+8-1-24.04-Ubuntu
+1 💚 javac 0m 43s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 0m 44s /results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 141 unchanged - 0 fixed = 142 total (was 141)
+1 💚 mvnsite 0m 51s the patch passed
+1 💚 javadoc 0m 34s the patch passed with JDK Ubuntu-21.0.12+8-1-24.04-Ubuntu
+1 💚 javadoc 0m 35s the patch passed with JDK Ubuntu-17.0.20+8-1-24.04-Ubuntu
+1 💚 spotbugs 2m 5s the patch passed
+1 💚 shadedclient 16m 46s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 191m 37s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch passed.
+1 💚 asflicense 0m 29s The patch does not generate ASF License warnings.
269m 18s
Reason Tests
Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeLifeline
Subsystem Report/Notes
Docker ClientAPI=1.56 ServerAPI=1.56 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8741/2/artifact/out/Dockerfile
GITHUB PR #8741
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux cd68d6bfc6bc 5.15.0-190-generic #200-Ubuntu SMP Fri Aug 7 15:06:04 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 65649ae
Default Java Ubuntu-17.0.20+8-1-24.04-Ubuntu
Multi-JDK versions /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.12+8-1-24.04-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.20+8-1-24.04-Ubuntu
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8741/2/testReport/
Max. process+thread count 4114 (vs. ulimit of 10000)
modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8741/2/console
versions git=2.43.0 maven=3.9.15 spotbugs=4.9.7
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants