HDDS-15945. [OEP] Support S3 WORM (Object Lock) by chungen0126 · Pull Request #11053 · apache/ozone

chungen0126 · 2026-08-18T20:57:13Z

What changes were proposed in this pull request?

This is the OEP doc for S3 WORM (Object Lock) feature.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-15945

How was this patch tested?

No tests.

devmadhuu

Thanks @chungen0126 for putting up the design. Curiously had a look and compared against AWS S3 specs. Got few doubts and points.

devmadhuu · 2026-08-24T06:21:13Z

+
+> _**Note**:
+> AWS S3 supports enabling Object Lock and configuring RetentionConfig directly during bucket creation. 
+> In Apache Ozone's current design, these settings must be configured via dedicated API calls after the bucket has been created.


In contrast to AWS S3, how do we present this to compliance auditors? Or should there be a "one-way" flag that, once set, cannot be cleared?

Thanks for pointing that out.

Enabling at Creation: We can definitely support enabling Object Lock directly at bucket creation time (e.g., via the S3 CreateBucket header/parameters) to eliminate any compliance window. I updated the doc.

One-way Flag: Exactly, objectLockEnabled is designed as a one-way flag—once enabled on a bucket, it cannot be cleared or disabled.

devmadhuu · 2026-08-24T07:00:07Z

+
+- Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode. 
+- Retention Modes:
+  - Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires. 


Is it really true for Ozone. In AWS S3, it is being made sure to be immutable at all levels or atleast audited or blocked including admin/operators users, but in Ozone, this may not be true, currently design talks about a happy path using validateAndUpdateCache when hit on normal S3/ozone APIs. In Ozone there are other ways where someone can directly edit rocksDb or delete a key meta for a locked key. Another way, where ozone repair command also allows to restore OM's DB directory from a ratis snapshot taken before the lock was applied.

Probably the design should talk about behavior when such actions can happen in Ozone , that what is trusted boundary and upto what extent the data is protected because otherwise this design may not suffice the complete compliance audit implementation.

Can think of below:

Whether ozone repair subcommands should refuse to operate on Raft indexes that carry lock state changes without an explicit unsafe flag.

Whether locked keys should be detected on OM startup via cross-check between RocksDB state and a tamper-evident external audit log — so that a manual DB-copy recovery that silently removes a lock can at least be detected on the next start.

an operator can run skip-ratis-transaction at the Raft index of a SetRetention(compliance) entry across all 3 OMs while stopped, then restart, and the audit trail of the lock is gone (though RocksDB may still hold applied state

Thanks for pointing this out!

I've added a Trusted Boundary section to clarify that any operations requiring root access bypass the application-level API and are outside the trusted boundary.

devmadhuu · 2026-08-24T07:04:24Z

+
+  message RetentionConfig {
+  optional RetentionMode retentionMode = 23;
+  optional uint64 retainUntilDate = 24;


Is it possible to move the OM leader's clock forward: The WORM check in validateAndUpdateCache will compare retainUntilDate (stored) against "now". "Now" is whatever the OM leader believes the time is.

sumitagrawl · 2026-08-24T12:53:17Z

+
+#### Bucket Table
+
+Two new fields: objectLockEnabled & defaultRetention.


Since this object Locking is from S3, what interface via s3 api will be used to configure or manage? or its directly via ozone cli?

Object locking can be configured and managed via both the Ozone API and the S3 API.

Ozone API: It uses OzoneBucket#setRetentionConfig. If no default retention is set, the rule field within retention will simply be null.

S3 API: It is managed through PutBucketObjectLockConfiguration, which basically follows the standard S3 specifications. You can refer to the official AWS S3 documentation for more details.

sumitagrawl · 2026-08-24T12:54:36Z

+
+### Ranger Access Control
+
+- Legal Hold: Introduces three new Access Types—GET_LEGAL_HOLD, PUT_LEGAL_HOLD, and CLEAR_LEGAL_HOLD—and adds them to accessTypeRestrictions for Keys.


If we introduce this feature in Ozone for validation via above api, Having same at ranger do have any specific advantage ?

how ranger and ozone role will combine in validation?

sumitagrawl · 2026-08-24T13:01:04Z

+
+#### Impact on Delete Object Flow
+
+All delete operations must perform WORM validation during the `validateAndUpdateCache` phase to preserve linearizability and prevent accidental deletion of protected data.


do impact for bucket removal or volume removal or directory removal where as per soft delete, it moves to deleted directory table, but generally validation is not done to deep child ?

sumitagrawl · 2026-08-24T13:02:01Z

+
+## Security
+
+Centralized access control via Ranger ensures all Object Lock operations (such as Retention Policy configuration and Legal Hold management) are enforced under strict access permissions. Dedicated Access Types (BYPASS_GOVERNANCE, PUT_LEGAL_HOLD, etc.) enforce the Principle of Least Privilege, preventing unauthorized tampering or removal of locks and enhancing data immutability.


Like ranger can clear / update policy, and super admin role to update / delete bucket or file policy ?

Thanks for pointing this out!

I've added a Trusted Boundary section to clarify that any operations requiring root access bypass the application-level API and are outside the trusted boundary.

ChenSammi · 2026-09-08T05:28:21Z

+  message BucketInfo {
+  // ... existing fields
+  required bool objectLockEnabled = 24 [default = false];
+  optional RetentionConfig defaultRetention = 25;


Thanks @chungen0126 for working on this design.

It looks like bucket level retention configuration is different than object level. It support days and years, not the retainUntilDate.

https://docs.aws.amazon.com/cli/latest/reference/s3api/put-object-lock-configuration.html

ChenSammi · 2026-09-08T05:33:03Z

+
+* **`GET_LEGAL_HOLD`**: Allows querying the current Legal Hold status of an object (maps to the `GetObjectLegalHold` API).
+* **`PUT_LEGAL_HOLD`**: Grants permission to apply a Legal Hold to an object (maps to `PutObjectLegalHold` with status ON). Once applied, the object is locked indefinitely, blocking all overwrite and delete operations.
+* **`CLEAR_LEGAL_HOLD`**: Grants permission to remove a Legal Hold from an object (maps to `PutObjectLegalHold` with status OFF). Because this clears immutability protection, it is treated as a high-privilege action restricted only to authorized users.


CLEAR_LEGAL_HOLD can be covered by PUT_LEGAL_HOLD, since AWS S3 doesn't have CLEAR_LEGAL_HOLD permission too, so let's avoid introduce a new extra permission.

hi all - you won't need to create new access types for this. From the STS project, the concept of actions are created in Ranger, which map exactly to the AWS actions without the s3: prefix. From the AWS documentation (https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock.html), there are 6 new actions:

s3:BypassGovernanceRetention s3:GetBucketObjectLockConfiguration s3:GetObjectLegalHold s3:GetObjectRetention s3:PutBucketObjectLockConfiguration s3:PutObjectLegalHold s3:PutObjectRetention

so in Ranger these would be:

BypassGovernanceRetention GetBucketObjectLockConfiguration GetObjectLegalHold GetObjectRetention PutBucketObjectLockConfiguration PutObjectLegalHold PutObjectRetention

For example, if you want the role to have access PutObjectLegalHold, you would grant PutObjectLegalHold action and write permission (i.e. access type) in Ranger. Or you could grant Put* and all permission and get access to all the Put actions.

The mappings in Ranger for actions to permissions/access types are found here: https://github.com/apache/ranger/blob/master/security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json.

Also, please note that when STS is merged to master, the IamSessionPolicyResolver would need to get updated with the new actions as well as S3GActionIamMapper.

Also, like @ChenSammi says, I believe you need to separate the authorization from the value being set. For example, the same PutObjectLegalHold authorization can be used with values ON or OFF in the API (see https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectLegalHold.html)

Also, to use the actions in Ranger, ensure to set the Ranger feature flag to true: https://ozone.apache.org/docs/next/administrator-guide/operations/s3/sts#ranger-feature-flag-action-matches-policy-condition

ChenSammi · 2026-09-08T06:02:54Z

+> * **Object Lock Immutability & Future Versioning Binding**:
+>   - **Current State (No Versioning)**: Once Object Lock is enabled on a bucket (`objectLockEnabled = true`), it cannot be disabled.
+>   - **Future Evolution (With Versioning)**: When S3 Versioning is supported, Object Lock and Versioning will be tightly coupled following AWS S3 semantics: once Object Lock is enabled, Versioning cannot be suspended or disabled.
+> * **Bucket Creation Semantics**:


There is one more S3 behavior, versioning is enabled automatically when Object Lock is enabled.

ChenSammi · 2026-09-08T07:31:50Z

+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
+* `DeleteObject` / `DeleteObjects` (Deletions will be rejected)
+* `CreateMultipartUpload` / `CompleteMultipartUpload` / `UploadPart` (Uploads will be rejected if the object is locked)


Reject upload in CreateMultipartUpload is good enough for the MPU, CompleteMultipartUpload and UploadPart can be untouched.

ChenSammi · 2026-09-08T07:39:00Z

+
+## Compatibility
+
+Because this design alters the schemas of Bucket Table and Key Table, OM (Ozone Manager) version leveling will be introduced to maintain forward and backward compatibility across rolling upgrades.


I think the major reason is we have added new OM APIs for the feature, so we need a new OM version to tell client whether these APIs can be called or not. And table wise, since all new fields are optional fields, they are naturally backward compatible.

ChenSammi · 2026-09-08T07:47:21Z

+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:


Do we have an JIRA open in Ranger project to track all the changes required for this feature?

@fmorg-git , would you mind sharing some knowledge with @chungen0126 about how to expose new permissions for Ranger, and what are the major tasks needed for Ranger integration, including UI changes?

I added comments

ChenSammi · 2026-09-08T07:48:41Z

+* **Compliance Mode**: Serves as the strictest compliance tier. Until the retention period expires, **no role or Ranger permission can bypass or overwrite the lock**, including cluster administrators.
+* **Governance Mode and `BYPASS_GOVERNANCE`**:
+  * **Mechanism**: Governance Mode allows authorized users to overwrite, delete, or alter the retention duration of a locked object before its expiration date.
+  * **Authorization Binding**: Introduces the **`BYPASS_GOVERNANCE`** Access Type, configured under `accessTypeRestrictions` at the **Volume level**.


Why BYPASS_GOVERNANCE is configured at volume level?

I believe BypassGovernanceRetention should be checked at the key level.

chungen0126

Thanks @ChenSammi, @devmadhuu @sumitagrawl, and @fmorg-git for the review! I’ve updated the design doc based on your feedback—please take another look.

Regarding the Ranger integration part, I'm still relatively new to it and actively catching up, so I would really appreciate it if @fmorg-git could help take a look. I'll continue refining the Ranger-related details in upcoming updates.

devmadhuu · 2026-09-11T10:13:00Z

@chungen0126 , just want to highlight to be in compliance with AWS S3 WORM, that AWS has recently added variable retention with event holds to S3 Object Lock, so may be we can think about adding EVENT_TRIGGERED_RETENTION in our existing enum for extended support. Enterprise S3 semantics are increasingly about governance workflows, not just object durability. Reference : AWS S3 doc

object created
      |
event hold
      |
business event occurs
      |
hold released
      |
retention timer starts
      |
WORM until timer expires

chungen0126 added 2 commits August 19, 2026 04:41

draft

846114d

draft

974a5d7

jojochuang added the design label Aug 20, 2026

devmadhuu reviewed Aug 24, 2026

View reviewed changes

sumitagrawl reviewed Aug 24, 2026

View reviewed changes

chungen0126 added 2 commits September 5, 2026 17:58

update trusted boundary

a805c82

update design doc

803c296

ChenSammi reviewed Sep 8, 2026

View reviewed changes

chungen0126 added 2 commits September 11, 2026 16:13

Merge branch 'master' into HDDS-15945-design-doc

4d41ad2

address comments

d45c025

chungen0126 commented Sep 11, 2026

View reviewed changes

address comments

da3e137


		#### Bucket Table

		Two new fields: objectLockEnabled & defaultRetention.


		### Ranger Access Control

		- Legal Hold: Introduces three new Access Types—GET_LEGAL_HOLD, PUT_LEGAL_HOLD, and CLEAR_LEGAL_HOLD—and adds them to accessTypeRestrictions for Keys.


		#### Impact on Delete Object Flow

		All delete operations must perform WORM validation during the `validateAndUpdateCache` phase to preserve linearizability and prevent accidental deletion of protected data.


		## Security

		Centralized access control via Ranger ensures all Object Lock operations (such as Retention Policy configuration and Legal Hold management) are enforced under strict access permissions. Dedicated Access Types (BYPASS_GOVERNANCE, PUT_LEGAL_HOLD, etc.) enforce the Principle of Least Privilege, preventing unauthorized tampering or removal of locks and enhancing data immutability.


		## Compatibility

		Because this design alters the schemas of Bucket Table and Key Table, OM (Ozone Manager) version leveling will be introduced to maintain forward and backward compatibility across rolling upgrades.


		### Ranger Access Control

		Ozone Object Lock enforces a dual-gate mechanism combining Ranger authorization checks and underlying WORM state validation:

Uh oh!

Conversation

chungen0126 commented Aug 18, 2026

What changes were proposed in this pull request?

What is the link to the Apache JIRA

How was this patch tested?

Uh oh!

devmadhuu left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ChenSammi Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ChenSammi Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ChenSammi Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ChenSammi Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fmorg-git Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chungen0126 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ChenSammi Sep 8, 2026 •

edited

Loading

ChenSammi Sep 8, 2026 •

edited

Loading

ChenSammi Sep 8, 2026 •

edited

Loading

ChenSammi Sep 8, 2026 •

edited

Loading

fmorg-git Sep 9, 2026 •

edited

Loading

chungen0126 left a comment •

edited

Loading