Conversation
PR Reviewer Guide 🔍(Review updated until commit 6e8cd12)Here are some key observations to aid the review process:
|
PR Code Suggestions ✨Latest suggestions up to 6e8cd12 Explore these optional code suggestions:
Previous suggestionsSuggestions up to commit 086f134
Suggestions up to commit a7d06dc
|
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #6409 +/- ##
==========================================
- Coverage 75.38% 75.35% -0.03%
==========================================
Files 456 460 +4
Lines 30254 30402 +148
Branches 4574 4584 +10
==========================================
+ Hits 22806 22909 +103
- Misses 5310 5351 +41
- Partials 2138 2142 +4
🚀 New features to boost your workflow:
|
DarshitChanpura
left a comment
There was a problem hiding this comment.
Design's fine. Sync parity and the fails-closed transport path check out (header is stripped cross-node, so the privileged write can't be reached off the REST path). One thing to fix before merge, rest is minor.
Fix: TransportSecurityConfigWriteAction#doExecute re-implements the IndexRequest build and ConfigUpdate fan-out that already exist as createIndexRequestForConfig and ConfigUpdatingActionListener. Identical today, so a change to the write semantics hits sync only. Extract the shared builder/listener so they can't drift.
Minor:
SecurityConfigWriteResponse#writeToskipssuper.writeTo(out)but the read ctor callssuper(in)— align them.not_helper cites a static-import collision, butnotisn't imported here — useCoreMatchers.not(...).- Async DELETE/PATCH go through the submitter but only PUT is tested; add them.
- Task
descriptioniscType/entityName, readable via_tasks— names not contents, noting only.
Docs and api-specification companions are marked "to follow" — nothing blocks them, link the PRs.
…nsearch-project#6337) Adds the standard OpenSearch wait_for_completion query parameter to Security configuration write endpoints. Callers can now submit a change with ?wait_for_completion=false and receive a task id immediately; the operation is tracked through the task framework and its outcome is retrievable via GET /_tasks/{task_id}. Omitting the parameter or setting it to true preserves the existing synchronous behavior byte-for-byte. Shared plumbing in AbstractApiAction + a new TransportSecurityConfigWriteAction lets any writable endpoint opt in with a single method override. Opted-in endpoints in this PR: Roles, RolesMapping, InternalUsers, Tenants, Audit (everything a Terraform provider needs when provisioning a cluster). Design decisions worth calling out for review: - Task completes only after all-node ack of the config reload (matches the point at which the sync path returns today). - Task result payload stored in .tasks contains only status and message — never configuration contents — so standard _tasks authz satisfies the 'do not expose configuration contents' acceptance criterion. - Tasks are NOT cancellable: createTask returns a plain Task rather than a CancellableTask, so _tasks/{id}/_cancel is rejected with 'doesn't support cancellation'. A mid-fan-out cancel could leave the index write committed with a partial cluster reload, which is worse than letting the operation run to completion. Closes opensearch-project#6337 Signed-off-by: Nishtha Mittal <nishthm@amazon.com>
@DarshitChanpura's review on opensearch-project#6409: 1. Required — 'TransportSecurityConfigWriteAction re-implements what already exists as createIndexRequestForConfig and ConfigUpdatingActionListener'. Extracted so the shared IndexRequest builder and fan-out listener are reused by both sync and async paths: - AbstractApiAction.createIndexRequestForConfig: private -> public. - AbstractApiAction.ConfigUpdatingActionListener: protected -> public. - SecurityConfigWriteRequest now carries a fully-built IndexRequest (constructed on the coordinator via createIndexRequestForConfig) instead of raw bytes/seqNo/primaryTerm/index, so the async path has nothing left to duplicate. - TransportSecurityConfigWriteAction.doExecute is now a one-liner: it hands the IndexRequest to client.index wrapped in ConfigUpdatingActionListener — same code path the sync save uses. Any future change to write semantics (thread context, refresh policy, fan-out ack behavior) automatically applies to both paths. 2. Minor — SecurityConfigWriteResponse#writeTo skips super.writeTo but the read ctor calls super(in). Documented why the asymmetry is inherent: Writeable.writeTo is abstract at every level up to ActionResponse, and TransportResponse(StreamInput) is documented as a no-op. Kept the read ctor's super(in) call to match the OpenSearch idiom used by every other ActionResponse in the repo. On the Request side super.writeTo IS necessary and is now called explicitly — TransportRequest writes the parent task id. 3. Minor — not_ helper 'cites a static-import collision, but not isn't imported here'. Removed the wrapper; use CoreMatchers.not directly via static import. 4. Minor — 'Async DELETE/PATCH go through the submitter but only PUT is tested; add them'. Added asyncDeleteRole_returnsTaskIdAndRemoves and asyncPatchRole_returnsTaskIdAndUpdates. Also addressing PR-Agent bot feedback: - 'No stashContext around the index write.' The refactor addresses this automatically — the async path now goes through the exact same ConfigUpdatingActionListener the sync path uses, so any context handling in the sync flow (which is already correct — AbstractApiAction.prepareRequest wraps handling in stashContext) applies identically. - codecov gap on the new Request/Response classes. Added SecurityConfigWriteSerializationTest covering: - writeTo/readFrom round-trip for the Request (all fields including the embedded IndexRequest's seqNo/primaryTerm/refresh policy) - writeTo/readFrom round-trip for the Response - getShouldStoreResult always returns true - validate() rejects empty cType. Signed-off-by: Nishtha Mittal <nishthm@amazon.com>
a7d06dc to
086f134
Compare
|
Persistent review updated to latest commit 086f134 |
|
Pushed 086f134 on top of a rebase on latest main. Required — reuse createIndexRequestForConfig and ConfigUpdatingActionListener. Done. Widened both to public static on SecurityConfigWriteResponse#writeTo / read-ctor asymmetry. Kept super(in) and added a comment explaining why the asymmetry is inherent: Writeable.writeTo is abstract up through ActionResponse so there's no concrete super.writeTo to call, and TransportMessage(StreamInput) is documented as a no-op — nothing is actually asymmetric on the wire. Matches how every other ActionResponse in the repo is written (ConfigUpdateResponse in this same package included). On the Request side TransportRequest.writeTo IS concrete (writes the parent task id), so super.writeTo is called explicitly there. not_ wrapper. Removed; static-imported CoreMatchers.not directly. Async DELETE/PATCH not tested. Added asyncDeleteRole_returnsTaskIdAndRemoves and asyncPatchRole_returnsTaskIdAndUpdates — both verify task shape, completion, the stored task-result message, and the post-condition via sync GET. Docs / api-spec companions. Will open both and link here. |
|
Persistent review updated to latest commit 6e8cd12 |
Companion to opensearch-project/security#6409 (issue opensearch-project/security#6337). Adds an 'Asynchronous configuration writes' section to the Security APIs page describing the new wait_for_completion query parameter — the list of supported endpoints, an example submit + task-lookup pair, and a note on non-cancellable task semantics. Default behavior (wait_for_completion=true) is unchanged so existing docs for individual operations remain accurate. Signed-off-by: Nishtha Mittal <nishthm@amazon.com>
This would also be good to do following up to this PR for consistency. |
| // not 400 | ||
| consumeParameters(request); | ||
| // Consume the async opt-in flag centrally (not in consumeParameters), so subclasses that | ||
| // override consumeParameters — and don't call super — still don't reject |
There was a problem hiding this comment.
Are there subclasses that don't call super? Should we consider adding to the subclasses?
Description
Category: Enhancement
Adds support for the standard
wait_for_completionquery parameter to Security configuration write APIs. Callers can now submit a configuration change with?wait_for_completion=falseand receive an OpenSearch task ID immediately ({"task":"<nodeId>:<taskId>"}), then poll the outcome via the standard Tasks API (GET /_tasks/{task_id}). Omitting the parameter — or setting it totrue— preserves the existing synchronous behavior byte-for-byte.Why: Security configuration writes today fan out a cluster-wide reload after the index write, and the caller has to hold the HTTP connection open for the entire lifecycle. If the connection drops there's no way to tell whether the change committed. Every other long-running OpenSearch API (reindex, update-by-query, force-merge, open-index…) solves this via the Tasks framework; Security did not. This change brings Security in line with that convention.
Old vs new behavior:
wait_for_completiontrue(default)false{"task":"nodeId:taskId"}is returned as soon as the task is registered — before the index write or fan-out reload has completed.Task completion boundary. The task completes only after
ConfigUpdatingActionListenerreceives all-node acknowledgements of the reload — exactly the point at which the sync path returns today. Any per-node reload failure surfaces as a task-level error.Task result payload. The stored task result in
.taskscontains only{"status":"...","message":"'entity' created."}— never configuration contents. Standard_tasksauthorization is therefore sufficient to satisfy the "does not expose Security configuration contents" acceptance criterion; no security-plugin-specific gating on task lookup was added.Cancellation. Deliberately not supported.
SecurityConfigWriteRequest#createTaskreturns a plainTaskrather than aCancellableTask, soPOST /_tasks/{id}/_cancelis rejected byTransportCancelTasksActionwith"task [...] doesn't support cancellation". Rationale: a mid-fan-out cancel could leave the index write committed but only a subset of nodes reloaded — worse than letting the request run to completion.Scope
In this PR — the shared plumbing plus opt-in for the 5 endpoints that a Terraform provider calls when provisioning a cluster:
RolesApiAction)RolesMappingApiAction)InternalUsersApiAction)TenantsApiAction)AuditApiAction)Deferred:
ActionGroupsApiAction,SecurityConfigApiAction— same shape (config write + fan-out reload), share the same plumbing, but not called by the Terraform provider. Opt-in is a one-line change per subclass; follow-up PR.AccountApiAction(self-service password change) — explicitly excluded per feedback in the issue thread; async has near-zero value for an interactive endpoint.AllowlistApiAction,NodesDnApiAction,RateLimitersApiAction,MultiTenancyConfigApiAction— same pattern, follow-up as demand appears.Companion PRs to follow (separate repositories):
opensearch-project/documentation-website— document the new query parameter, response shape, task lookup.opensearch-project/opensearch-api-specification— addwait_for_completionto the affected endpoint schemas so generated client SDKs pick it up.Implementation
New files (
src/main/java/org/opensearch/security/action/configupdate/):SecurityConfigWriteAction—ActionTypewith namecluster:admin/opendistro_security/api/write_config.SecurityConfigWriteRequest— carriescType, serialized config bytes, seqNo/primaryTerm for optimistic concurrency, security index name, task description, success message, and success status.createTaskreturns a plain (non-cancellable)Task.getShouldStoreResult()returnstruesoTransportAction.executewraps the listener withTaskResultStoringActionListenerautomatically.SecurityConfigWriteResponse—ActionResponse+ToXContentObject; body is{"status":"OK","message":"..."}matching the sync path exactly. Deliberately does not include configuration contents.TransportSecurityConfigWriteAction—HandledTransportActionthat performs theIndexRequestand then broadcastsConfigUpdateAction; only completes the listener after all-node ack, mirroring the sync path'sConfigUpdatingActionListenerchain.Modified files:
OpenSearchSecurityPlugin.java— registers the new transport action.AbstractApiAction.java— addssupportsAsync()hook (defaultfalse) and a privatemaybeSubmitAsTaskhelper wired viawithAsyncTaskSubmitter(...). Consumeswait_for_completioninprepareRequestdirectly (not insideconsumeParameters) so subclass overrides that don't callsuperstill don't reject the parameter as unrecognized.RequestHandler.java— introduces theAsyncTaskSubmitterfunctional interface and pre-branchesPUT/PATCH/DELETEon it. If the endpoint doesn't opt in,AsyncTaskSubmitter.NEVERreturnsfalseand the sync path is unchanged. Response message and status are precomputed once from the loaded configuration state so the sync body and the stored task result are byte-identical.supportsAsync()to returntrue.Backwards compatibility. The pre-branch runs only when the endpoint has opted in and
wait_for_completion=falseis explicitly present. The sync response body was refactored to compute(status, message)once instead of twice, but produces the same JSON:{"status":"CREATED","message":"'my_role' created."}for a new PUT,{"status":"OK","message":"'my_role' updated."}for an update,{"status":"OK","message":"'my_role' deleted."}for a delete. The public static helperAbstractApiAction.saveAndUpdateConfigsAsyncis left untouched becauseRollbackVersionApiActionandConfigUpgradeApiActionstill depend on its signature.Issues Resolved
Closes #6337
Is this a backport? No. New feature, targets
main.Do these changes introduce new permission(s) to be displayed in the static dropdown on the front-end? No — the transport action's authorization runs through the same REST-admin check the existing sync path uses, at the REST layer, before task submission. No new plugin-specific action name needs a UI-side entry.
Testing
New integration test file:
src/integrationTest/java/org/opensearch/security/api/WaitForCompletionRestApiIntegrationTest.javaCovers each acceptance criterion in the issue:
true) — response status and body match the pre-PR behavior.wait_for_completion=falseis requested, because validation runs before task submission (no phantom task IDs for requests that never execute).wait_for_completion=false, verify the returned task ID follows<nodeId>:<taskId>shape, poll_tasks/{id}?wait_for_completion=true&timeout=30s, assertcompleted=trueand — for Roles and InternalUsers — that the stored task result contains the exact success message the sync path would have returned.POST /_tasks/{id}/_cancelreturns a response whose body contains"doesn't support cancellation".Regression coverage: the existing
RolesRestApiIntegrationTestandInternalUsersRestApiIntegrationTestsuites pass unmodified against these changes (17 tests, 0 failures).Manual verification — run against a real multi-node local cluster built from this branch:
PUT /roles/sync_role(no param)"created"HTTP 201 {"status":"CREATED","message":"'sync_role' created."}PUT /roles/async_role?wait_for_completion=false{"task":"nodeId:taskId"}HTTP 200 {"task":"WkDLHW0bRzG7OmOx3bpeRg:133"}GET /_tasks/{id}?wait_for_completion=truecompleted:true, cancellable:false, action:"cluster:admin/opendistro_security/api/write_config", description:"roles/async_role", response:{"status":"CREATED","message":"'async_role' created."}POST /_tasks/{id}/_cancelcaused_by: illegal_argument_exception "task [...] doesn't support cancellation"wait_for_completion=falseHTTP 400 {"status":"error","reason":"Invalid configuration","invalid_keys":{"keys":"unknown_field"}}HTTP 200 {"task":"...:218"}actiongroups)HTTP 201 {"status":"CREATED","message":"'manual_ag' created."}The task-manager output in row 3 also confirms
"cancellable":falseat the framework level, so_tasks/{id}/_cancelrejection in row 4 comes fromTransportCancelTasksActionrather than any custom check — non-cancellability is a property of theTasktype, not a plugin-level filter.Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.