[Paranitharan1110]

Description

Implement dual-store mechanism to store sensitive workflow data in Kubernetes Secrets instead of ConfigMaps.

Fixes: kubevela/kubevela#7153
Depends on: kubevela/pkg#133

Problem

KubeVela's workflow engine stores inter-step runtime state in ConfigMaps. When workflow steps pass sensitive data (tokens, credentials, API keys), that data is persisted in plaintext in the ConfigMap, creating a security vulnerability.

Solution

This PR implements a dual-store mechanism:

• ConfigMap - stores non-sensitive workflow variables (existing behavior)
• Secret - stores sensitive workflow variables (new)

Two paths are covered:

1. workflow.steps[].outputs[] - via pkg/hooks/data_passing.go

   • When sensitive: true is set on an output, it routes to the Secret store

2. op.ws.#DoVar - via pkg/providers/builtin/workspace.go

   • Added Sensitive bool field to VarVars struct
   • Put with sensitive: true stores in Secret
   • Get tries sensitive store first, falls back to regular store

Changes

• pkg/context/context.go - Added secretStore, sensitiveVars, GetSensitiveVar(), SetSensitiveVar(), syncSecret(), LoadFromSecret()
• pkg/context/interface.go - Added interface methods for sensitive vars
• pkg/context/storage.go - Added in-memory Secret storage for testing
• pkg/context/context_test.go - Added 8 tests for sensitive vars functionality
• pkg/hooks/data_passing.go - Routes outputs with sensitive: true to Secret store
• pkg/hooks/data_passing_test.go - Added tests for output routing
• pkg/providers/builtin/workspace.go - Added Sensitive field to VarVars, updated DoVar()
• pkg/providers/builtin/workspace_test.go - Added 2 tests for sensitive DoVar
• pkg/providers/legacy/workspace/workspace.go - Same changes for legacy provider

Usage

Via outputs:

workflow:
  steps:
    - name: read-secret
      type: read-object
      outputs:
        - name: secretData
          valueFrom: output.value.data
          sensitive: true  # Store in Secret instead of ConfigMap

Via DoVar in CUE templates:

#DoVar: op.ws.#DoVar & {
  method: "Put"
  path: "myToken"
  value: secretData
  sensitive: true  # Store in Secret instead of ConfigMap
}

Backward Compatibility

• Existing workflows without sensitive flag continue to work unchanged
• Secret store is created lazily only when sensitive data is stored
• Get operations automatically fall back to ConfigMap if data not in Secret

Testing

• 14 tests pass in pkg/context/... (73% coverage)
• 3 DoVar tests pass in pkg/providers/builtin/... (80% coverage)
• pkg/hooks/... tests will pass after feat: add Sensitive field to OutputItem for secure workflow data storage pkg#133 is merged

Note

⚠️ CI will fail until kubevela/pkg#133 is merged and go.mod is updated. This is expected - raising as draft per maintainer request.

---

Summary by cubic

Store sensitive workflow variables in Kubernetes Secrets instead of ConfigMaps, with lazy Secret creation, clear read/write precedence, and safety fixes to prevent leaking secrets and runtime panics.

• New Features

  • Dual store: non-sensitive in ConfigMap; sensitive in Secret workflow-<name>-context-secret (created lazily with owner refs).
  • Read precedence: Inputs read ConfigMap first, then Secret; op.ws.#DoVar Get reads Secret first, then ConfigMap. Writes route by outputs[].sensitive: true and op.ws.#DoVar Sensitive: true.
  • Context API: GetSensitiveVar/SetSensitiveVar, GetSecretStore, secret sync/load, and in-memory secret support; requires kubevela/pkg #133.

• Bug Fixes

  • Prevent in-memory mode panic by returning early in syncSecret.
  • Consistent secret naming via stored workflowName to avoid owner-ref conflicts.
  • LoadContext uses caller ctx and surfaces non-NotFound errors when loading Secrets.
  • Fix races with RW locks, normalize namespaces in in-memory getters, and make delete match across namespaces.

^{Written for commit f7a178d. Summary will update on new commits.}

[Anaswara-Suresh]

@Paranitharan1110,

Tested this locally. Two bugs came up — could you please fix these and re-test locally?

1. syncSecret() panics when EnableInMemoryContext = true.

cli.Patch(ctx, wf.secretStore, ...) sits after the if/else if, not inside an else, so the in-memory branch falls through into it and tries to Patch a Secret that was never created. Reproduced against a live k3d cluster:

panic: save context to configMap(...): save sensitive context to secret(default/workflow-crash-app-context-secret): secrets "workflow-crash-app-context-secret" not found

2. A Secret gets created for every workflow, sensitive or not.

newContext() seeds secretStore.Data = map[string][]byte{SecretKeyVars: []byte("")} unconditionally, and sync()'s guard is len(wf.secretStore.Data) > 0 — a map-key count, always 1. This contradicts the "Secret store is created lazily only when sensitive data is stored" line in the PR description.

[cubic-dev-ai]

All reported issues were addressed across 9 files

<sub>Tip: instead of fixing issues one by one fix them all with cubic

Re-trigger cubic</sub>

[cubic-dev-ai]

3 issues found across 9 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="pkg/context/storage.go">

<violation number="1" location="pkg/context/storage.go:135">
P3: In-memory secrets for non-default workflow namespaces are never deleted: stored keys always include their namespace, while this condition only matches `default`. Match the secret-name suffix (or accept a namespace) so cleanup behavior matches the method contract.</violation>
</file>

<file name="pkg/context/context.go">

<violation number="1" location="pkg/context/context.go:114">
P1: Sensitive data written through an owner-conflict context is saved to a different Secret than later loads read, so resumed workflows lose those values. Preserve the workflow name independently of the ConfigMap backend name when generating the Secret name.</violation>

<violation number="2" location="pkg/context/context.go:457">
P2: Secret access failures are silently discarded, causing workflows with sensitive inputs to continue with an empty sensitive context and fail as if variables were never written. Ignore only NotFound; return all other retrieval errors.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic</sub>