Skip to content

Feature: file-based secrets from environment - #3568

Open
parasite-lost wants to merge 1 commit into
opencloud-eu:mainfrom
parasite-lost:feat_secrets_secure_provisioning_by_file_env
Open

parasite-lost wants to merge 1 commit into
opencloud-eu:mainfrom
parasite-lost:feat_secrets_secure_provisioning_by_file_env

Conversation

@parasite-lost

Copy link
Copy Markdown

Description

Add support for all environment variables providing sensitive values to provide a path to a file via _FILE suffixed environment variable instead where the file contains the sensitive value.

This can be used for example in conjunction with container secrets or systemd credentials to securely provide sensitive values.

Related Issue

This is also related to #3034 (PR #3034).

Motivation and Context

Securely provide sensitive values by files (referenced via environment) instead of leaking sensitive values in the environment.

How Has This Been Tested?

  • added several unittests for envdecode

Types of changes

While maintaining full backwards compatibility (all prior options for configuring parameters haven't changed):

  • Minor refactoring
  • Support _FILE suffixed environment variables via envdecode struct tags (by adding ,file parameter to the environment variable definition)
  • Add support for all secrets, api keys, tokens, passwords that can be configured via the environment that I found

Checklist:

  • Code changes
  • Unit tests added
  • Acceptance tests added
  • Documentation for envdecode
  • Documentation added

@codacy-production

codacy-production Bot commented Sep 20, 2026 •

Copy link
Copy Markdown

Not up to standards ⛔

🔴 Issues 1 critical

Alerts:
⚠ 1 issue (≤ 0 issues of at least minor severity)

Results:
1 new issue

Category Results
Security 1 critical

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

if file == "" {
return "", true, fmt.Errorf("no file provided: %s=", envVariable)
}
content, err := os.ReadFile(file)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codacy Static Code Analysis complains that this line is not up to standards as it would allow reading user-defined files.

But this is exactly the intent - let the user (admin in this case) provide paths to arbitrary files that should be read to provide sensitive values.

question: how can I suppress this finding?

@dschmidt

Copy link
Copy Markdown
Contributor

To my eyes the comma syntax is a bit weird, it looks like a list (especially when there's only one env var)

Two thoughts on this:

  1. You can already avoid having secrets as env vars by using config files
  2. If adding an additional mechanism for handling _FILE in a generic way, why limit it to specific env vars? Why not handle it for all env vars? Then you don't need to invent a new declaration syntax for that

@parasite-lost

Copy link
Copy Markdown
Author

@dschmidt

To my eyes the comma syntax is a bit weird, it looks like a list (especially when there's only one env var)

That's the syntax of envdecode struct tags. You can already define default values by appending the definition with ,default=my-default-value or appending ,required to denote that the variable must be provided. I just extended this with ,file.

Two thoughts on this:

1. You can already avoid having secrets as env vars by using config files

Using config files feels a bit unwieldy and verbose as (if I understand things correctly) you have to provide the full configuration including default settings. Plus the documentation states that the aim is that opencloud can easily be configured by setting just the necessary parameters via environment variables.

2. If adding an additional mechanism for handling _FILE in a generic way, why limit it to specific env vars? Why not handle it for all env vars? Then you don't need to invent a new declaration syntax for that

I'm fine with adding the _FILE suffix for all environment variables - if that is preferable (it would make things much easier, code and documentation).

@aduffeck aduffeck left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I only really see a use case for file-based keys and secrets I would also prefer to support the _FILE vars in general for simplicity. That also avoids issues with forgotten file tags in the future and os.LookupEnv is cheap enough to not bother I think.

As a micro optimization we could maybe spare a few cycles by checking the "regular" env vars first and only fall back to the _FILE variant if it isn't set, though.

Support providing configuration parameters file-based via the
environment in envdecode.

For every environment variable supported by any opencloud component
support additionally the same environment variable suffixed with `_FILE`
to retrieve the configuration parameter from file instead of directly
from the environment.

This allows configuring opencloud components more securely, preventing
sensitive values being leaked via the environment and providing
sensitive values for example via systemd credentials or file-based
container secrets that can be encrypted at rest.

Example: instead of setting `MY_SENSITIVE_VALUE=secret-password` in the
environment you can now set
`MY_SENSITIVE_VALUE_FILE=/run/secrets/sensitive-value` where
`/run/secrets/sensitive-value` (arbitrary path) is a file containing
`secret-password` (trailing newlines are ignored).
@parasite-lost
parasite-lost force-pushed the feat_secrets_secure_provisioning_by_file_env branch from 5ee79f4 to 04245f1 Compare September 27, 2026 13:26
@parasite-lost

parasite-lost commented Sep 27, 2026 •

Copy link
Copy Markdown
Author

@aduffeck

While I only really see a use case for file-based keys and secrets I would also prefer to support the _FILE vars in general for simplicity. That also avoids issues with forgotten file tags in the future and os.LookupEnv is cheap enough to not bother I think.

As a micro optimization we could maybe spare a few cycles by checking the "regular" env vars first and only fall back to the _FILE variant if it isn't set, though.

Works for me - this makes everything much simpler. Indeed, there are several environment variables that are repeatedly parsed by different components - missing one ,file tag somewhere would cause nefarious bugs with different components whose configuration does not match.

There are however a couple of environment variables that already reference files, such as OC_GRPC_TLS_CERTIFICATE. Luckily, none of them ends with _FILE so this shouldn't cause too much confusion.

I've adjusted my changes to code and (envdecode) documentation accordingly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support file-based secrets from environment

3 participants