Problem using sops with passphrase-protected age key file

Hey there,

I am currently trying out sops with the freshly released version 1.0.0 of age.
Creating passphrase-protected age key files seems rather easy:
https://github.com/FiloSottile/age#passphrase-protected-key-files

However, when I try to use these with sops I get the following error:

# sops values.enc.yaml
Failed to get the data key required to decrypt the SOPS file.

Group 0: FAILED
  age18v8fr5zpnq84u9pque8ey7sdng3j79qnc7jk2p38r3wsza9dgufszjufnn: FAILED
    - | error at line 1: malformed secret key: separator '1' at
      | invalid position: pos=20, len=21

Recovery failed because no master key was able to decrypt the file. In
order for SOPS to recover the file, at least one key has to be successful,
but none were.

Am I missing something or is this currently not supported by sops?
I have not yet found any sops issues or documentation on this topic.

So, can anyone help on this?

since 086c11d (3.7.2 #1006 ), there exists a env-var which should enable an interactive password prompt via SOPS_AGE_KEY=$(age -d ~/.config/sops/age/keys.age) sops secrets.yml

since 086c11d (3.7.2 #1006 ), there exists a env-var which should enable an interactive password prompt via SOPS_AGE_KEY=$(age -d ~/.config/sops/age/keys.age) sops secrets.yml

This is definitely a work around as the documentation for it isn't included in the sops command help output.

Are passphrase encrypted AGE secret keys designated an anti-pattern for sops usage? The documentation doesn't say much about how AGE should be used with sops, and this env var exists so I'm thinking the answer is no here.

If not, I believe the user experience should be improved here by automatically prompting for the key decryption passphrase much like age itself does. I believe consistency in behavior here is what most users familiar with age would expect.

An alternative approach: decrypt to a file on tempfs (won't be written to disk, will disappear on restart), schedule the file to be removed some time in the future. The following works on linux with systemd:

# decrypt to an ephemeral file (will be asked to enter the passphrase)
age -d -o /run/shm/keys.txt ~/.config/sops/age/keys

# schedule the file deletion in 1 hour from now
systemd-run --user -u age-clean --on-calendar "@$(date -d "+1hour" +%s)" rm /run/shm/keys.txt

The command's are tedious but can be integrated in some kind of workflow and sops needs to be configured to look for private keys in /run/shm/keys.txt (e.g. by making the default path - ~/.config/sops/age/keys a symlink to the ephemeral path /run/shm/keys.txt).

The benefit of this approach is that it can be used with the editor-sops integration (as long as you keep recreating the ephemeral decrypted key file).

Disclaimer: I have just thought of this right now and haven't really tested it yet.

since 086c11d (3.7.2 #1006 ), there exists a env-var which should enable an interactive password prompt via SOPS_AGE_KEY=$(age -d ~/.config/sops/age/keys.age) sops secrets.yml

For me this isn't working anymore since sops 3.8.0. Regardless if the SOPS_AGE_KEY env var is exported or put directly in front of the binary.

Failed to get the data key required to decrypt the SOPS file.

Group 0: FAILED
  age1w8egqmyze2hrf2vcyjkxylny9qg8nw8rp9ux9kl9y95rrruthdmqxlzwpp: FAILED
    - | failed to load age identities: failed to parse
      | '/home/USER/.config/sops/age/keys.txt' age identities: error
      | at line 1: malformed secret key: separator '1' at invalid
      | position: pos=20, len=21

Recovery failed because no master key was able to decrypt the file. In order for SOPS to recover the file, at least one key has to be successful, but none were.

Update:

This seems to happen because setting SOPS_AGE_KEY doesn't skip loading the other keysources.
Sops always looks into ~/.config/sops/age/keys.txt and if this file exists and is encrypted the decryption obviously fails.
So just moving/renaming the keys.txt file is enough to workaround this 🥳

When using the latest version of SOPS v3.8.1, I too notice the issue where sops fails to read the SOPS_AGE_KEY environment variable. SOPS version 3.7.3 does not exhibit this problem. The issue persists even when you remove the key files (~/.config/sops/age/keys.txt or ~/Library/Application\ Support/sops/age/keys.txt).

➜  SOPS_AGE_KEY=$(age -d ~/.config/sops/age/age-key.txt)

➜  ~/apps/sops/sops-v3.7.3.darwin.arm64 -d secret.enc.json|cat
───────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ STDIN
───────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ {
   2   │     "password": "42"
   3   │ }
───────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

➜  ~/apps/sops/sops-v3.8.1.darwin.arm64 -d secret.enc.json
Failed to get the data key required to decrypt the SOPS file.

Group 0: FAILED
  age1zwzgl7q9l6hzhl4f2gqgcxc83gfgj3t932fw4wgrgluvxf6g7s0q6345vf: FAILED
    - | failed to load age identities: failed to open
      | SOPS_AGE_KEY_FILE file: open : no such file or directory

  age1spwwvmnwyhe7urhckwwjejumcy45sch4htmrjur2g4ektetphulqysj7hm: FAILED
    - | failed to load age identities: failed to open
      | SOPS_AGE_KEY_FILE file: open : no such file or directory

Recovery failed because no master key was able to decrypt the file. In
order for SOPS to recover the file, at least one key has to be successful,
but none were.

➜

When using the latest version of SOPS v3.8.1, I too notice the issue where sops fails to read the SOPS_AGE_KEY environment variable. SOPS version 3.7.3 does not exhibit this problem. The issue persists even when you remove the key files (~/.config/sops/age/keys.txt or ~/Library/Application\ Support/sops/age/keys.txt).

The issue you are experiencing is not related to sops.
If you want sops to use SOPS_AGE_KEY you need to set it in your environment using export or put SOPS_AGE_KEY directly in front of the binary name!

Still getting the same error when exporting or adding the environment variable before the binary:

➜  export SOPS_AGE_KEY=$(age -d ~/.config/sops/age/age-key.txt)

➜  ~/apps/sops/sops-v3.7.3.darwin.arm64 -d secret.enc.json|cat
───────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ STDIN
───────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ {
   2   │     "password": "42"
   3   │ }
───────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

➜  ~/apps/sops/sops-v3.8.1.darwin.arm64 -d secret.enc.json
Failed to get the data key required to decrypt the SOPS file.

Group 0: FAILED
  age1zwzgl7q9l6hzhl4f2gqgcxc83gfgj3t932fw4wgrgluvxf6g7s0q6345vf: FAILED
    - | failed to load age identities: failed to open
      | SOPS_AGE_KEY_FILE file: open : no such file or directory

  age1spwwvmnwyhe7urhckwwjejumcy45sch4htmrjur2g4ektetphulqysj7hm: FAILED
    - | failed to load age identities: failed to open
      | SOPS_AGE_KEY_FILE file: open : no such file or directory

Recovery failed because no master key was able to decrypt the file. In
order for SOPS to recover the file, at least one key has to be successful,
but none were.

➜  SOPS_AGE_KEY=$(age -d ~/.config/sops/age/age-key.txt) ~/apps/sops/sops-v3.8.1.darwin.arm64 -d secret.enc.json
Failed to get the data key required to decrypt the SOPS file.

Group 0: FAILED
  age1zwzgl7q9l6hzhl4f2gqgcxc83gfgj3t932fw4wgrgluvxf6g7s0q6345vf: FAILED
    - | failed to load age identities: failed to open
      | SOPS_AGE_KEY_FILE file: open : no such file or directory

  age1spwwvmnwyhe7urhckwwjejumcy45sch4htmrjur2g4ektetphulqysj7hm: FAILED
    - | failed to load age identities: failed to open
      | SOPS_AGE_KEY_FILE file: open : no such file or directory

Recovery failed because no master key was able to decrypt the file. In
order for SOPS to recover the file, at least one key has to be successful,
but none were.

➜

Did you set the SOPS_AGE_KEY_FILE env var? Try unsetting it.

For me everything is working as expected, which means something is wrong on your end.

The SOPS_AGE_KEY_FILE env var is not set. If I use/set the SOPS_AGE_KEY_FILE env var with the decrypted (i.e. non password-protected) key file then the decryption works fine in both versions.

Are you 100% sure that the SOPS_AGE_KEY_FILE isn't set because the error failed to open SOPS_AGE_KEY_FILE can only be returned if SOPS_AGE_KEY_FILE is set to a non empty string value.

sops/age/keysource.go

Lines 248 to 255 in b6d3c97

    
           if ageKeyFile, ok := os.LookupEnv(SopsAgeKeyFileEnv); ok { 
        
           	f, err := os.Open(ageKeyFile) 
        
           	if err != nil { 
        
           		return nil, fmt.Errorf("failed to open %s file: %w", SopsAgeKeyFileEnv, err) 
        
           	} 
        
           	defer f.Close() 
        
           	readers[SopsAgeKeyFileEnv] = f 
        
           }

As sure as I can be :)

➜  unset SOPS_AGE_KEY ; unset SOPS_AGE_KEY_FILE

➜  export SOPS_AGE_KEY_FILE= ; export SOPS_AGE_KEY= ;

➜  echo $SOPS_AGE_KEY_FILE $SOPS_AGE_KEY


➜  export SOPS_AGE_KEY=$(age -d ~/.config/sops/age/age-key.txt)

➜  ~/apps/sops/sops-v3.7.3.darwin.arm64 -d secret.enc.json|cat
───────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ STDIN
───────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ {
   2   │     "password": "42"
   3   │ }
───────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

➜  SOPS_AGE_KEY=$(age -d ~/.config/sops/age/age-key.txt) ~/apps/sops/sops-v3.8.1.darwin.arm64 -d secret.enc.json
Failed to get the data key required to decrypt the SOPS file.

Group 0: FAILED
  age1zwzgl7q9l6hzhl4f2gqgcxc83gfgj3t932fw4wgrgluvxf6g7s0q6345vf: FAILED
    - | failed to load age identities: failed to open
      | SOPS_AGE_KEY_FILE file: open : no such file or directory

  age1spwwvmnwyhe7urhckwwjejumcy45sch4htmrjur2g4ektetphulqysj7hm: FAILED
    - | failed to load age identities: failed to open
      | SOPS_AGE_KEY_FILE file: open : no such file or directory

Recovery failed because no master key was able to decrypt the file. In
order for SOPS to recover the file, at least one key has to be successful,
but none were.

➜  export SOPS_AGE_KEY_FILE=~/.config/sops/age/key.txt

➜  ~/apps/sops/sops-v3.7.3.darwin.arm64 -d secret.enc.json|cat
───────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ STDIN
───────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ {
   2   │     "password": "42"
   3   │ }
───────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

➜  ~/apps/sops/sops-v3.8.1.darwin.arm64 -d secret.enc.json|cat
───────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ STDIN
───────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ {
   2   │     "password": "42"
   3   │ }
───────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

➜

With export SOPS_AGE_KEY_FILE= you are setting the envvar to an empty value but it needs to be unset.
To verify if a envvar is unset you can do something like this env | grep SOPS_AGE_KEY_FILE.

You were correct, it was my bad between unset vs export. Once I only unset the env var that worked. Thanks for helping with that!

It should be fixed by #1400

@tomaszduda23

Thanks for adding that feature @tomaszduda23 !
It seems though the solution is forcing the password via terminal input, right? The variable used in your contribution (SOPS_AGE_PASSWORD) is only respected during testing and I haven't figured out how to pass the passphrase to age in any other method.

@tomaszduda23

@tomaszduda23

This is still a problem, because if ~/.config/sops/age/keys.txt is encrypted sops will still refuse to try to decrypt it.

It only works if you pass it manually like this: SOPS_AGE_KEY_FILE=~/.config/sops/age/keys.txt sops edit secrets.json.

I’m not sure why it isn’t working on your side. In my case, it appears encrypted and GNOME simply prompts me for a password.

cat .config/sops/age/keys.txt 
-----BEGIN AGE ENCRYPTED FILE-----

Both ~/.config/sops/age/keys.txt and the file pointed to by SOPS_AGE_KEY_FILE are processed by the same piece of code, so if one works, the other should work as well. And if one doesn't, neither should.

This is how I produced the issue:

$ sops --version
sops 3.12.1

$ age-keygen | age --armor --passphrase >~/.config/sops/age/keys.txt
Public key: age16tpaw4svm8z97vnx8yzgzrmggd5p0925v8yuwvuh0gfwp6928gwqz6hr0h

$ head --lines 1 ~/.config/sops/age/keys.txt
-----BEGIN AGE ENCRYPTED FILE-----

$ echo "FOO=BAR" | sops encrypt --age age16tpaw4svm8z97vnx8yzgzrmggd5p0925v8yuwvuh0gfwp6928gwqz6hr0h --filename-override secrets.env >secrets.env

$ gpgconf --reload gpg-agent

$ sops edit secrets.env
Failed to get the data key required to decrypt the SOPS file.

Group 0: FAILED
  age16tpaw4svm8z97vnx8yzgzrmggd5p0925v8yuwvuh0gfwp6928gwqz6hr0h: FAILED
    - | failed to create reader for decrypting sops data key with
      | age: failed to decrypt identity file: could not read
      | passphrase: gpg-agent passphrase request errored: ERR
      | 67109144 IPC parameter error <GPG Agent> - invalid length of
      | cacheID
      | . Did not find keys in locations
      | 'SOPS_AGE_SSH_PRIVATE_KEY_FILE',
      | 'SOPS_AGE_SSH_PRIVATE_KEY_CMD', '/home/$USER/.ssh/id_rsa',
      | 'SOPS_AGE_KEY', 'SOPS_AGE_KEY_FILE', and 'SOPS_AGE_KEY_CMD'.

Recovery failed because no master key was able to decrypt the file. In
order for SOPS to recover the file, at least one key has to be successful,
but none were.

$ SOPS_AGE_KEY_FILE=~/.config/sops/age/keys.txt sops edit secrets.env
File has not changed, exiting.

Am I doing something wrong here?

Thanks for providing a reproducer. For me, running sops edit secrets.env already pops up the password dialog from GPG agent, same as SOPS_AGE_KEY_FILE=~/.config/sops/age/keys.txt sops edit secrets.env. (I do use SOPS 3.12.2 though, not 3.12.1. But that shouldn't make a difference.)

I think you're hitting #1977 though. This can happen if ~ expands to something too long. SOPS_AGE_KEY_FILE is not affected by this since the identity string is not the path, but SOPS_AGE_KEY_FILE.

Thank you, that is probably the issue. My config path is only 40 chars long, so it's odd that I ended up hitting this issue:

$ wc --chars <<<~/.config/sops/age/keys.txt
40

Is the max length for the cache key documented anywhere?

I haven't found any documentation, but I found the piece of code that rejects too long ones. Unfortunately that link is currently broken to a broken server config: https://dev.gnupg.org/source/gnupg/browse/master/agent/command.c$2082

Fortunately there's a mirror on GitHub of the repository: https://github.com/gpg/gnupg/blob/master/agent/command.c#L2082
The constant in that line hasn't been modified since it was introduced in gpg/gnupg@eac306f in 2002, according to the git history.

BTW, you need to use wc --bytes instead of wc --chars. The number of UTF-8 encoded bytes is what matters. This doesn't make a difference if you stick to ASCII characters, but makes a potentially big difference if you use non-ASCII characters in your username, for example.

I was not having issues with an age-encrypted sops keys.txt file until recently. I've just confirmed that there is a regression bug with the latest version 3.13.0. See below:

Problem:

$ sops apps/eg/helm_secrets.yaml
Failed to get the data key required to decrypt the SOPS file.

Group 0: FAILED
  age1hyw8w5vnp0kxrrcdd02f30r3euyszyaxs04qnw8d0jnmf9vuhyuq6qz9k8: FAILED
    - | failed to create reader for decrypting sops data key with
      | age: failed to decrypt identity file: could not read
      | passphrase: gpg-agent passphrase request errored: ERR
      | 67109144 IPC parameter error <GPG Agent> - invalid length of
      | cacheID
      | . Did not find keys in locations
      | 'SOPS_AGE_SSH_PRIVATE_KEY_FILE',
      | 'SOPS_AGE_SSH_PRIVATE_KEY_CMD', '/home/user/.ssh/id_rsa',
      | 'SOPS_AGE_KEY', 'SOPS_AGE_KEY_FILE', and 'SOPS_AGE_KEY_CMD'.
$ sops --version
sops 3.13.0 (latest)

Solution:

$ sudo downgrade sops
[sudo] password for user: 
loading packages...
warning: downgrading package sops (3.13.0-1 => 3.12.2-1)
resolving dependencies...
looking for conflicting packages...

Packages (1) sops-3.12.2-1

Total Installed Size:  48.85 MiB
Net Upgrade Size:      -0.78 MiB

:: Proceed with installation? [Y/n] 
(1/1) checking keys in keyring                                                                                                                                            [##########################################################################################################] 100%
(1/1) checking package integrity                                                                                                                                          [##########################################################################################################] 100%
(1/1) loading package files                                                                                                                                               [##########################################################################################################] 100%
(1/1) checking for file conflicts                                                                                                                                         [##########################################################################################################] 100%
(1/1) checking available disk space                                                                                                                                       [##########################################################################################################] 100%
:: Processing package changes...
(1/1) downgrading sops                                                                                                                                                    [##########################################################################################################] 100%
:: Running post-transaction hooks...
(1/1) Arming ConditionNeedsUpdate...
add sops to IgnorePkg? [y/N] 
$ sops --version
sops 3.12.2
$ sops -d apps/eg/helm_secrets.yaml
REDACTED SECRETS

Interesting, as 3.13.0 contains a fix to make sure the ID is never too long (#2145). Which version of GPG / gpg-agent are you using?

If you don't mind trying around a bit, you could compile SOPS from source and adjust the :30 in https://github.com/getsops/sops/blob/main/age/encrypted_keys.go#L130 to - say - :20, and see whether that helps. (If you clone the repo, you can run make vendor to download the dependencies, and then make install to compile SOPS and copy the binary to ~/go/bin/sops.)

While GPG agent accepts a cache ID of length <= 50, it seems that our GPG agent code is percent-encoding the cache ID (https://
github.com/getsops/gopgagent/blob/master/gpgagent.go#L177, https://pkg.go.dev/net/url#QueryEscape), and since our cache ID contains Base64-encoded data (https://github.com/getsops/sops/blob/main/age/encrypted_keys.go#L130C59-L130C73), it is likely possible that our 7 + 40 character cache key is exploded by percent encoding to something > 50 characters - basically depending on how many / and + show up in the Base64 encoding... My test keys obviously didn't have too many... 🤦

I guess it's better to use another encoding that doesn't produce anything that's percent encoded...

$ gpg --version
gpg (GnuPG) 2.4.9
libgcrypt 1.12.2

Yup, changing that to 20 worked.

#2180 should fix this.

Navigation Menu

Repository navigation

Problem using sops with passphrase-protected age key file #933

Update:

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	if ageKeyFile, ok := os.LookupEnv(SopsAgeKeyFileEnv); ok {
	f, err := os.Open(ageKeyFile)
	if err != nil {
	return nil, fmt.Errorf("failed to open %s file: %w", SopsAgeKeyFileEnv, err)
	}
	defer f.Close()
	readers[SopsAgeKeyFileEnv] = f
	}

Problem using sops with passphrase-protected age key file #933

Description

Activity

skuethe commented on Jan 29, 2022

pogoba commented on Mar 28, 2022

jmpolom commented on Feb 8, 2023

artm commented on Apr 10, 2023

DerRockWolf commented on Oct 1, 2023

Update:

sidhanti-ctl commented on Dec 5, 2023

DerRockWolf commented on Dec 5, 2023

sidhanti-ctl commented on Dec 5, 2023

DerRockWolf commented on Dec 5, 2023

sidhanti-ctl commented on Dec 5, 2023

DerRockWolf commented on Dec 5, 2023

sidhanti-ctl commented on Dec 5, 2023

DerRockWolf commented on Dec 5, 2023

sidhanti-ctl commented on Dec 5, 2023

tomaszduda23 commented on May 3, 2025

skuethe commented on Jul 8, 2025

musjj commented on Apr 7, 2026

tomaszduda23 commented on Apr 7, 2026

felixfontein commented on Apr 7, 2026

musjj commented on Apr 7, 2026

felixfontein commented on Apr 7, 2026

musjj commented on Apr 7, 2026

felixfontein commented on Apr 8, 2026

felixfontein commented on Apr 8, 2026

3uzbcqje commented on May 11, 2026

felixfontein commented on May 14, 2026

3uzbcqje commented on May 14, 2026

felixfontein commented on May 15, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions