Repository navigation
YAML store converts inline comments to preceding-line comments on encrypt/edit roundtrip #2130
Description
Hi! I ran into this while setting up unencrypted_comment_regex for a project.
When using unencrypted_comment_regex (or encrypted_comment_regex) with YAML files, inline trailing comments are converted to preceding-line comments after the first encrypt. The feature works correctly - values are encrypted/unencrypted as expected - but the comment position changes permanently.
Input:
DATABASE_HOST: postgres # unencrypted
DATABASE_PORT: "5432" # unencrypted
DATABASE_PASSWORD: secretAfter sops encrypt --in-place and sops decrypt:
# unencrypted
DATABASE_HOST: postgres
# unencrypted
DATABASE_PORT: "5432"
DATABASE_PASSWORD: secretSubsequent sops edit cycles keep the preceding-line format. The docs say "a trailing comment on the same line" is supported, so users expect inline comments to be preserved.
Possible root cause:
Looking at the source, I believe the issue is in stores/yaml/store.go. During parsing (appendYamlNodeToTreeBranch, line 122), value.LineComment is converted to a sops.Comment tree item appended before the TreeItem. On write-back (appendTreeBranch, line 244), accumulated comments are placed as HeadComment on the next key node via addCommentsHead. There doesn't appear to be a path to restore them as LineComment on the original value node.
Expected behavior:
Inline comments should be preserved as inline comments through encrypt/decrypt cycles.
SOPS version: 3.12.2
Thanks for maintaining SOPS!
Activity
knowald commentedon Mar 31, 2026
If my understanding of the root cause is correct and preserving inline comments as
LineCommenton the original value node would be an acceptable approach, I'd be happy to open an MR with a proposed fix. Let me know if that would be welcome or if you'd prefer a different direction.felixfontein commentedon Mar 31, 2026
This is unrelated to
unencrypted_comment_regexorencrypted_comment_regex, but to how SOPS processes comments in general (as you noticed when looking into the YAML store's code).This is basically how SOPS has been working for a long time, and changing this requires a good design of how this should work first.
knowald commentedon Mar 31, 2026
Now that you say it - yeah, you're right. I've run into the comment position shift before with regular YAML comments and somehow completely forgot about it while being focused on the encrypt/unencrypt behavior.
Would you be open to discussing a fix for the general YAML comment roundtrip issue, or should I take this as "better left untouched" and close the issue?
felixfontein commentedon Mar 31, 2026
I'm fine with discussing it. You just have to note that perfect roundtrip preserving won't be possible, since not even the underlying YAML library supports it, and even if it would, I don't think we would adjust the internal format to be too YAML-centric (it also has to work for other formats, i.e. JSON, INI, DotEnv, and in the future TOML and maybe also XML). Generally SOPS' aim isn't to preserve formatting, but to preserve structure. Comments are one big exception; for YAML, comments basically don't count as part of the structure, which is also why most YAML libraries struggle with preserving comments when round-tripping.
knowald commentedon Mar 31, 2026
Looks like the YAML store already reads
HeadComment,LineComment, andFootCommentseparately during parsing - thego.yaml.in/yaml/v3library provides all three onyaml.Node.FootCommentroundtrips correctly atm because its position in the internal tree preserves the distinction.LineCommentdoesn't as it's placed in the same position asHeadComment, so the two become indistinguishable and everything gets written back asHeadComment.I think the minimal fix would be adding a single
Inline boolto theCommentstruct so the distinction survives the conversion? The YAML store would set it when the source is aLineComment, and restore it during write-back instead of collapsing it intoHeadComment. This follows the same pattern as the existingaddCommentsHead/addCommentsFoot- just the missing third piece i think.Defaulting to
falsewould mean no changes, and only code that sets or reads the flag would be the YAML store's parser and writer.Would that kind of scope be acceptable? Happy to put together a PR if so.
Edit: TOML has the same inline vs preceding-line distinction, so a future store could use the same field.
Edit2: Another thought. Making
Commentan interface with format-specific implementations would be cleaner long-term, but that would be massive -walkBranch, the cipher layer and every store would need changes. As for theInlineflag itself, looking at the current INI store, it could be used if someone adds inline comment support there at some point. DotEnv and JSON are unaffected. XML comments are a different beast though - they're standalone nodes in the DOM tree with no attachment to adjacent elements, so they'd break the current model entirely. That would need the interface rework, but i think that's a separate effort.fix: preserve YAML inline comments through encrypt/edit roundtrip
knowald commentedon Apr 1, 2026
I've put together a draft PR as an example of the idea.
#2131
It adds the
Inlinebool, sets it in the YAML store's parser and restores it during write-back via a new helper alongside the existingaddCommentsHead/addCommentsFoothelpers. No changes outside the YAML store except the struct field.Would love to hear if I'm reading the comment handling right and if this direction works for the project.
felixfontein commentedon Apr 7, 2026
After looking at the implementation for some time now, I think this should be fine.
I don't think this would be a good idea. SOPS can also use different formats for input than for output, and having to be able to convert between different kind of comments there would make this way too complex. (Also I don't think it would work well with potential future "format plugins".)
fix: preserve YAML inline comments through encrypt/edit roundtrip