Skip to content

getopts: ---foo or +++foo not treated as options #1053

Open

Description

@stephane-chazelas
$ ksh93 -c 'while getopts "[-+][f:-foo]" opt; do typeset -p opt OPTARG OPTIND; done' myscript ++-foo
opt=+f
OPTARG=1
typeset -i OPTIND=2
$ ksh93 -c 'while getopts "[-+][f:-foo]" opt; do typeset -p opt OPTARG OPTIND; done' myscript --no-foo
opt=f
OPTARG=0
typeset -i OPTIND=2

Are OK but:

$ ksh93 -c 'while getopts "[-+][f:-foo]" opt; do typeset -p opt OPTARG OPTIND; done' myscript ---foo
$ ksh93 -c 'while getopts "[-+][f:-foo]" opt; do typeset -p opt OPTARG OPTIND; done' myscript +++foo
$

Where those ---foo or +++foo are treated as non-option arguments and stop option processing.

Same in things like:

$ ksh -c 'print --+version'
ksh: print: --+version: unknown option
Usage: print [ options ] [string ...]
 Help: print [ --help | --man ] 2>&1

vs

$ ksh -c 'print ---version'
---version

(expecting a unknown option as well)

Activity

  1. stephane-chazelas commented on Oct 11, 2026

    @stephane-chazelas
    Author

    Also note:

    $ ksh93 -c 'while getopts "[-+][f:-foo]" opt; do typeset -p opt OPTARG OPTIND; done' myscript --foo
    opt=f
    OPTARG=1
    typeset -i OPTIND=2
    

    As if [f:-foo] was treated as [f:foo] though there's still a difference in:

    $ ksh93 -c 'while getopts "[-+][f:foo]" opt; do typeset -p opt OPTARG OPTIND; done' myscript ++--foo
    myscript: ++--foo: section not found
    opt='?'
    typeset -i OPTIND=2
    $ ksh93 -c 'while getopts "[-+][f:-foo]" opt; do typeset -p opt OPTARG OPTIND; done' myscript ++--foo
    opt=+f
    OPTARG=1
    typeset -i OPTIND=2
    
  2. stephane-chazelas commented on Oct 11, 2026

    @stephane-chazelas
    Author

    It rather looks -s are ignored to some extent as long as there aren't more than one in sequence.

    [f:foobar] for instance alllows --foo-bar but not --foo--bar nor ---foobar (treated as non-option which is my main issue here), while [f:foo-bar] allows --foobar, --foo-bar and --foo--bar.

  3. McDutchie commented on Oct 12, 2026

    @McDutchie

    It rather looks -s are ignored to some extent as long as there aren't more than one in sequence.

    That is a feature, not a bug. It's similar with ksh long-form shell options (although they consistently ignore all dashes except in the no prefix).

  4. stephane-chazelas commented on Oct 12, 2026

    @stephane-chazelas
    Author

    Yes, I agree ignoring -s looks like a feature (even if undocumented AFAICT like for the no/NO prefix) and there appears to be some support (intentional or accidental I don't know) to support both --foobar and --foo-bar if that's the user 's intention, but to me ---foobar being treated as non-options is a bug. The only thing starting with - that should not be treated as an option should be the -- option delimiter (and discarded).

  5. stephane-chazelas commented on Oct 12, 2026

    @stephane-chazelas
    Author

    FYI, the few getopts-related issues I've raised lately were as a result of trying to answer https://unix.stackexchange.com/questions/482119/does-ksh93-support-o-type-options-with-its-getopts1-builtin

    Some of the findings I mention there may be worth adding to the documentation.

    Shells such as bash, zsh or fish have information in yold / texinfo / (not sure for fish) format, which results in many pages (info / html / man) on the backends. ksh93 documentation was split into several pages as being part of ast-open, and there are terse builtin --man pages for most builtins. Is there a plan to consolidate all that into some more global documentation that one could refer to?

  6. McDutchie commented on Oct 13, 2026

    @McDutchie

    Is there a plan to consolidate all that into some more global documentation that one could refer to?

    Some others already had a similar thought. See #902. That project seems to have gone quiet though.

    My plate is more than full with fixing endless bugs. I'm not even getting to the new feature development I've wanted to do for years, and quite frankly, I'm worn out.

    Any other work people want done, they'll have to do themselves, and submit a PR.

  7. McDutchie commented on Oct 13, 2026

    @McDutchie

    Looks like the reported behaviour was intentional. In src/lib/libast/RELEASE:

    08-03-06 misc/optget.c: ---* and +++* are now operands
    
  8. McDutchie commented on Oct 13, 2026

    @McDutchie

    The offending code:

    else if (*s == c)
    {
    /*
    * ---* or +++* are operands
    */
    return 0;
    }

    So, since it was intentional, there are three options:

    1. keep it as it is;
    2. revert for POSIX mode only;
    3. revert outright.
  9. stephane-chazelas commented on Oct 13, 2026

    @stephane-chazelas
    Author

    Looks like the reported behaviour was intentional. In src/lib/libast/RELEASE:
    08-03-06 misc/optget.c: ---* and +++* are now operands

    Good find.

    So, since it was intentional, there are three options:

    1. keep it as it is;

    2. revert for POSIX mode only;

    3. revert outright.

    I can't imagine why anyone would want, need or rely on that behaviour. It is quite surprising.

    Having ---foobar for undocumented / hidden options is somewhat common. See for instance https://unix.stackexchange.com/questions/778721/convention-of-parameter-naming-undocumented-options/

    One may/might want to use that in ksh scripts, if only as wrappers to those other commands that handle ---options (though that seems very unlikely).

    I don't think POSIX has any bearing on this. I'd expect it would be all unspecified behaviour there. As far as the getopts spec is concerned, "The use of other option characters that are not alphanumeric produces unspecified results." would cover it. As for any some-builtin-specified-by-POSIX ---qwe not returning an error, again I'd expect it would be unspecified.

    If I had to venture a guess, I'd say maybe that change was introduced so as not to have to try and reconciliate the -s somewhat ignored in long option names and allowing users to specify options with - in their name. You might find that reverting that change introduces all sorts of different bugs.

  10. McDutchie commented on Oct 13, 2026

    @McDutchie

    I can't imagine why anyone would want, need or rely on that behaviour. It is quite surprising.

    Agreed.

    If I had to venture a guess, I'd say maybe that change was introduced so as not to have to try and reconciliate the -s somewhat ignored in long option names and allowing users to specify options with - in their name. You might find that reverting that change introduces all sorts of different bugs.

    Or then again it might be just fine, because the AST/ksh folks have a long and distinguished history of committing intentional dysfunctionality in their code for no discernible reason. My list of bugs fixed purely by removing code now has 41 items on it and this might be the 42nd. Only one way to find out... remove it and see what happens.

  11. McDutchie commented on Oct 13, 2026

    @McDutchie

    FWIW, after removing it, there are 0 regression test failures (so they didn't even add a regression test for this).

  12. McDutchie commented on Oct 13, 2026

    @McDutchie

    After fixing this, there could be incompatibilities if a script does 'set $foo' with foo containing a first field that starts with at least three dashes.

    So, to be on the safe side, this should probably be fixed on the dev branch only.

  13. added
    1.1Issue relevant to dev branch (v1.1.*)
    on Oct 13, 2026
  14. McDutchie commented on Oct 13, 2026

    @McDutchie

    On the other hand... my policy item 4 says: "No 100% bug compatibility. Broken and undocumented behaviour gets fixed."

    They listed the change, extremely tersely, in the RELEASE file, and nowhere else. Does that count as "documented"? It's iffy.

  15. McDutchie commented on Oct 13, 2026

    @McDutchie

    By the way, I found an undocumented feature: terse columnar format for the current shell options state.

    $ ksh -c 'set -o -'
    noallexport       backslashctrl     nobgnice          braceexpand       clobber           noemacs           noerrexit
    exec              nofunctrace       glob              noglobcasedetect  noglobstar        nogmacs           nohistexpand
    nohistreedit      nohistverify      noignoreeof       nointeractive     nokeyword         noletoctal        log
    nologin_shell     nomarkdirs        nomonitor         multiline         nonotify          nopipefail        noposix
    noprivileged      norc              norestricted      noshowme          trackall          unset             noverbose
    novi              noviraw           noxtrace          
    

    Implemented in args.c, lines 153-160, 314-315, and 572-618.

    After removing those getopts.c lines that feature also works via set ---.

  16. removed
    1.1Issue relevant to dev branch (v1.1.*)
    on Oct 13, 2026
  17. McDutchie commented on Oct 13, 2026

    @McDutchie

    I don't think POSIX has any bearing on this. I'd expect it would be all unspecified behaviour there. As far as the getopts spec is concerned, "The use of other option characters that are not alphanumeric produces unspecified results." would cover it.

    Maybe. Another way to look at it is that, POSIXly, either a -- argument or any argument not starting with - terminates options and everything else is an operand, and the change in question violates that.

  18. stephane-chazelas commented on Oct 14, 2026

    @stephane-chazelas
    Author

    Maybe. Another way to look at it is that, POSIXly, either a -- argument or any argument not starting with - terminates options and everything else is an operand, and the change in question violates that.

    Ah yes, sorry, you're right. I was only considering getopts users wanting to specify a - option but of course POSIX requires:

    getopts :o opt ---foo; echo "$? $opt $OPTARG"
    

    For instance to output 0 ? -, whilst ksh outputs 1 ?.

    But then again, that also applies to:

    $ ksh --posix -c 'getopts o opt --help; echo "$? $opt $OPTARG"'
    Usage: ksh [-o]
     Help: ksh [ --help | --man ] 2>&1
    

    Even though in that case, that --help automated handling is actually useful even if breaking POSIX compliance.

    BTW:

    $ ksh -c 'getopts :o opt --usage; echo "$? $opt $OPTARG"'
    0 ? ?
    

    I suppose that's because --usage is short for --??usage.

    I see some changes in that regards on current dev version:

    $ ksh --posix -c 'getopts o opt --help; echo "$? $opt $OPTARG"'
    ksh: --: unknown option
    0 ?
    $ ksh --posix -c 'getopts o opt ---foo; echo "$? $opt $OPTARG"'
    ksh: --: unknown option
    0 ?
    

    While this is still a problem:

    $ ksh -c 'getopts :o opt --usage; echo "$? $opt $OPTARG"'
    0 ? ?
    
  19. stephane-chazelas commented on Oct 14, 2026

    @stephane-chazelas
    Author

    Also:

    $ ksh --posix -c 'getopts o o -é'
    ksh: -�: unknown option
    
    $ ksh --posix -c 'getopts é o -é; echo "$? $o"'
    0 �
    

    (where � is my terminal replacing the first byte of the encoding of é).

    But then again, all shells I tried (and all utilities using the getopt() / getopt_long() APIs at least on my Debian system) except yash have the same problem. Likely an issue to clarify at POSIX first given that how https://pubs.opengroup.org/onlinepubs/9799919799.2024edition/functions/getopt.html would cope with multibyte characters is not very clear.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething is not working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions