Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Rewrite parse_intermixed_args() in argparse #125355

Closed
serhiy-storchaka opened this issue Oct 12, 2024 · 3 comments · Fixed by #125356
Closed

Rewrite parse_intermixed_args() in argparse #125355

serhiy-storchaka opened this issue Oct 12, 2024 · 3 comments · Fixed by #125356
Assignees
@serhiy-storchaka
Labels
stdlib Python modules in the Lib dir type-feature A feature request or enhancement

Comments

@serhiy-storchaka
Copy link
Member

serhiy-storchaka commented Oct 12, 2024 •
edited by bedevere-app bot
Loading

parse_intermixed_args() and parse_known_intermixed_args() are implemented by parsing command lines twice -- first only optional arguments, then the remaining is parsed as positional arguments. This approach has some issues.

  • The parser is temporary modified to suppress positional or required optional arguments in these two stages. This is not good, because the parser can no longer be used concurrently. This also smells bad in general, this can hide some bugs.
  • Default values are handled twice.
  • Unknown options in parse_known_intermixed_args() cannot be intermixed with positional arguments. Well, "parsing only known arguments" is a dubious feature, but still...

I tried to rewrite the implementation by moving the code deeper. No longer parser patching, defaults are handled only once, unknown options are excluded from positionals parsing. @hpaulj, @bitdancer, as the authors of the original implementation, could you please review this code?

Linked PRs
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Oct 12, 2024
@serhiy-storchaka
pythongh-125355: Rewrite parse_intermixed_args() in argparse
342eb90 
* The parser is no longer temporary modified and can be used
  concurrently.
* Default values are no long handled twice.
* Report about missed arguments now contains names of all required optional
  and positional arguments.
* Unknown options can be intermixed with positionals in
  parse_known_intermixed_args().
@bedevere-app bedevere-app bot mentioned this issue Oct 12, 2024
Merged
@serhiy-storchaka
Copy link
Member Author

For reference, the original feature was implemented in bpo-14191(gh-58399), PR #3319, 0f6b9d2.

@serhiy-storchaka
Copy link
Member Author

This also fixes support for required mutually exclusive groups containing positionals.

@serhiy-storchaka serhiy-storchaka self-assigned this Oct 12, 2024
@picnixz picnixz added type-feature A feature request or enhancement stdlib Python modules in the Lib dir labels Oct 13, 2024
@savannahostrowski savannahostrowski linked a pull request Oct 18, 2024 that will close this issue
gh-125355: Rewrite parse_intermixed_args() in argparse #125356
Merged
serhiy-storchaka added a commit that referenced this issue Oct 22, 2024
@serhiy-storchaka
gh-125355: Rewrite parse_intermixed_args() in argparse (GH-125356)
759a54d 
* The parser no longer changes temporarily during parsing.
* Default values are not processed twice.
* Required mutually exclusive groups containing positional arguments are
  now supported.
* The missing arguments report now includes the names of all required
  optional and positional arguments.
* Unknown options can be intermixed with positional arguments in
  parse_known_intermixed_args().
@github-project-automation github-project-automation bot moved this to Doc issues in Argparse issues Oct 22, 2024
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Oct 22, 2024
@serhiy-storchaka @miss-islington
pythongh-125355: Rewrite parse_intermixed_args() in argparse (pythonG…
46f2af1 
…H-125356)

* The parser no longer changes temporarily during parsing.
* Default values are not processed twice.
* Required mutually exclusive groups containing positional arguments are
  now supported.
* The missing arguments report now includes the names of all required
  optional and positional arguments.
* Unknown options can be intermixed with positional arguments in
  parse_known_intermixed_args().
(cherry picked from commit 759a54d)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
@bedevere-app bedevere-app bot mentioned this issue Oct 22, 2024
Merged
serhiy-storchaka added a commit that referenced this issue Oct 22, 2024
@miss-islington @serhiy-storchaka
[3.13] gh-125355: Rewrite parse_intermixed_args() in argparse (GH-125356
1fe63b1 
) (GH-125834)

* The parser no longer changes temporarily during parsing.
* Default values are not processed twice.
* Required mutually exclusive groups containing positional arguments are
  now supported.
* The missing arguments report now includes the names of all required
  optional and positional arguments.
* Unknown options can be intermixed with positional arguments in
  parse_known_intermixed_args().
(cherry picked from commit 759a54d)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Oct 22, 2024
@serhiy-storchaka
[3.12] pythongh-125355: Rewrite parse_intermixed_args() in argparse (p…
3dbe0ae 
…ythonGH-125356)

* The parser no longer changes temporarily during parsing.
* Default values are not processed twice.
* Required mutually exclusive groups containing positional arguments are
  now supported.
* The missing arguments report now includes the names of all required
  optional and positional arguments.
* Unknown options can be intermixed with positional arguments in
  parse_known_intermixed_args().
(cherry picked from commit 759a54d)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
@bedevere-app bedevere-app bot mentioned this issue Oct 22, 2024
Merged
serhiy-storchaka added a commit that referenced this issue Oct 22, 2024
@serhiy-storchaka
[3.12] gh-125355: Rewrite parse_intermixed_args() in argparse (GH-125356
da3d81d 
) (GH-125839)

* The parser no longer changes temporarily during parsing.
* Default values are not processed twice.
* Required mutually exclusive groups containing positional arguments are
  now supported.
* The missing arguments report now includes the names of all required
  optional and positional arguments.
* Unknown options can be intermixed with positional arguments in
  parse_known_intermixed_args().

(cherry picked from commit 759a54d)
@aitap aitap mentioned this issue Nov 21, 2024
Closed
@cshimmin cshimmin mentioned this issue Dec 6, 2024
Closed
ebonnal pushed a commit to ebonnal/cpython that referenced this issue Jan 12, 2025
@serhiy-storchaka @ebonnal
pythongh-125355: Rewrite parse_intermixed_args() in argparse (pythonG…
0765a23 
…H-125356)

* The parser no longer changes temporarily during parsing.
* Default values are not processed twice.
* Required mutually exclusive groups containing positional arguments are
  now supported.
* The missing arguments report now includes the names of all required
  optional and positional arguments.
* Unknown options can be intermixed with positional arguments in
  parse_known_intermixed_args().
@Erotemic
Copy link

Wish the internal API for _parse_known_args had a default of intermixed=False so usage of it wasn't broken. This broke scriptconfig. I'm able to work around it on my end though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@serhiy-storchaka serhiy-storchaka

Labels
stdlib Python modules in the Lib dir type-feature A feature request or enhancement
Projects
Argparse issues
Status: Doc issues
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

gh-125355: Rewrite parse_intermixed_args() in argparse serhiy-storchaka/cpython
3 participants
@Erotemic @serhiy-storchaka @picnixz