Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mutually exclusive group w/ all parameters argparse.SUPPRESS'd causes an assert #98666

Closed
kenyee opened this issue Oct 25, 2022 · 2 comments
Closed
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@kenyee
Copy link

kenyee commented Oct 25, 2022

Bug report

state_group = parser.add_mutually_exclusive_group()
state_group.add_argument('--param1', nargs='?', const='default', metavar='NAME',
                         help=SUPPRESS)
state_group.add_argument('--param2', nargs='?', const='default', metavar='NAME',
                         help=SUPPRESS)

will cause an assert in
assert ' '.join(opt_parts) == opt_usage
of argparse.py

because the empty group [] is converted into two spaces
e.g.
[group1] [] [group 2]
gets converted to this by the opt_usage = format(optionals, groups) code above the assert:
[group1] [group 2]
and the two spaces don't match the single space of the assert

Your environment

  • CPython versions tested on: 3.10.4
  • Operating system and architecture: OSX, arm64
@kenyee kenyee added the type-bug An unexpected behavior, bug, or error label Oct 25, 2022
@AlexWaygood AlexWaygood added the stdlib Python modules in the Lib dir label Oct 25, 2022
@hpaulj
Copy link

hpaulj commented Nov 4, 2022

The usage formatting is known for being brittle, especially when working with the extra notation of mutually_exclusive_groups. This assert was intended to catch formatting errors that would cause problems if the usage has to be split onto several lines. Without getting into the details, I'm not surprised that a rarely used feature like SUPPRESS causes problems.

hamdanal added a commit to hamdanal/cpython that referenced this issue May 28, 2023
Rationale
=========

argparse performs a complex formatting of the usage for argument grouping
and for line wrapping to fit the terminal width. This formatting has been
a constant source of bugs for at least 10 years (see linked issues below)
where defensive assertion errors are triggered or brackets and paranthesis
are not properly handeled.

Problem
=======

The current implementation of argparse usage formatting relies on regular
expressions to group arguments usage only to separate them again later
with another set of regular expressions. This is a complex and error prone
approach that caused all the issues linked below. Special casing certain
argument formats has not solved the problem. The following are some of
the most common issues:
- empty `metavar`
- mutually exclusive groups with `SUPPRESS`ed arguments
- metavars with whitespace
- metavars with brackets or paranthesis

Solution
========

The following two comments summarize the solution:
- python#82091 (comment)
- python#77048 (comment)

Mainly, the solution is to rewrite the usage formatting to avoid the
group-then-separate approach. Instead, the usage parts are kept separate
and only joined together at the end. This allows for a much simpler
implementation that is easier to understand and maintain. It avoids the
regular expressions approach and fixes the corresponding issues.

This closes the following issues:
- Closes python#62090
- Closes python#62549
- Closes python#77048
- Closes python#82091
- Closes python#89743
- Closes python#96310
- Closes python#98666

These PRs become obsolete:
- Closes python#15372
- Closes python#96311
@terryjreedy terryjreedy moved this to Bugs in Argparse issues Aug 16, 2023
encukou pushed a commit that referenced this issue May 7, 2024
Rationale
=========

argparse performs a complex formatting of the usage for argument grouping
and for line wrapping to fit the terminal width. This formatting has been
a constant source of bugs for at least 10 years (see linked issues below)
where defensive assertion errors are triggered or brackets and paranthesis
are not properly handeled.

Problem
=======

The current implementation of argparse usage formatting relies on regular
expressions to group arguments usage only to separate them again later
with another set of regular expressions. This is a complex and error prone
approach that caused all the issues linked below. Special casing certain
argument formats has not solved the problem. The following are some of
the most common issues:
- empty `metavar`
- mutually exclusive groups with `SUPPRESS`ed arguments
- metavars with whitespace
- metavars with brackets or paranthesis

Solution
========

The following two comments summarize the solution:
- #82091 (comment)
- #77048 (comment)

Mainly, the solution is to rewrite the usage formatting to avoid the
group-then-separate approach. Instead, the usage parts are kept separate
and only joined together at the end. This allows for a much simpler
implementation that is easier to understand and maintain. It avoids the
regular expressions approach and fixes the corresponding issues.

This closes the following GitHub issues:
-  #62090
-  #62549
-  #77048
-  #82091
-  #89743
-  #96310
-  #98666

These PRs become obsolete:
-  #15372
-  #96311
@encukou
Copy link
Member

encukou commented May 7, 2024

I cannot reproduce with the above reproducer, which I expanded to:

import argparse
from argparse import SUPPRESS

parser = argparse.ArgumentParser(prog='PROG')
state_group = parser.add_mutually_exclusive_group()
state_group.add_argument('--param1', nargs='?', const='default', metavar='NAME',
                         help=SUPPRESS)
state_group.add_argument('--param2', nargs='?', const='default', metavar='NAME',
                         help=SUPPRESS)
parser.parse_args(['--help'])

This is giving me the output:

usage: PROG [-h]

optional arguments:
  -h, --help  show this help message and exit

on Python versions 3.6 to 3.13.

If you have a complete example of failing behavior (even on an old version of Python), please share it here.

I think this problem was fixed in #105039; I'll close the issue.

@encukou encukou closed this as completed May 7, 2024
@github-project-automation github-project-automation bot moved this from Bugs to Doc issues in Argparse issues May 7, 2024
SonicField pushed a commit to SonicField/cpython that referenced this issue May 8, 2024
Rationale
=========

argparse performs a complex formatting of the usage for argument grouping
and for line wrapping to fit the terminal width. This formatting has been
a constant source of bugs for at least 10 years (see linked issues below)
where defensive assertion errors are triggered or brackets and paranthesis
are not properly handeled.

Problem
=======

The current implementation of argparse usage formatting relies on regular
expressions to group arguments usage only to separate them again later
with another set of regular expressions. This is a complex and error prone
approach that caused all the issues linked below. Special casing certain
argument formats has not solved the problem. The following are some of
the most common issues:
- empty `metavar`
- mutually exclusive groups with `SUPPRESS`ed arguments
- metavars with whitespace
- metavars with brackets or paranthesis

Solution
========

The following two comments summarize the solution:
- python#82091 (comment)
- python#77048 (comment)

Mainly, the solution is to rewrite the usage formatting to avoid the
group-then-separate approach. Instead, the usage parts are kept separate
and only joined together at the end. This allows for a much simpler
implementation that is easier to understand and maintain. It avoids the
regular expressions approach and fixes the corresponding issues.

This closes the following GitHub issues:
-  python#62090
-  python#62549
-  python#77048
-  python#82091
-  python#89743
-  python#96310
-  python#98666

These PRs become obsolete:
-  python#15372
-  python#96311
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
Projects
Status: Doc issues
Development

No branches or pull requests

4 participants