argparse.ArgumentParser.add_mutually_exclusive_group : metavar create parenthesis undefined behavior

[AbcSxyZ]

BPO	45580
Nosy	@lysnikolaou, @pablogsal, @sobolevn, @AbcSxyZ

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-10-22.19:31:30.976>
labels = ['type-bug', 'library', '3.9', '3.10', '3.11']
title = 'argparse.ArgumentParser.add_mutually_exclusive_group : metavar create parenthesis undefined behavior'
updated_at = <Date 2021-10-25.16:32:34.545>
user = 'https://github.com/AbcSxyZ'

bugs.python.org fields:

activity = <Date 2021-10-25.16:32:34.545>
actor = 'paul.j3'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2021-10-22.19:31:30.976>
creator = 'AbcSxyZ'
dependencies = []
files = []
hgrepos = []
issue_num = 45580
keywords = []
message_count = 4.0
messages = ['404815', '404891', '404893', '404981']
nosy_count = 5.0
nosy_names = ['paul.j3', 'lys.nikolaou', 'pablogsal', 'sobolevn', 'AbcSxyZ']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue45580'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

[AbcSxyZ]

Hi,

I'm getting a kind of undefined behavior where parenthesis seem handled in a strange way. On display, it has a conflict between parenthesis of the option, and nested parenthesis within a metavar.

## Reproduction script

import argparse

def main():
    parser = argparse.ArgumentParser()

    group = parser.add_mutually_exclusive_group(required=True)

    group.add_argument("-p", "--path", metavar="/var/www/html", 
            help="DocumentRoot path")
    group.add_argument("-r", "--reverse", metavar="http)s(://Host:Port",
            help="Reverse proxy address")

    parser.add_argument("--last-args")
    return parser.parse_args()

main()

## Output of help menu

usage: crash.py [-h] (-p /var/www/html | -r http)s://Host:Port [--last-args LAST_ARGS]

## Expected behavior

usage: crash.py [-h] (-p /var/www/html | -r http)s(://Host:Port) [--last-args LAST_ARGS]

[sobolevn]

I confirm this happens on all recent Python versions.

The source of this problem is that argparse uses regex module to replace some substrings. Direct link: https://github.com/python/cpython/blame/8ce20bbdd6d2b1277a5e74154fcdcef2cb0fee49/Lib/argparse.py#L487

Quick debug showed that without this line these tests fail:

======================================================================
FAIL: test_help_when_required (test.test_argparse.TestMutuallyExclusiveFirstSuppressed)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_argparse.py", line 2649, in test_help_when_required
    self.assertEqual(format_help(), textwrap.dedent(help))
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 'usage: PROG [-h] (-y)\n\noptions:\n  -h, --help  show this[42 chars]lp\n' != 'usage: PROG [-h] -y\n\noptions:\n  -h, --help  show this h[40 chars]lp\n'
- usage: PROG [-h] (-y)
?                  -  -
+ usage: PROG [-h] -y
  
  options:
    -h, --help  show this help message and exit
    -y          y help


======================================================================
FAIL: test_usage_when_required (test.test_argparse.TestMutuallyExclusiveFirstSuppressed)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_argparse.py", line 2639, in test_usage_when_required
    self.assertEqual(format_usage(), textwrap.dedent(expected_usage))
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 'usage: PROG [-h] (-y)\n' != 'usage: PROG [-h] -y\n'
- usage: PROG [-h] (-y)
?                  -  -
+ usage: PROG [-h] -y


======================================================================
FAIL: test_help_when_required (test.test_argparse.TestMutuallyExclusiveFirstSuppressedParent)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_argparse.py", line 2649, in test_help_when_required
    self.assertEqual(format_help(), textwrap.dedent(help))
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 'usage: PROG [-h] (-y)\n\noptions:\n  -h, --help  show this[42 chars]lp\n' != 'usage: PROG [-h] -y\n\noptions:\n  -h, --help  show this h[40 chars]lp\n'
- usage: PROG [-h] (-y)
?                  -  -
+ usage: PROG [-h] -y
  
  options:
    -h, --help  show this help message and exit
    -y          y help


======================================================================
FAIL: test_usage_when_required (test.test_argparse.TestMutuallyExclusiveFirstSuppressedParent)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_argparse.py", line 2639, in test_usage_when_required
    self.assertEqual(format_usage(), textwrap.dedent(expected_usage))
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 'usage: PROG [-h] (-y)\n' != 'usage: PROG [-h] -y\n'
- usage: PROG [-h] (-y)
?                  -  -
+ usage: PROG [-h] -y


----------------------------------------------------------------------
Ran 1672 tests in 23.258s

FAILED (failures=4)
test test_argparse failed
test_argparse failed (4 failures)

== Tests result: FAILURE ==

1 test failed:
    test_argparse

Total duration: 25.6 sec
Tests result: FAILURE

[sobolevn]

Maybe instead we can show users something like:

usage: ex.py [-h] (-p '/var/www/html' | -r 'http)s(://Host:Port') [--last-args LAST_ARGS]

?

[paulj3]

The usage formatting is fragile, with many associated bug reports. Until someone does a major rewrite, it is best to avoid special characters, especially () and [] in the dest or metavar.

Usage uses () to encolde mutually_exclusive_groups and [] to mark non-required arguments. Don't confuse your users (or argparse) with other uses of these characters.

[encukou]

This was fixed in #102318.