Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

argparse usage should preserve () in metavars such as range(20) #62549

Closed
paulj3 mannequin opened this issue Jul 3, 2013 · 6 comments
Closed

argparse usage should preserve () in metavars such as range(20) #62549

paulj3 mannequin opened this issue Jul 3, 2013 · 6 comments
Labels
3.9 only security fixes 3.10 only security fixes 3.11 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@paulj3
Copy link
Mannequin

paulj3 mannequin commented Jul 3, 2013

BPO 18349
Nosy @iritkatriel
Files
  • metaparen.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2013-07-03.06:16:51.100>
    labels = ['type-bug', 'library', '3.9', '3.10', '3.11']
    title = 'argparse usage should preserve () in metavars such as range(20)'
    updated_at = <Date 2021-12-10.15:48:45.716>
    user = 'https://bugs.python.org/paulj3'

    bugs.python.org fields:

    activity = <Date 2021-12-10.15:48:45.716>
    actor = 'iritkatriel'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = ['Library (Lib)']
    creation = <Date 2013-07-03.06:16:51.100>
    creator = 'paul.j3'
    dependencies = []
    files = ['30754']
    hgrepos = []
    issue_num = 18349
    keywords = ['patch']
    message_count = 5.0
    messages = ['192222', '192717', '193077', '193188', '408214']
    nosy_count = 2.0
    nosy_names = ['paul.j3', 'iritkatriel']
    pr_nums = []
    priority = 'normal'
    resolution = None
    stage = None
    status = 'open'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue18349'
    versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

    @paulj3
    Copy link
    Mannequin Author

    paulj3 mannequin commented Jul 3, 2013

    As discussed in bpo-16468, a metavar may be used to provide an alternative representation of a choices option. However if a metvar like 'range(20)' is used, usage formatter strips off the '()'.

        >>> parser.add_argument('foo', type=int, 
            choices=range(20), metavar='range(0,20)')
        >>> parser.format_usage()
        # expect: 'usage: PROG [-h] range(0,20)\n'
        # actual: 'usage: PROG [-h] range0,20\n'

    This is done by a line in the help formater that removes excess mutually exclusive group notation:

    HelpFormatter._format_actions_usage
       ...
       text = _re.sub(r'\(([^|]*)\)', r'\1', text)
    

    A solution is to change this line to distinguish between a case like ' (...)' and 'range(...)'

        text = _re.sub(r'( )\(([^|]*)\)', r'\1\2', text)

    @paulj3 paulj3 mannequin added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Jul 3, 2013
    @paulj3
    Copy link
    Mannequin Author

    paulj3 mannequin commented Jul 9, 2013

    I just posted a patch to http://bugs.python.org/issue16468 that uses (and tests) this fix.

    @paulj3
    Copy link
    Mannequin Author

    paulj3 mannequin commented Jul 15, 2013

    This issue should also preserve a metavar like: '(one)two', i.e. '(' at the start.

    In http://bugs.python.org/issue10984 these _re replacements are applied to individual action strings as well as the whole usage line. So if () are to be removed from '[-h] (-y)', they should also be removed from '(-y)'.

    @paulj3
    Copy link
    Mannequin Author

    paulj3 mannequin commented Jul 16, 2013

    I just submitted at patch to http://bugs.python.org/issue11874 that takes care of this issue as well.

    I rewrote _format_actions_usage() so it formats the parts directly, so there is no need cleanup or parse the full text string.

    @iritkatriel
    Copy link
    Member

    Reproduced on 3.11:

    >>> import argparse
    >>> parser = argparse.ArgumentParser()
    >>> parser.add_argument('foo', type=int, choices=range(20), metavar='range(0,20)')
    _StoreAction(option_strings=[], dest='foo', nargs=None, const=None, default=None, type=<class 'int'>, choices=range(0, 20), help=None, metavar='range(0,20)')
    >>> parser.format_usage()
    'usage: [-h] range0,20\n'

    @iritkatriel iritkatriel added 3.9 only security fixes 3.10 only security fixes 3.11 only security fixes labels Dec 10, 2021
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    hamdanal added a commit to hamdanal/cpython that referenced this issue May 28, 2023
    Rationale
    =========
    
    argparse performs a complex formatting of the usage for argument grouping
    and for line wrapping to fit the terminal width. This formatting has been
    a constant source of bugs for at least 10 years (see linked issues below)
    where defensive assertion errors are triggered or brackets and paranthesis
    are not properly handeled.
    
    Problem
    =======
    
    The current implementation of argparse usage formatting relies on regular
    expressions to group arguments usage only to separate them again later
    with another set of regular expressions. This is a complex and error prone
    approach that caused all the issues linked below. Special casing certain
    argument formats has not solved the problem. The following are some of
    the most common issues:
    - empty `metavar`
    - mutually exclusive groups with `SUPPRESS`ed arguments
    - metavars with whitespace
    - metavars with brackets or paranthesis
    
    Solution
    ========
    
    The following two comments summarize the solution:
    - python#82091 (comment)
    - python#77048 (comment)
    
    Mainly, the solution is to rewrite the usage formatting to avoid the
    group-then-separate approach. Instead, the usage parts are kept separate
    and only joined together at the end. This allows for a much simpler
    implementation that is easier to understand and maintain. It avoids the
    regular expressions approach and fixes the corresponding issues.
    
    This closes the following issues:
    - Closes python#62090
    - Closes python#62549
    - Closes python#77048
    - Closes python#82091
    - Closes python#89743
    - Closes python#96310
    - Closes python#98666
    
    These PRs become obsolete:
    - Closes python#15372
    - Closes python#96311
    encukou pushed a commit that referenced this issue May 7, 2024
    Rationale
    =========
    
    argparse performs a complex formatting of the usage for argument grouping
    and for line wrapping to fit the terminal width. This formatting has been
    a constant source of bugs for at least 10 years (see linked issues below)
    where defensive assertion errors are triggered or brackets and paranthesis
    are not properly handeled.
    
    Problem
    =======
    
    The current implementation of argparse usage formatting relies on regular
    expressions to group arguments usage only to separate them again later
    with another set of regular expressions. This is a complex and error prone
    approach that caused all the issues linked below. Special casing certain
    argument formats has not solved the problem. The following are some of
    the most common issues:
    - empty `metavar`
    - mutually exclusive groups with `SUPPRESS`ed arguments
    - metavars with whitespace
    - metavars with brackets or paranthesis
    
    Solution
    ========
    
    The following two comments summarize the solution:
    - #82091 (comment)
    - #77048 (comment)
    
    Mainly, the solution is to rewrite the usage formatting to avoid the
    group-then-separate approach. Instead, the usage parts are kept separate
    and only joined together at the end. This allows for a much simpler
    implementation that is easier to understand and maintain. It avoids the
    regular expressions approach and fixes the corresponding issues.
    
    This closes the following GitHub issues:
    -  #62090
    -  #62549
    -  #77048
    -  #82091
    -  #89743
    -  #96310
    -  #98666
    
    These PRs become obsolete:
    -  #15372
    -  #96311
    @encukou
    Copy link
    Member

    encukou commented May 7, 2024

    This was fixed in #102318.

    @encukou encukou closed this as completed May 7, 2024
    @github-project-automation github-project-automation bot moved this from Bugs to Doc issues in Argparse issues May 7, 2024
    SonicField pushed a commit to SonicField/cpython that referenced this issue May 8, 2024
    Rationale
    =========
    
    argparse performs a complex formatting of the usage for argument grouping
    and for line wrapping to fit the terminal width. This formatting has been
    a constant source of bugs for at least 10 years (see linked issues below)
    where defensive assertion errors are triggered or brackets and paranthesis
    are not properly handeled.
    
    Problem
    =======
    
    The current implementation of argparse usage formatting relies on regular
    expressions to group arguments usage only to separate them again later
    with another set of regular expressions. This is a complex and error prone
    approach that caused all the issues linked below. Special casing certain
    argument formats has not solved the problem. The following are some of
    the most common issues:
    - empty `metavar`
    - mutually exclusive groups with `SUPPRESS`ed arguments
    - metavars with whitespace
    - metavars with brackets or paranthesis
    
    Solution
    ========
    
    The following two comments summarize the solution:
    - python#82091 (comment)
    - python#77048 (comment)
    
    Mainly, the solution is to rewrite the usage formatting to avoid the
    group-then-separate approach. Instead, the usage parts are kept separate
    and only joined together at the end. This allows for a much simpler
    implementation that is easier to understand and maintain. It avoids the
    regular expressions approach and fixes the corresponding issues.
    
    This closes the following GitHub issues:
    -  python#62090
    -  python#62549
    -  python#77048
    -  python#82091
    -  python#89743
    -  python#96310
    -  python#98666
    
    These PRs become obsolete:
    -  python#15372
    -  python#96311
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.9 only security fixes 3.10 only security fixes 3.11 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    Status: Doc issues
    Development

    No branches or pull requests

    2 participants