Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix 3678 "problem parsing with statements" #4538

Merged

Conversation

MeGaGiGaGon
Copy link
Collaborator

Description

This fixes #3678. The fix is done by adding two new functions to node.py, is_tuple_containing_star and is_generator, which are then added to the checks in maybe_make_parens_invisible_in_atom inside linegen.py, which is used by the function remove_with_parens that Jelle suggested modifying in a comment on #3678

Both functions are based on the existing is_tuple_containing_walrus. That makes me fairly confident in is_tuple_containing_star, since it does the same thing as is_tuple_containing_walrus, just with syms.namedexpr_test swapped for syms.star_expr. I'm less confident in is_generator since for a in b being called syms.old_comp_for seems weird and I don't totally understand syms/_python_symbols, but it seems to work.

Side note, _python_symbols is really hard to work with since node.type prints as an integer and the _python_symbol variable <--> integer relationship is done via setattr, so I ended up having to do [print(x, eval(f"syms.{x}") for x in dir(syms) if x[0:2] != "__"] to figure out what type generators get.

As usual, both names subject to bikeshedding. I chose is_tuple_containing_star because it compares with syms.star_expr, but is_tuple_containing_unpacking could also work. I chose is_generator since the issue is with a generator, though that could be something else since it doesn't match the syms.old_comp_for comparison.

The tests contain both a top level case and one where the issue is nested, since the nested case caused issues for me while making the fix.

Checklist - did you ...

  • Add an entry in CHANGES.md if necessary?
  • Add / update tests if necessary?
  • Add new / update outdated documentation?

Copy link
Collaborator

@JelleZijlstra JelleZijlstra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@JelleZijlstra JelleZijlstra merged commit 3b00112 into psf:main Dec 24, 2024
46 checks passed
@MeGaGiGaGon MeGaGiGaGon deleted the fix-3678-with-statement-parsing-problems branch December 24, 2024 20:29
luketainton pushed a commit to luketainton/PwnedPW that referenced this pull request Feb 10, 2025
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [black](https://github.com/psf/black) ([changelog](https://github.com/psf/black/blob/main/CHANGES.md)) | dependency-groups | major | `<25.0.0,>=24.10.0` -> `<25.2.0,>=25.1.0` |

---

### Release Notes

<details>
<summary>psf/black (black)</summary>

### [`v25.1.0`](https://github.com/psf/black/blob/HEAD/CHANGES.md#2510)

[Compare Source](psf/black@24.10.0...25.1.0)

##### Highlights

This release introduces the new 2025 stable style ([#&#8203;4558](psf/black#4558)), stabilizing
the following changes:

-   Normalize casing of Unicode escape characters in strings to lowercase ([#&#8203;2916](psf/black#2916))
-   Fix inconsistencies in whether certain strings are detected as docstrings ([#&#8203;4095](psf/black#4095))
-   Consistently add trailing commas to typed function parameters ([#&#8203;4164](psf/black#4164))
-   Remove redundant parentheses in if guards for case blocks ([#&#8203;4214](psf/black#4214))
-   Add parentheses to if clauses in case blocks when the line is too long ([#&#8203;4269](psf/black#4269))
-   Whitespace before `# fmt: skip` comments is no longer normalized ([#&#8203;4146](psf/black#4146))
-   Fix line length computation for certain expressions that involve the power operator ([#&#8203;4154](psf/black#4154))
-   Check if there is a newline before the terminating quotes of a docstring ([#&#8203;4185](psf/black#4185))
-   Fix type annotation spacing between `*` and more complex type variable tuple ([#&#8203;4440](psf/black#4440))

The following changes were not in any previous release:

-   Remove parentheses around sole list items ([#&#8203;4312](psf/black#4312))
-   Generic function definitions are now formatted more elegantly: parameters are
    split over multiple lines first instead of type parameter definitions ([#&#8203;4553](psf/black#4553))

##### Stable style

-   Fix formatting cells in IPython notebooks with magic methods and starting or trailing
    empty lines ([#&#8203;4484](psf/black#4484))
-   Fix crash when formatting `with` statements containing tuple generators/unpacking
    ([#&#8203;4538](psf/black#4538))

##### Preview style

-   Fix/remove string merging changing f-string quotes on f-strings with internal quotes
    ([#&#8203;4498](psf/black#4498))
-   Collapse multiple empty lines after an import into one ([#&#8203;4489](psf/black#4489))
-   Prevent `string_processing` and `wrap_long_dict_values_in_parens` from removing
    parentheses around long dictionary values ([#&#8203;4377](psf/black#4377))
-   Move `wrap_long_dict_values_in_parens` from the unstable to preview style ([#&#8203;4561](psf/black#4561))

##### Packaging

-   Store license identifier inside the `License-Expression` metadata field, see
    [PEP 639](https://peps.python.org/pep-0639/). ([#&#8203;4479](psf/black#4479))

##### Performance

-   Speed up the `is_fstring_start` function in Black's tokenizer ([#&#8203;4541](psf/black#4541))

##### Integrations

-   If using stdin with `--stdin-filename` set to a force excluded path, stdin won't be
    formatted. ([#&#8203;4539](psf/black#4539))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xNjQuMSIsInVwZGF0ZWRJblZlciI6IjM5LjE2NC4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJsaW50aW5nIl19-->

Reviewed-on: https://git.tainton.uk/repos/PwnedPW/pulls/283
Reviewed-by: Luke Tainton <luke@tainton.uk>
Co-authored-by: Renovate [BOT] <renovate-bot@git.tainton.uk>
Co-committed-by: Renovate [BOT] <renovate-bot@git.tainton.uk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

problem parsing with statements
2 participants