Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Call chain formatting in fluent style #6151

Merged
merged 15 commits into from
Aug 4, 2023
Merged

Call chain formatting in fluent style #6151

merged 15 commits into from
Aug 4, 2023

Conversation

konstin
Copy link
Member

@konstin konstin commented Jul 28, 2023

Implement fluent style/call chains. See the call_chains.py formatting for examples.

This isn't fully like black because in raise A from B they allow A breaking can influence the formatting of B even if it is already multiline.

Similarity index:

project main PR
build ??? 0.753
django 0.991 0.998
transformers 0.993 0.994
typeshed 0.723 0.723
warehouse 0.978 0.994
zulip 0.992 0.994

Call chain formatting is affected by #627, but i'm cutting scope here.

Closes #5343

Test Plan:

  • Added a dedicated call chains test file
  • The ecosystem checks found some bugs
  • I manually check django and zulip formatting

@konstin
Copy link
Member Author

konstin commented Jul 28, 2023

Current dependencies on/for this PR:

This comment was auto-generated by Graphite.

@github-actions
Copy link
Contributor

github-actions bot commented Jul 28, 2023

PR Check Results

Benchmark

Linux

group                                      main                                   pr
-----                                      ----                                   --
formatter/large/dataset.py                 1.00      9.3±0.02ms     4.4 MB/sec    1.11     10.3±0.09ms     3.9 MB/sec
formatter/numpy/ctypeslib.py               1.00   1885.3±8.59µs     8.8 MB/sec    1.06      2.0±0.01ms     8.3 MB/sec
formatter/numpy/globals.py                 1.02   231.6±14.37µs    12.7 MB/sec    1.00    226.2±7.00µs    13.0 MB/sec
formatter/pydantic/types.py                1.00      4.0±0.03ms     6.3 MB/sec    1.06      4.3±0.04ms     6.0 MB/sec
linter/all-rules/large/dataset.py          1.00     13.1±0.10ms     3.1 MB/sec    1.02     13.3±0.14ms     3.1 MB/sec
linter/all-rules/numpy/ctypeslib.py        1.00      3.3±0.02ms     5.0 MB/sec    1.01      3.4±0.01ms     5.0 MB/sec
linter/all-rules/numpy/globals.py          1.00    458.6±0.76µs     6.4 MB/sec    1.00    457.8±1.70µs     6.4 MB/sec
linter/all-rules/pydantic/types.py         1.00      6.0±0.05ms     4.3 MB/sec    1.01      6.0±0.02ms     4.2 MB/sec
linter/default-rules/large/dataset.py      1.00      6.5±0.24ms     6.3 MB/sec    1.05      6.8±0.04ms     6.0 MB/sec
linter/default-rules/numpy/ctypeslib.py    1.00   1356.7±7.79µs    12.3 MB/sec    1.05   1428.3±6.01µs    11.7 MB/sec
linter/default-rules/numpy/globals.py      1.01    159.5±7.98µs    18.5 MB/sec    1.00    157.3±0.61µs    18.8 MB/sec
linter/default-rules/pydantic/types.py     1.00      2.8±0.02ms     9.0 MB/sec    1.05      3.0±0.03ms     8.5 MB/sec

Windows

group                                      main                                   pr
-----                                      ----                                   --
formatter/large/dataset.py                 1.00     10.4±0.13ms     3.9 MB/sec    1.06     11.1±0.11ms     3.7 MB/sec
formatter/numpy/ctypeslib.py               1.00  1974.8±38.30µs     8.4 MB/sec    1.04      2.0±0.04ms     8.1 MB/sec
formatter/numpy/globals.py                 1.00   222.4±10.07µs    13.3 MB/sec    1.02    225.9±8.00µs    13.1 MB/sec
formatter/pydantic/types.py                1.00      4.3±0.07ms     5.9 MB/sec    1.03      4.5±0.06ms     5.7 MB/sec
linter/all-rules/large/dataset.py          1.00     14.7±0.16ms     2.8 MB/sec    1.01     14.8±0.16ms     2.8 MB/sec
linter/all-rules/numpy/ctypeslib.py        1.00      3.8±0.06ms     4.4 MB/sec    1.03      3.9±0.06ms     4.3 MB/sec
linter/all-rules/numpy/globals.py          1.00    452.9±8.58µs     6.5 MB/sec    1.02    462.0±6.85µs     6.4 MB/sec
linter/all-rules/pydantic/types.py         1.00      6.7±0.09ms     3.8 MB/sec    1.01      6.8±0.10ms     3.8 MB/sec
linter/default-rules/large/dataset.py      1.00      7.3±0.11ms     5.6 MB/sec    1.08      7.9±0.15ms     5.2 MB/sec
linter/default-rules/numpy/ctypeslib.py    1.00  1470.6±26.20µs    11.3 MB/sec    1.07  1568.9±21.62µs    10.6 MB/sec
linter/default-rules/numpy/globals.py      1.00    165.7±3.23µs    17.8 MB/sec    1.08    178.2±5.09µs    16.6 MB/sec
linter/default-rules/pydantic/types.py     1.00      3.2±0.08ms     7.9 MB/sec    1.07      3.4±0.08ms     7.4 MB/sec

@konstin konstin changed the title Fluent style Call chain formatting in fluent style Jul 30, 2023
@konstin konstin force-pushed the fluent_style branch 3 times, most recently from 9bcc65a to 202c7f0 Compare July 31, 2023 07:33
@konstin konstin marked this pull request as ready for review July 31, 2023 07:33
@konstin konstin marked this pull request as draft July 31, 2023 07:53
@konstin
Copy link
Member Author

konstin commented Jul 31, 2023

Blocked on unstable formatting of

y = (
    x.a()  #
    .b()
)

y = x.a().b()  #

y = (
    x.aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa()  #
    .b()
)

y = x.aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa().b()  #

@konstin konstin force-pushed the fluent_style branch 2 times, most recently from ee69abe to ca3ca2a Compare August 2, 2023 09:51
@konstin konstin marked this pull request as ready for review August 2, 2023 12:43
Copy link
Member

@MichaReiser MichaReiser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Woah, nice improvement on the compatibility!

The fluent style formatting now requires to route through the fluent style in many positions (which I like more than setting in on context).

Have you considered to, instead "unroll" the call chain in the CallExpression formatting? Meaning, we would have a single formatting that owns the whole call chain formatting without calling into format attribute and format subscript (maybe parts of it). I'm asking because I find it difficult to "unroll" the recursion in my head and wonder if it would be easier if the whole call chain formatting would be in its own file.

Comment on lines 74 to 77

/// Switch call chain and attribute formatting to fluent style. This is otherwise identical to
/// `Never`, fluent style implies a set of outer parentheses
FluentStyle { outermost: bool },
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FluentStyle doesn't fit well into the Parentheses concept, which is intended to be generally applicable to all expressions.

@konstin konstin force-pushed the fluent_style branch 3 times, most recently from c3801b2 to 58fc27c Compare August 3, 2023 12:12
@konstin
Copy link
Member Author

konstin commented Aug 3, 2023

Good point, i made fluent style a bool that gets passed through the call chain formatting. It's still recursive but i think that's better than unrolling

@konstin
Copy link
Member Author

konstin commented Aug 3, 2023

This took some rotations but now it's just a is_fluent_style_call_chain function we call when formatting an expression. There is still a case we miss (not a().b().c()), but i'm happy how it looks now.

@MichaReiser
Copy link
Member

Good point, i made fluent style a bool that gets passed through the call chain formatting. It's still recursive but i think that's better than unrolling

Could you explain your reasoning of why?

@konstin
Copy link
Member Author

konstin commented Aug 3, 2023

Good point, i made fluent style a bool that gets passed through the call chain formatting. It's still recursive but i think that's better than unrolling

Could you explain your reasoning of why?

I think consistency, mainly: Every other expression is formatted from outermost to innermost, so this is no difference. The main thing we do differently is to put a newline between the closing parentheses and the dot.

crates/ruff_python_formatter/src/expression/mod.rs Outdated Show resolved Hide resolved
Comment on lines 68 to 71
// Nested expressions that are in some outer expression's parentheses may be formatted in
// fluent style
let fluent_style = f.context().node_level() == NodeLevel::ParenthesizedExpression
&& is_fluent_style_call_chain(expression, f.context().source());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this still necessary here? It seems that Call, Attribute and Subscript now by-pass the Expr formatting.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I spent way to much time trying to come up with something better, but i'm afraid the answer is yes. I think the node level is the biggest problem here because e.g. just inlining helpers doesn't seem to work.

Comment on lines 177 to 188
Expr::Attribute(expr) => {
return parenthesize_if_expands(&group(&expr.format().with_options(true)))
.fmt(f)
}
Expr::Call(expr) => {
return parenthesize_if_expands(&group(&expr.format().with_options(true)))
.fmt(f)
}
Expr::Subscript(expr) => {
return parenthesize_if_expands(&group(&expr.format().with_options(true)))
.fmt(f)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to remove the code from here if:

  • Replace the boolean option with an enum that has three variants:
    • Fluent
    • NonFluent: Left side of a call chain, but we don't use the fluent style
    • Root: The root of a fluent chain (default)
  • CallExpression tests if it is inside of a fluent chain if the variant is Root. (or any node that can be the end of a call chain). If so, use Fluent, otherwise NonFluent
  • Subscript, Call, or Attribute wrap their content in a group if the layout is Fluent.
  • Change can_omit_parentheses to return false for call chains.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think i really follow; Call chains can appear about everywhere except on statement level. For the MaybeParenthesize it's special because we need to switch parentheses mode, call chain formatting always has parentheses when breaking.

Change can_omit_parentheses to return false for call chains.

This is already the case.

Unfortunately I don't understand why exactly the current solution works but the alternatives i tried don't, the interaction of node level, context, parentheses, MaybeParenthesize and FormatExpr or not is still opaque to me.

}
_ => {
// We to format the following in fluent style: `f2 = (a).w().t(1,)`
if is_expression_parenthesized(AnyNodeRef::from(expr), source) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is_expression_parenthesized returns false positives if it is the first argument in a call expression. I guess this is safe because the expression never has attributes_after_parentheses that is larger than 1.

crates/ruff_python_formatter/src/expression/mod.rs Outdated Show resolved Hide resolved
@konstin konstin enabled auto-merge (squash) August 4, 2023 11:34
@konstin konstin merged commit 99baad1 into main Aug 4, 2023
16 checks passed
@konstin konstin deleted the fluent_style branch August 4, 2023 13:58
@konstin
Copy link
Member Author

konstin commented Aug 4, 2023

oh i didn't turn of automerge; it should be fine anyway, i'll do a follow-up PR if anything comes up

@MichaReiser
Copy link
Member

oh i didn't turn of automerge; it should be fine anyway, i'll do a follow-up PR if anything comes up

This looks good to me.

Does this implementation handle the case you shared with me earlier correctly where it must preserve parentheses around members of the call chain?

a = (
    b().c(
        "asdfasfaefinitive Guidddddde to Django: Web Developfdddddddment Done Rdddight"
    )
).d()
a = (
    b()
    .c("asdfasfaefinitive Guidddddde to Django: Web Developfdddddddment Done Rdddight")
    .d()
)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Formatter: Attribute and call chains "fluent interface"
2 participants