Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ParseError with Ruby 3.3.x #81

Closed
rdimartino opened this issue Apr 24, 2024 · 6 comments
Closed

ParseError with Ruby 3.3.x #81

rdimartino opened this issue Apr 24, 2024 · 6 comments

Comments

@rdimartino
Copy link

Noticed that tests are failing with Ruby v3.3.0 and v3.3.1

Here is there error I get

  1) Error:
SyntaxTree::FormattingTest#test_layout:
SyntaxTree::Parser::ParseError: Invalid yield
    /.rbenv/versions/3.3.1/lib/ruby/gems/3.3.0/gems/syntax_tree-6.1.1/lib/syntax_tree/parser.rb:2864:in `on_parse_error'
    /.rbenv/versions/3.3.1/lib/ruby/gems/3.3.0/gems/syntax_tree-6.1.1/lib/syntax_tree.rb:123:in `parse'
    /.rbenv/versions/3.3.1/lib/ruby/gems/3.3.0/gems/syntax_tree-6.1.1/lib/syntax_tree.rb:123:in `parse'
    /syntax_tree-erb/lib/syntax_tree/erb/nodes.rb:467:in `initialize'
    /syntax_tree-erb/lib/syntax_tree/erb/nodes.rb:346:in `new'
    /syntax_tree-erb/lib/syntax_tree/erb/nodes.rb:346:in `rescue in prepare_content'
    /syntax_tree-erb/lib/syntax_tree/erb/nodes.rb:335:in `prepare_content'
    /syntax_tree-erb/lib/syntax_tree/erb/nodes.rb:298:in `initialize'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:589:in `new'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:589:in `parse_erb_tag'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:48:in `block (3 levels) in parse_any_tag'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:327:in `maybe'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:48:in `block (2 levels) in parse_any_tag'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:335:in `atleast'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:46:in `block in parse_any_tag'
    <internal:kernel>:187:in `loop'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:44:in `parse_any_tag'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:423:in `block in parse_html_element'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:347:in `block in many'
    <internal:kernel>:187:in `loop'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:345:in `many'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:423:in `parse_html_element'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:49:in `block (3 levels) in parse_any_tag'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:327:in `maybe'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:49:in `block (2 levels) in parse_any_tag'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:335:in `atleast'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:46:in `block in parse_any_tag'
    <internal:kernel>:187:in `loop'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:44:in `parse_any_tag'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:423:in `block in parse_html_element'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:347:in `block in many'
    <internal:kernel>:187:in `loop'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:345:in `many'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:423:in `parse_html_element'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:49:in `block (3 levels) in parse_any_tag'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:327:in `maybe'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:49:in `block (2 levels) in parse_any_tag'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:335:in `atleast'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:46:in `block in parse_any_tag'
    <internal:kernel>:187:in `loop'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:44:in `parse_any_tag'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:423:in `block in parse_html_element'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:347:in `block in many'
    <internal:kernel>:187:in `loop'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:345:in `many'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:423:in `parse_html_element'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:49:in `block (3 levels) in parse_any_tag'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:327:in `maybe'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:49:in `block (2 levels) in parse_any_tag'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:335:in `atleast'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:46:in `block in parse_any_tag'
    <internal:kernel>:187:in `loop'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:44:in `parse_any_tag'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:27:in `block in parse'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:347:in `block in many'
    <internal:kernel>:187:in `loop'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:345:in `many'
    /syntax_tree-erb/lib/syntax_tree/erb/parser.rb:27:in `parse'
    /syntax_tree-erb/lib/syntax_tree/erb.rb:21:in `parse'
    /syntax_tree-erb/lib/syntax_tree/erb.rb:17:in `block in format'
    /.rbenv/versions/3.3.1/lib/ruby/gems/3.3.0/gems/prettier_print-1.2.1/lib/prettier_print.rb:385:in `format'
    /syntax_tree-erb/lib/syntax_tree/erb.rb:17:in `format'
    /syntax_tree-erb/test/formatting_test.rb:44:in `assert_formatting'
    /syntax_tree-erb/test/formatting_test.rb:32:in `test_layout'

I did a little digging and syntax_tree-erb v0.10.2 doesn't have the ParseError with Ruby 3.3.1 (but the layout test still fails for a difference in formatting). v0.10.3 through v0.11.0 all have the ParseError

@rdimartino
Copy link
Author

Actually seems to be an upstream issue with Ruby 3.3.x and syntax_tree.

❯ rbenv shell 3.2.4
❯ stree ast -e 'yield'
(program (statements ((yield nil))))
❯ rbenv shell 3.3.0
❯ stree ast -e 'yield'
Error: Invalid yield
> 1 | yield
    | ^

@rdimartino
Copy link
Author

Hmmm. Seems like it's actually upstream of syntax_tree too. Appears to be from Ripper, but now I'm out of my depth

❯ rbenv shell 3.2.4    
❯ ruby -e 'require "ripper"; puts Ripper.sexp("yield", raise_errors: true)'
program
yield0

❯ rbenv shell 3.3.0                                                        
❯ ruby -e 'require "ripper"; puts Ripper.sexp("yield", raise_errors: true)'
/Users/robertdimartino/.rbenv/versions/3.3.0/lib/ruby/3.3.0/ripper/sexp.rb:40:in `sexp': Invalid yield (SyntaxError)
	from -e:1:in `<main>'

@rdimartino
Copy link
Author

Aha! It's a known bug https://bugs.ruby-lang.org/issues/20186

See also ruby-formatter/rufo#319

@davidwessman
Copy link
Owner

Just realized I had notifications off, I just merge a version were I put a placeholder in place when parsing + formatting.

Also discussed it with Jean Boussier from Shopify and created an issue with syntax_tree:
ruby-syntax-tree/syntax_tree#457

Earlopain added a commit to Earlopain/ruby-prism that referenced this issue Apr 28, 2024
This is similar to davidwessman/syntax_tree-erb#81 but for RuboCop
The parser gem doesn't support these types of checks,
see https://github.com/whitequark/parser?tab=readme-ov-file#syntax-check-of-block-exits

While this is technically a bug in the parser gem, it does increase compatibility
and allows prism to be used when linting erb or haml with a RuboCop extension.
matzbot pushed a commit to ruby/ruby that referenced this issue May 3, 2024
This is similar to davidwessman/syntax_tree-erb#81 but for RuboCop
The parser gem doesn't support these types of checks,
see https://github.com/whitequark/parser?tab=readme-ov-file#syntax-check-of-block-exits

While this is technically a bug in the parser gem, it does increase compatibility
and allows prism to be used when linting erb or haml with a RuboCop extension.

ruby/prism@6c59ae6a00
@kzkn
Copy link

kzkn commented May 29, 2024

FYI
Rufo's workaround is here: ruby-formatter/rufo#323

@davidwessman
Copy link
Owner

FYI

Rufo's workaround is here: ruby-formatter/rufo#323

Nice 🙂
Will need to implement something similar!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants