Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use RubyLex for yamatanooroti test #169

Merged
merged 2 commits into from
Sep 10, 2020
Merged

Use RubyLex for yamatanooroti test #169

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7746c55
Use RubyLex for yamatanooroti test
aycabta Sep 10, 2020
c01dda4
Drop 2.5 for yamatanooroti test
aycabta Sep 10, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/workflows/reline.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ jobs:
strategy:
matrix:
os: [ ubuntu-latest ]
ruby: [ 2.7, 2.6, 2.5 ]
ruby: [ 2.7, 2.6 ]
steps:
- uses: actions/checkout@v2
- name: Set up Ruby
Expand All @@ -107,5 +107,7 @@ jobs:
sudo make install
- name: Install yamatanooroti
run: gem install yamatanooroti vterm
- name: Install latest irb
run: gem install irb -f
- name: rake test_yamatanooroti
run: rake test_yamatanooroti
180 changes: 2 additions & 178 deletions bin/termination_checker.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
require 'ripper'
require 'irb/ruby-lex'

class TerminationChecker
class TerminationChecker < RubyLex
def terminated?(code)
code.gsub!(/\n*$/, '').concat("\n")
@tokens = Ripper.lex(code)
Expand All @@ -14,181 +15,4 @@ def terminated?(code)
true
end
end

def process_continue
continued_bits = Ripper::EXPR_BEG | Ripper::EXPR_FNAME | Ripper::EXPR_DOT
# last token is always newline
if @tokens.size >= 2 and @tokens[-2][1] == :on_regexp_end
# end of regexp literal
return false
elsif @tokens.size >= 2 and @tokens[-2][1] == :on_semicolon
return false
elsif @tokens.size >= 2 and @tokens[-2][1] == :on_kw and (@tokens[-2][2] == 'begin' or @tokens[-2][2] == 'else')
return false
elsif !@tokens.empty? and @tokens.last[2] == "\\\n"
return true
elsif @tokens.size >= 2 and @tokens[-2][3].anybits?(continued_bits)
# end of literal except for regexp
return true
end
false
end

def check_code_block(code)
return true if @tokens.empty?
if @tokens.last[1] == :on_heredoc_beg
return true
end

begin # check if parser error are available
RubyVM::InstructionSequence.compile(code)
rescue SyntaxError => e
case e.message
when /unterminated (?:string|regexp) meets end of file/
# "unterminated regexp meets end of file"
#
# example:
# /
#
# "unterminated string meets end of file"
#
# example:
# '
return true
when /syntax error, unexpected end-of-input/
# "syntax error, unexpected end-of-input, expecting keyword_end"
#
# example:
# if ture
# hoge
# if false
# fuga
# end
return true
when /syntax error, unexpected keyword_end/
# "syntax error, unexpected keyword_end"
#
# example:
# if (
# end
#
# example:
# end
return false
when /unexpected tREGEXP_BEG/
# "syntax error, unexpected tREGEXP_BEG, expecting keyword_do or '{' or '('"
#
# example:
# method / f /
return false
end
end

last_lex_state = @tokens.last[3]
if last_lex_state.allbits?(Ripper::EXPR_BEG)
return false
elsif last_lex_state.allbits?(Ripper::EXPR_DOT)
return true
elsif last_lex_state.allbits?(Ripper::EXPR_CLASS)
return true
elsif last_lex_state.allbits?(Ripper::EXPR_FNAME)
return true
elsif last_lex_state.allbits?(Ripper::EXPR_VALUE)
return true
elsif last_lex_state.allbits?(Ripper::EXPR_ARG)
return false
end

false
end

def process_nesting_level
@tokens.inject(0) { |indent, t|
case t[1]
when :on_lbracket, :on_lbrace, :on_lparen
indent += 1
when :on_rbracket, :on_rbrace, :on_rparen
indent -= 1
when :on_kw
case t[2]
when 'def', 'do', 'case', 'for', 'begin', 'class', 'module'
indent += 1
when 'if', 'unless', 'while', 'until', 'rescue'
# postfix if/unless/while/until/rescue must be Ripper::EXPR_LABEL
indent += 1 unless t[3].allbits?(Ripper::EXPR_LABEL)
when 'end'
indent -= 1
end
end
# percent literals are not indented
indent
}
end

def check_string_literal
i = 0
start_token = []
end_type = []
while i < @tokens.size
t = @tokens[i]
case t[1]
when :on_tstring_beg
start_token << t
end_type << :on_tstring_end
when :on_regexp_beg
start_token << t
end_type << :on_regexp_end
when :on_symbeg
if (i + 1) < @tokens.size and @tokens[i + 1][1] != :on_ident
start_token << t
end_type << :on_tstring_end
end
when :on_backtick
start_token << t
end_type << :on_tstring_end
when :on_qwords_beg, :on_words_beg, :on_qsymbols_beg, :on_symbols_beg
start_token << t
end_type << :on_tstring_end
when :on_heredoc_beg
start_token << t
end_type << :on_heredoc_end
when end_type.last
start_token.pop
end_type.pop
end
i += 1
end
start_token.last.nil? ? '' : start_token.last
end

def process_literal_type
start_token = check_string_literal
case start_token[1]
when :on_tstring_beg
case start_token[2]
when ?" then ?"
when /^%.$/ then ?"
when /^%Q.$/ then ?"
when ?' then ?'
when /^%q.$/ then ?'
end
when :on_regexp_beg then ?/
when :on_symbeg then ?:
when :on_backtick then ?`
when :on_qwords_beg then ?]
when :on_words_beg then ?]
when :on_qsymbols_beg then ?]
when :on_symbols_beg then ?]
when :on_heredoc_beg
start_token[2] =~ /<<[-~]?(['"`])[_a-zA-Z0-9]+\1/
case $1
when ?" then ?"
when ?' then ?'
when ?` then ?`
else ?"
end
else
nil
end
end
end