NullPointerException in FrameUtil.getObjectSafe escaping from Interpreter

[petenorth]

I've started seeing the following exceptions when running Cucumber tests

https://github.com/petenorth/schools-experience/runs/1399517944?check_suite_focus=true#step:8:90

  Background:              # features/candidates/bookings/cancelling_a_booking.feature:6
    Given I have a booking # features/step_definitions/candidates/bookings/cancelling_a_booking_steps.rb:1
truffleruby: an exception escaped out of the interpreter - this is an implementation bug
org.graalvm.polyglot.PolyglotException: java.lang.NullPointerException
	at com.oracle.truffle.api.frame.FrameUtil.getObjectSafe(FrameUtil.java:59)
	at org.truffleruby.core.kernel.TruffleKernelNodes$GetSpecialVariableStorage.getFromKnownFrameDescriptor(TruffleKernelNodes.java:248)
	at org.truffleruby.core.kernel.TruffleKernelNodesFactory$GetSpecialVariableStorageNodeGen.execute(TruffleKernelNodesFactory.java:615)
	at org.truffleruby.language.methods.BlockDefinitionNode.execute(BlockDefinitionNode.java:97)
	at org.truffleruby.language.methods.BlockDefinitionNode.execute(BlockDefinitionNode.java:30)
	at org.truffleruby.core.cast.ProcOrNullNodeGen.executeProcOrNull(ProcOrNullNodeGen.java:51)
	at org.truffleruby.language.dispatch.RubyCallNode.executeBlock(RubyCallNode.java:131)
	at org.truffleruby.language.dispatch.RubyCallNode.execute(RubyCallNode.java:100)
	at org.truffleruby.language.control.FrameOnStackNode.execute(FrameOnStackNode.java:37)
	at org.truffleruby.language.methods.CatchBreakNode.execute(CatchBreakNode.java:40)
	at org.truffleruby.language.control.SequenceNode.execute(SequenceNode.java:36)
	at org.truffleruby.language.arguments.CheckArityNode.execute(CheckArityNode.java:41)
	at org.truffleruby.language.control.SequenceNode.execute(SequenceNode.java:36)
	at org.truffleruby.language.methods.CatchForMethodNode.execute(CatchForMethodNode.java:42)
	at org.truffleruby.language.methods.ExceptionTranslatingNode.execute(ExceptionTranslatingNode.java:33)
	at org.truffleruby.language.RubyRootNode.execute(RubyRootNode.java:63)
	at <ruby> Selenium::WebDriver::Remote::W3C::Capabilities#camel_case(Unknown)
	at <ruby> block in Selenium::WebDriver::Remote::W3C::Capabilities#as_json(../truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/w3c/capabilities.rb:275:10127-10176)
	at <ruby> Selenium::WebDriver::Remote::W3C::Capabilities#as_json(../truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/w3c/capabilities.rb:262:9539-9584)
	at <ruby> Selenium::WebDriver::Remote::W3C::Capabilities#to_json(../truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/w3c/capabilities.rb:285:10402-10434)
	at <ruby> block in JSON::Pure::Generator::GeneratorMethods::Array#json_transform(../truffleruby-head/lib/json/lib/json/pure/generator.rb:349:11732-11777)
	at <ruby> JSON::Pure::Generator::GeneratorMethods::Array#json_transform(../truffleruby-head/lib/json/lib/json/pure/generator.rb:345:11562-11587)
	at <ruby> JSON::Pure::Generator::GeneratorMethods::Array#to_json(../truffleruby-head/lib/json/lib/json/pure/generator.rb:332:11230-11262)
	at <ruby> ActiveSupport::ToJsonWithActiveSupportEncoder#to_json(../truffleruby-head/lib/gems/gems/activesupport-5.2.4.1/lib/active_support/core_ext/object/json.rb:38:1957-1978)
	at <ruby> block in JSON::Pure::Generator::GeneratorMethods::Hash#json_transform(../truffleruby-head/lib/json/lib/json/pure/generator.rb:310:10470-10515)
	at <ruby> JSON::Pure::Generator::GeneratorMethods::Hash#json_transform(../truffleruby-head/lib/json/lib/json/pure/generator.rb:302:10141-10170)
	at <ruby> JSON::Pure::Generator::GeneratorMethods::Hash#to_json(../truffleruby-head/lib/json/lib/json/pure/generator.rb:284:9672-9704)
	at <ruby> ActiveSupport::ToJsonWithActiveSupportEncoder#to_json(../truffleruby-head/lib/gems/gems/activesupport-5.2.4.1/lib/active_support/core_ext/object/json.rb:38:1957-1978)
	at <ruby> block in JSON::Pure::Generator::GeneratorMethods::Hash#json_transform(../truffleruby-head/lib/json/lib/json/pure/generator.rb:310:10470-10515)
	at <ruby> JSON::Pure::Generator::GeneratorMethods::Hash#json_transform(../truffleruby-head/lib/json/lib/json/pure/generator.rb:302:10141-10170)
	at <ruby> JSON::Pure::Generator::GeneratorMethods::Hash#to_json(../truffleruby-head/lib/json/lib/json/pure/generator.rb:284:9672-9704)
	at <ruby> ActiveSupport::ToJsonWithActiveSupportEncoder#to_json(../truffleruby-head/lib/gems/gems/activesupport-5.2.4.1/lib/active_support/core_ext/object/json.rb:38:1957-1978)
	at <ruby> JSON::Pure::Generator::State#generate(../truffleruby-head/lib/json/lib/json/pure/generator.rb:242:8156-8191)
	at <ruby> JSON#generate(../truffleruby-head/lib/json/lib/json/common.rb:224:8264-8286)
	at <ruby> Selenium::WebDriver::Remote::Http::Common#call(../truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/http/common.rb:54:1797-1865)
	at <ruby> Selenium::WebDriver::Remote::Bridge#execute(../truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/bridge.rb:167:5572-5616)
	at <ruby> Selenium::WebDriver::Remote::Bridge#create_session(../truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/bridge.rb:102:3559-3656)
	at <ruby> Selenium::WebDriver::Remote::Bridge.handshake(../truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/bridge.rb:56:1913-2003)
	at <ruby> Selenium::WebDriver::Remote::Driver#initialize(../truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/driver.rb:39:1364-1405)
	at <ruby> Selenium::WebDriver::Driver.for(../truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/driver.rb:58:1683-1718)
	at <ruby> Selenium::WebDriver.for(../truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver.rb:88:3260-3293)
	at <ruby> Capybara::Selenium::Driver#browser(../truffleruby-head/lib/gems/gems/capybara-3.31.0/lib/capybara/selenium/driver.rb:51:1739-1816)
	at <ruby> Capybara::Selenium::Driver#visit(../truffleruby-head/lib/gems/gems/capybara-3.31.0/lib/capybara/selenium/driver.rb:70:2198-2226)
	at <ruby> Capybara::Session#visit(../truffleruby-head/lib/gems/gems/capybara-3.31.0/lib/capybara/session.rb:278:9966-9999)
	at <ruby> Capybara::DSL#visit(/truffleruby-head/lib/gems/gems/capybara-3.31.0/lib/capybara/dsl.rb:2:15-46)
	at <ruby> block in Object#<top (required)>(features/step_definitions/candidates/bookings/cancelling_a_booking_steps.rb:7:239-284)
	at <ruby> block in Cucumber::Glue::InvokeInWorld.cucumber_instance_exec_in(../truffleruby-head/lib/gems/gems/cucumber-3.1.2/lib/cucumber/glue/invoke_in_world.rb:39:1567-1612)
	at <ruby> Cucumber::Glue::InvokeInWorld.cucumber_run_with_backtrace_filtering(../truffleruby-head/lib/gems/gems/cucumber-3.1.2/lib/cucumber/glue/invoke_in_world.rb:54:1964-1978)
	at <ruby> Cucumber::Glue::InvokeInWorld.cucumber_instance_exec_in(../truffleruby-head/lib/gems/gems/cucumber-3.1.2/lib/cucumber/glue/invoke_in_world.rb:27:1019-1081)
	at <ruby> Cucumber::Glue::StepDefinition#invoke(../truffleruby-head/lib/gems/gems/cucumber-3.1.2/lib/cucumber/glue/step_definition.rb:110:3363-3475)
	at <ruby> Cucumber::StepMatch#invoke(../truffleruby-head/lib/gems/gems/cucumber-3.1.2/lib/cucumber/step_match.rb:31:954-992)
	at <ruby> block in Cucumber::StepMatch#activate(../truffleruby-head/lib/gems/gems/cucumber-3.1.2/lib/cucumber/step_match.rb:24:751-830)
	at <ruby> Cucumber::Core::Test::Action#execute(../truffleruby-head/lib/gems/gems/cucumber-core-3.2.1/lib/cucumber/core/test/action.rb:24:639-666)
	at <ruby> Cucumber::Core::Test::Step#execute(../truffleruby-head/lib/gems/gems/cucumber-core-3.2.1/lib/cucumber/core/test/step.rb:32:756-787)
	at <ruby> Cucumber::Core::Test::Runner::RunningTestCase::Status::Base#execute(../truffleruby-head/lib/gems/gems/cucumber-core-3.2.1/lib/cucumber/core/test/runner.rb:104:2734-2802)
	at <ruby> Cucumber::Core::Test::Runner::RunningTestCase#execute(../truffleruby-head/lib/gems/gems/cucumber-core-3.2.1/lib/cucumber/core/test/runner.rb:51:1480-1533)
	at <ruby> Cucumber::Core::Test::Runner#test_step(../truffleruby-head/lib/gems/gems/cucumber-core-3.2.1/lib/cucumber/core/test/runner.rb:27:783-842)
	at <ruby> Cucumber::Core::Test::Step#describe_to(../truffleruby-head/lib/gems/gems/cucumber-core-3.2.1/lib/cucumber/core/test/step.rb:17:434-473)
	at <ruby> block (3 levels) in Cucumber::Core::Test::Case#describe_to(../truffleruby-head/lib/gems/gems/cucumber-core-3.2.1/lib/cucumber/core/test/case.rb:28:878-936)
	at <ruby> block (2 levels) in Cucumber::Core::Test::Case#describe_to(../truffleruby-head/lib/gems/gems/cucumber-core-3.2.1/lib/cucumber/core/test/case.rb:27:833-876)

[eregon]

@aardvark179 Could you take a look at this one?

[petenorth]

Hopefully the following is a simpler project with a GitHub workflow which shows the difference in behaviour between HEAD, 20.2 and MRI (the latter two behave in the same way).

https://github.com/petenorth/gsub-tests/runs/1409321836?check_suite_focus=true

The matrix provided to the workflow is made up of [2.6, truffleruby, truffleruby-head] these correspond to MRI, 20.2 and HEAD respectively.

MRI and 20.2 fail but in an expected way, HEAD fails but due to a null pointer escaping from the interpreter.

I'd like to create a simpler reproducer but this is the best i can do at the moment. It seems to be due to the passing of some ruby symbol to camel case in Selenium::WebDriver::Remote::W3C::Capabilities#camel_case

Whose method body is str.gsub(/_([a-z])/) { Regexp.last_match(1).upcase }

[chrisseaton]

I've seen these as well. Can't give you a reproducer right now I'm afraid.

[eregon]

@petenorth @chrisseaton Do you see this NullPointerException consistently or is it transient?

[aardvark179]

I've not been able to reproduce this locally, so I'm not sure if it's a race or compilation bug. What I can do for now is improve the error reporting when this problem occurs which should help use narrow down where the problem is occurring in Ruby code, and whether we've made some assumption that is not actually valid.

[petenorth]

I experience it consistently on every run.

[chrisseaton]

Sorry I can't remember where I've seen it, but I definitely recognise seeing it. Sorry that's not very helpful.

[eregon]

The -head builds are a bit old (5 days) due to some macOS/jt issue, will fix and then let's try again to repro with the newer builds.

[petenorth]

git clone https://github.com/petenorth/gsub-tests.git
cd gsub-tests.git
gem install rails
rails new cucumber_bdd_how_to
mv Gemfile.example cucumber_bdd_how_to/Gemfile
cd cucumber_bdd_how_to
bundler install
rails generate cucumber:install
mv ../hello_world.feature features
mv ../hello_world_steps.rb features/step_definitions
mv ../env.rb features/support
rm .ruby-version 
RUBYOPT=--experimental-options\ --engine.Mode=latency
CUC_DRIVER=chrome APP_URL='http://en.wikipedia.org' SELENIUM_HUB_HOSTNAME=selenium-chrome bundle exec cucumber features/hello_world.feature

[eregon]

@petenorth There are new truffleruby-head builds now, could you trigger the gsub-tests workflow?
Most likely it still happens, but the backtrace should be a bit more understandable at least.

[petenorth]

Here are the results of the latest run

https://github.com/petenorth/gsub-tests/runs/1418047935