BoostCalcQt segfaults on travis

[konserw]

Summary

Travis fails for all (3 of them) PRs: https://travis-ci.org/cucumber/cucumber-cpp/pull_requests since 2 weeks ago
Reason is boost version of calcqt example, not sure why as it runs well on my linux machine.

Possible Solution

I'll check if it fails also when using unix sockets, but it is more like workaround, not the solution.

Exact error message:

+wait %

+[ -f build/examples/CalcQt/QtTestCalculatorQtSteps -a -n :99 ]

+[ -f build/examples/CalcQt/BoostCalculatorQtSteps -a -n :99 ]

+sleep 1

+build/examples/CalcQt/BoostCalculatorQtSteps

+cucumber examples/CalcQt

# language: en

Feature: Addition

  In order to avoid silly mistakes

  As a math idiot 

  I want to be told the sum of two numbers

  Scenario Outline: Add two numbers       # examples/CalcQt/features/addition.feature:7

    Given I just turned on the calculator # examples/CalcQt/features/addition.feature:8

    When I press <button1>                # examples/CalcQt/features/addition.feature:9

    And I press add                       # examples/CalcQt/features/addition.feature:10

    And I press <button2>                 # examples/CalcQt/features/addition.feature:11

    And I press calculate                 # examples/CalcQt/features/addition.feature:12

    Then the display should show <result> # examples/CalcQt/features/addition.feature:13

    Examples: 

      | button1 | button2 | result |

      | 2       | 3       | 5      |

      Timed out calling wire server with message 'invoke' (Timeout::Error)

      examples/CalcQt/features/addition.feature:17:in `Given I just turned on the calculator'

      examples/CalcQt/features/addition.feature:8:in `Given I just turned on the calculator'

Timed out calling wire server with message 'step_matches' (Timeout::Error)

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-2.0.0/lib/cucumber/wire_support/connection.rb:43:in `block in fetch_data_from_socket'

/home/travis/.rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/timeout.rb:108:in `timeout'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-2.0.0/lib/cucumber/wire_support/connection.rb:43:in `fetch_data_from_socket'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-2.0.0/lib/cucumber/wire_support/connection.rb:20:in `call_remote'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-2.0.0/lib/cucumber/wire_support/request_handler.rb:10:in `execute'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-2.0.0/lib/cucumber/wire_support/wire_protocol/requests.rb:14:in `execute'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-2.0.0/lib/cucumber/wire_support/wire_protocol.rb:8:in `step_matches'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-2.0.0/lib/cucumber/wire_support/wire_language.rb:34:in `block in step_matches'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-2.0.0/lib/cucumber/wire_support/wire_language.rb:34:in `map'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-2.0.0/lib/cucumber/wire_support/wire_language.rb:34:in `step_matches'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-2.0.0/lib/cucumber/runtime/support_code.rb:214:in `block in matches'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-2.0.0/lib/cucumber/runtime/support_code.rb:213:in `map'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-2.0.0/lib/cucumber/runtime/support_code.rb:213:in `matches'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-2.0.0/lib/cucumber/runtime/support_code.rb:201:in `step_match_without_cache'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-2.0.0/lib/cucumber/runtime/support_code.rb:195:in `step_match'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-2.0.0/lib/cucumber/runtime/support_code.rb:138:in `find_match'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-2.0.0/lib/cucumber/filters/activate_steps.rb:28:in `attempt_to_activate'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-2.0.0/lib/cucumber/filters/activate_steps.rb:24:in `map'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-2.0.0/lib/cucumber/filters/activate_steps.rb:24:in `new_test_steps'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-2.0.0/lib/cucumber/filters/activate_steps.rb:18:in `test_case'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-2.0.0/lib/cucumber/filters/activate_steps.rb:8:in `test_case'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-core-1.1.3/lib/cucumber/core/test/case.rb:21:in `describe_to'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-2.0.0/lib/cucumber/filters/quit.rb:11:in `test_case'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-core-1.1.3/lib/cucumber/core/test/case.rb:21:in `describe_to'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-core-1.1.3/lib/cucumber/core/test/filters/locations_filter.rb:17:in `block in done'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-core-1.1.3/lib/cucumber/core/test/filters/locations_filter.rb:16:in `each'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-core-1.1.3/lib/cucumber/core/test/filters/locations_filter.rb:16:in `done'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-core-1.1.3/lib/cucumber/core/filter.rb:61:in `done'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-core-1.1.3/lib/cucumber/core/test/filters/tag_filter.rb:18:in `done'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-core-1.1.3/lib/cucumber/core/compiler.rb:23:in `done'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-core-1.1.3/lib/cucumber/core/gherkin/parser.rb:31:in `done'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-core-1.1.3/lib/cucumber/core.rb:29:in `parse'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-core-1.1.3/lib/cucumber/core.rb:18:in `compile'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-2.0.0/lib/cucumber/runtime.rb:70:in `run!'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-2.0.0/lib/cucumber/cli/main.rb:38:in `execute!'

/home/travis/.rvm/gems/ruby-2.4.1/gems/cucumber-2.0.0/bin/cucumber:9:in `<top (required)>'

/home/travis/.rvm/gems/ruby-2.4.1/bin/cucumber:23:in `load'

/home/travis/.rvm/gems/ruby-2.4.1/bin/cucumber:23:in `<main>'

/home/travis/.rvm/gems/ruby-2.4.1/bin/ruby_executable_hooks:15:in `eval'

/home/travis/.rvm/gems/ruby-2.4.1/bin/ruby_executable_hooks:15:in `<main>'

The command "./travis.sh" exited with 1.

Done. Your build exited with 1.

[konserw]

I've triggered travis build for my new branch based on master without any changes and it fails as well:
https://travis-ci.org/konserw/cucumber-cpp/jobs/275224230
See here if it works with unix socks: https://travis-ci.org/konserw/cucumber-cpp/builds/275225967
EDIT:
Ok unix sockets didn't help. There is still timeout. I wonder what might have changed in travis during last 2 weeks.

[konserw]

I've created PR #170 - It is more like workaround - using old travis image fixes our false fails in CI.
I think we should merge it ASAP and then think about next steps. Maybe contact travis team? What else can we do apart of disabling xvfb - based tests?

[muggenhor]

Could you create a bug report at https://github.com/travis-ci/travis-ci for this?

[konserw]

done: travis-ci/travis-ci#8481 Care to add anything ?

[muggenhor]

I've added some extra code to get some extra debugging from the travis script. That produces this: https://travis-ci.org/muggenhor/cucumber-cpp/jobs/279609591

It seems that we're getting an exit code 139 from the step definition executable, which is a bashism for signal 139-128=11 aka SIGSEGV. I.e. we're getting a memory access violation, so probably have a bug somewhere that's being triggered. So maybe this is not a buggy Travis but instead something flaky with our code that we're lucky enough to have Travis trigger for us.

[konserw]

I wonder how is this possible as new (failing) and old (OK) travis images have the exact same versions of gcc, gmock, boost, qt, bash, docker, cmake, ruby, xvfb (although on the new image it is preinstlled, not from apt)
There is small difference in versions of git, kernel, clang
And the code is the same. And it's fine on OSX (on travis), Windows (appveyor), Arch (my PC)

[muggenhor]

I'm wondering the same, which is why I tried to make it produce a coredump and have gdb analyze it: for some reason the problem turns into an apparent deadlock then (10 minutes no terminal output timeout causes Travis to kill it then).