PHPUnit tests do not accept testsuite as argument

[nbennett25]

When using the bin/magento command line tool to run Integration testing, setting a testsuite argument does not work, per the instructions at the bottom of:

http://devdocs.magento.com/guides/v2.0/test/integration/integration_test_setup.html

Preconditions

1. Magento version 2.1.5
2. PHP 7.0

Steps to reproduce

1. Run bin/magento dev:test:run integration --testsuite "Memory Usage Tests" per the documentation

Expected result

1. "Memory Usage Tests" integration testsuite tests should be run

Actual result

1. Error: [RuntimeException] The "--testsuite" option does not exist.

[TomashKhamlai]

Hello! Thank you for reporting this issue! At first glance, it doesn't look like a bug.
Let assign a number for the ways of executing integration tests to simplify communication.

1. PHPUnit ⚓

$ cd dev/tests/integration
$ ../../../vendor/bin/phpunit --testsuite "Memory Usage Tests"

2. Magento CLI ⚓.

$ php bin/magento dev:tests:run integration

Probably, you think that keys and arguments that are available in a 1-st way of running integration testing suites will do the same job in a 2-nd way. Didn't find any reason for that to be true. Magento CLI module is not an alias for the PHPUnit and it only provides the simple tool to check the integrity. Please provide a link for article or code sample that states the opposite.

There is the way to show the usage of the magento dev:tests:run command:

php bin/magento dev:tests:run --help
Cannot load Xdebug - it was already loaded
Usage:
  dev:tests:run [<type>]

Arguments:
  type                  Type of test to run. Available types: all, unit, integration, integration-all, static, static-all, integrity, legacy, default [default: "default"]

Options:
  -h, --help            Display this help message
  -q, --quiet           Do not output any message
  -V, --version         Display this application version
      --ansi            Force ANSI output
      --no-ansi         Disable ANSI output
  -n, --no-interaction  Do not ask any interactive question
  -v|vv|vvv, --verbose  Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug

Help:
  Runs tests

[orlangur]

Documentation states
$ php ../../../vendor/bin/phpunit --testsuite "Third Party Integration Tests"

It has nothing to do with bin/magento dev:test:run integration --testsuite "Memory Usage Tests"

So, obviously, this issue is not a bug but a feature request ("pass all arguments to PHPUnit").

As to me, I never used bin/magento dev:test:run and have no idea what's the reasoning behind its introduction. Thus it is quite doubtful to me that this functionality should be extended even more.

[TomashKhamlai]

@nbennett25 as @orlangur mentioned this issue is a feature improvement and not a bug. GitHub is not intended to be the place for feature improvements, you can read about it here.

Please spend 2 minutes to copy/paste your request to the Magento 2 Feature Requests and Improvements forum, also copy the link of this issue #9477 on Github and add to the message. Let me know when you're done and I will follow it and correct it if needed. Thank you in advance.