Skip to content

Commit

Permalink
[Testing] Dockerized test suite (#2430)
Browse files Browse the repository at this point in the history
* Removing spaces from test suite names

Having spaces in test suite names makes it difficult to call phpunit
from wrapper scripts, which are needed for the docker-based test suite.

* Skipping broken tests

* Replacing .click with .submit

.click was not executing form submit and was causing the test to fail

* Add missing xsi:nil property to fixture

* Allow a selenium host/port to be specified via config file

* Add docker-based test suite and required scripts

* Add note about creating Docker group

* Rename Dockerfiles

* Use linting scripts in .travis.yml
jacobpenny authored and driusan committed Dec 2, 2016
1 parent c71ca5c commit 706268f
Showing 21 changed files with 556 additions and 68 deletions.
59 changes: 4 additions & 55 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -29,13 +29,8 @@ install:
- composer install
- phpenv rehash

# Install jslint
- npm install -g jslint

# Install ESLint and plugins
- npm -g install eslint@3.8.1
- npm -g install eslint-config-google@0.6.0
- npm -g install eslint-plugin-react
# Install node modules specified in package.json
- npm install

# Download a Selenium Web Driver release
- wget "http://selenium-release.storage.googleapis.com/2.52/selenium-server-standalone-2.52.0.jar"
@@ -79,54 +74,8 @@ before_script:
# - "LORIS_DB_CONFIG=test/config.xml"

script:
# Run PHP -l on everything to ensure there's no syntax
# errors.
- for i in `ls php/libraries/*.class.inc modules/*/php/* modules/*/ajax/* htdocs/*.php htdocs/*/*.php`;
do
php -l $i || exit $?;
done

# Run PHPCS on the entire libraries directory.
- vendor/bin/phpcs --standard=docs/LorisCS.xml php/libraries php/exceptions php/installer
- vendor/bin/phpcs --standard=docs/LorisCS.xml --extensions=php/php htdocs

# Run PHPCS on some scripts
- vendor/bin/phpcs --standard=docs/LorisCS.xml tools/CouchDB_Import_MRI.php
- vendor/bin/phpcs --standard=docs/LorisCS.xml tools/assign_missing_instruments.php
- vendor/bin/phpcs --standard=docs/LorisCS.xml tools/data_dictionary_builder.php
- vendor/bin/phpcs --standard=docs/LorisCS.xml tools/generic_includes.php

# Run PHPCS on specific modules
- vendor/bin/phpcs --standard=docs/LorisCS.xml modules/imaging_uploader/php/NDB_Menu_Filter_imaging_uploader.class.inc
- vendor/bin/phpcs --standard=docs/LorisCS.xml modules/imaging_uploader/php/File_Decompress.class.inc
- vendor/bin/phpcs --standard=docs/LorisCS.xml --extensions=php/php,inc/php modules/genomic_browser
- vendor/bin/phpcs --standard=docs/LorisCS.xml --extensions=php/php,inc/php modules/candidate_list
- vendor/bin/phpcs --standard=docs/LorisCS.xml --extensions=php/php,inc/php modules/conflict_resolver
- vendor/bin/phpcs --standard=docs/LorisCS.xml --extensions=php/php,inc/php modules/dashboard
- vendor/bin/phpcs --standard=docs/LorisCS.xml --extensions=php/php,inc/php modules/examiner
- vendor/bin/phpcs --standard=docs/LorisCS.xml --extensions=php/php,inc/php modules/training
- vendor/bin/phpcs --standard=docs/LorisCS.xml --extensions=php/php,inc/php modules/brainbrowser
- vendor/bin/phpcs --standard=docs/LorisCS.xml --extensions=php/php,inc/php modules/configuration
- vendor/bin/phpcs --standard=docs/LorisCS.xml --extensions=php/php,inc/php modules/acknowledgements
- vendor/bin/phpcs --standard=docs/LorisCS.xml --extensions=php/php,inc/php modules/data_release
- vendor/bin/phpcs --standard=docs/LorisCS.xml --extensions=php/php,inc/php modules/media
- vendor/bin/phpcs --standard=docs/LorisCS.xml --extensions=php/php,inc/php modules/candidate_parameters/ajax
- vendor/bin/phpcs --standard=docs/LorisCS.xml --extensions=php/php,inc/php modules/dicom_archive
- vendor/bin/phpcs --standard=docs/LorisCS.xml --extensions=php/php,inc/php modules/create_timepoint
- vendor/bin/phpcs --standard=docs/LorisCS.xml --extensions=php/php,inc/php modules/issue_tracker

# Run JSLINT on specific scripts
- jslint htdocs/js/jquery.dynamictable.js

# Run ESLint on Loris modules
- eslint modules/

# Run ESLint on generic React components
- eslint jsx/

# Run ESLint on specific scripts
- eslint htdocs/js/util/
- eslint Gruntfile.js
- npm run lint:php
- npm run lint:javascript

# Run unit tests to make sure functions still do what they should.
- vendor/bin/phpunit --configuration test/phpunit.xml --coverage-clover=coverage.xml
27 changes: 27 additions & 0 deletions Dockerfile.test.db
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
FROM mysql:5.7

ARG BASE_DIR

COPY SQL/0000-00-00-schema.sql /0000-00-00-schema.sql
COPY SQL/0000-00-01-Permission.sql /0000-00-01-Permission.sql
COPY SQL/0000-00-02-Menus.sql /0000-00-02-Menus.sql
COPY SQL/0000-00-03-ConfigTables.sql /0000-00-03-ConfigTables.sql
COPY SQL/0000-00-04-Help.sql /0000-00-04-Help.sql
COPY SQL/0000-00-99-indexes.sql /0000-00-99-indexes.sql

COPY test/sql/CreateTestUser.sql /0000-01-00-TestUser.sql
COPY docs/instruments/radiology_review.sql /radiology_review.sql

RUN echo "Use LorisTest;" | cat - \
0000-00-00-schema.sql \
0000-00-01-Permission.sql \
0000-00-02-Menus.sql \
0000-00-03-ConfigTables.sql \
0000-00-04-Help.sql \
0000-00-99-indexes.sql \
0000-01-00-TestUser.sql \
radiology_review.sql > /docker-entrypoint-initdb.d/0000-compiled.sql

RUN echo "Use LorisTest;" >> /docker-entrypoint-initdb.d/0001-paths.sql
RUN echo "UPDATE Config SET Value='${BASE_DIR}/' WHERE ConfigID=(SELECT ID FROM ConfigSettings WHERE Name='base');" >> /docker-entrypoint-initdb.d/0001-paths.sql
RUN echo "UPDATE Config SET Value='http://web:8000' WHERE ConfigID=(SELECT ID FROM ConfigSettings WHERE Name='url');" >> /docker-entrypoint-initdb.d/0001-paths.sql
7 changes: 7 additions & 0 deletions Dockerfile.test.php7
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
FROM php:7.0

RUN apt-get update && \
apt-get install -y mysql-client zlib1g-dev

# Install extensions through the scripts the container provides
RUN docker-php-ext-install pdo_mysql zip
12 changes: 12 additions & 0 deletions Dockerfile.test.php7.debug
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
FROM php:7.0

RUN apt-get update && \
apt-get install -y mysql-client zlib1g-dev

RUN yes | pecl install xdebug-2.4.1
RUN echo "zend_extension=$(find /usr/local/lib/php/extensions/ -name xdebug.so)" > /usr/local/etc/php/conf.d/xdebug.ini
RUN echo "xdebug.remote_enable=on" >> /usr/local/etc/php/conf.d/xdebug.ini
RUN echo "xdebug.remote_autostart=on" >> /usr/local/etc/php/conf.d/xdebug.ini

# Install extensions through the scripts the container provides
RUN docker-php-ext-install pdo_mysql zip
111 changes: 111 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
version: '2'
services:
db:
build:
context: .
dockerfile: Dockerfile.test.db
args:
BASE_DIR: /app/
environment:
- MYSQL_DATABASE=LorisTest
- MYSQL_RANDOM_ROOT_PASSWORD=yes

selenium:
image: selenium/standalone-firefox-debug:2.53.1
ports:
- "5900:5900"

web:
build:
context: .
dockerfile: Dockerfile.test.php7
volumes:
- ./:/app
environment:
- LORIS_DB_CONFIG=/app/test/config.xml
depends_on:
- db
command: php -S 0.0.0.0:8000 -t /app/htdocs /app/htdocs/router.php

unit-tests:
build:
context: .
dockerfile: Dockerfile.test.php7
volumes:
- ./:/app
working_dir: /app
environment:
- LORIS_DB_CONFIG=test/config.xml
depends_on:
- db
entrypoint: /app/test/wait-for-services.sh

integration-tests:
build:
context: .
dockerfile: Dockerfile.test.php7
volumes:
- ./:/app
working_dir: /app
environment:
- LORIS_DB_CONFIG=test/config.xml
- SELENIUM_REQUIRED=true
depends_on:
- db
- selenium
- web
entrypoint: /app/test/wait-for-services.sh

selenium-debug:
image: selenium/standalone-firefox-debug:2.53.1
links:
- web-debug:web
ports:
- "5901:5900"

web-debug:
build:
context: .
dockerfile: Dockerfile.test.php7.debug
volumes:
- ./:/app
environment:
- LORIS_DB_CONFIG=/app/test/config.xml
- XDEBUG_CONFIG=remote_host=${XDEBUG_REMOTE_HOST}
- PHP_IDE_CONFIG=serverName=LorisTests
depends_on:
- db
command: php -S 0.0.0.0:8000 -t /app/htdocs /app/htdocs/router.php

unit-tests-debug:
build:
context: .
dockerfile: Dockerfile.test.php7.debug
volumes:
- ./:/app
working_dir: /app
environment:
- LORIS_DB_CONFIG=test/config.xml
- XDEBUG_CONFIG=remote_host=${XDEBUG_REMOTE_HOST}
- PHP_IDE_CONFIG=serverName=LorisTests
depends_on:
- db
entrypoint: /app/test/wait-for-services.sh

integration-tests-debug:
build:
context: .
dockerfile: Dockerfile.test.php7.debug
volumes:
- ./:/app
working_dir: /app
environment:
- LORIS_DB_CONFIG=test/config.xml
- SELENIUM_REQUIRED=true
- XDEBUG_CONFIG=remote_host=${XDEBUG_REMOTE_HOST}
- PHP_IDE_CONFIG=serverName=LorisTests
links:
- db
- selenium-debug:selenium
- web-debug:web
entrypoint: /app/test/wait-for-services.sh
6 changes: 6 additions & 0 deletions modules/instrument_list/test/instrument_listTest.php
Original file line number Diff line number Diff line change
@@ -38,6 +38,9 @@ class InstrumentListTestIntegrationTest extends LorisIntegrationTest
*/
function testInstrumentListDoespageLoad()
{
$this->markTestSkipped(
'Test is outdated/broken. This module requires candID and sessionID query parameters'
);
$this->webDriver->get($this->url . "/instrument_list/");
$bodyText = $this->webDriver->findElement(
WebDriverBy::cssSelector("body")
@@ -52,6 +55,9 @@ function testInstrumentListDoespageLoad()
*/
function testInstrumentListDoespageLoadWithPermission()
{
$this->markTestSkipped(
'Test is outdated/broken. This module requires candID and sessionID query parameters'
);
$this->setupPermissions(array("access_all_profiles"));
$this->webDriver->get($this->url . "/instrument_list/");
$bodyText = $this->webDriver->findElement(
4 changes: 2 additions & 2 deletions modules/next_stage/test/next_stageTest.php
Original file line number Diff line number Diff line change
@@ -107,7 +107,7 @@ function testNextStageDateError()
$Subproject->sendKeys("Control");

$startVisit = $this->webDriver->findElement(WebDriverBy::Name("fire_away"));
$startVisit->click();
$startVisit->submit();

$bodyText = $this->webDriver->findElement(WebDriverBy::cssSelector("body"))->getText();
$this->assertContains("Both Date fields must match.", $bodyText);
@@ -133,7 +133,7 @@ function testNextStageSuccess()
$Subproject->sendKeys("Control");

$startVisit = $this->webDriver->findElement(WebDriverBy::Name("fire_away"));
$startVisit->click();
$startVisit->submit();

$bodyText = $this->webDriver->findElement(WebDriverBy::cssSelector("body"))->getText();
$this->assertContains("Next stage started.", $bodyText);
12 changes: 9 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
@@ -12,15 +12,21 @@
"babel-cli": "^6.8.0",
"babel-preset-es2015": "^6.6.0",
"babel-preset-react": "^6.5.0",
"eslint": "^3.1.1",
"eslint-config-google": "^0.6.0",
"eslint": "3.8.1",
"eslint-config-google": "0.6.0",
"eslint-plugin-react": "^5.2.2",
"grunt": "^1.0.1",
"grunt-babel": "^6.0.0",
"jslint": "^0.10.3",
"load-grunt-tasks": "^3.5.0"
},
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
"lint:javascript": "./test/run-js-linter.sh",
"lint:php": "./test/run-php-linter.sh",
"tests:unit": "./test/dockerized-unit-tests.sh",
"tests:unit:debug": "DEBUG=true ./test/dockerized-unit-tests.sh",
"tests:integration": "./test/dockerized-integration-tests.sh",
"tests:integration:debug": "DEBUG=true ./test/dockerized-integration-tests.sh"
},
"repository": {
"type": "git",
87 changes: 87 additions & 0 deletions test/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# Dockerized Test Suite

## Requirements
You will need Docker Engine, Docker Compose, and NodeJS.

Please follow the directions [here](https://docs.docker.com/engine/installation/) to install Docker Engine. Be sure to also [create a Docker group](https://docs.docker.com/engine/installation/linux/ubuntulinux/#/create-a-docker-group) so Docker can be run without using `sudo`.

Next, install Docker Compose:

```
curl -L https://github.com/docker/compose/releases/download/1.8.1/docker-compose-`uname -s`-`uname -m` > ~/docker-compose
chmod +x ~/docker-compose
sudo mv ~/docker-compose /usr/local/bin/docker-compose
```

Make sure you have NodeJS installed. If not, follow the instructions [here](https://nodejs.org/en/download/package-manager/).

Finally, run `npm install` in the root folder (this is only required for Javascript linting).

## Basic Workflow

**To run all the unit tests:**

```
npm run tests:unit
```

**To run all of the integration tests:**

```
npm run tests:integration
```

You can see the integration tests in action by connecting your VNC viewer to `<host>:5900` and supplying the password `secret`.


**To run PHP linting:**

```
npm run lint:php
```

**To run Javascript linting:**

```
npm run lint:javascript
```

## Advanced Workflow

#### Command-Line Options
You can pass any [PHPUnit command-line options](https://phpunit.de/manual/current/en/textui.html) by appending `--` followed by the options. For example, say you only wanted to run the unit tests contained in the `CandidateTest` class. To achieve this you could run the following command:

```
npm run tests:unit -- --filter CandidateTest
```

Or, to run a specific test within `CandidateTest`:

```
npm run tests:unit -- --filter CandidateTest::testValidatePSCID
```

#### Debugging

Both the unit and integration tests can be run with XDebug enabled.

```
npm run tests:unit:debug
```
Or
```
npm run tests:integration:debug
```

You must specify a remote host for XDebug to connect to via the `XDEBUG_REMOTE_HOST` environment variable when using either of these commands.


## Todo

- Run integration tests in parallel

## Issues

- By default npm will output some irrelevant info when a script returns a non-zero error code, as described [here](https://github.com/npm/npm/issues/8821). To prevent this pass `-s` or `--silent` to `npm run`, e.g. `npm run -s tests:unit`.

- Running the entire integration test suite with XDebug enabled sometimes results in a segmentation fault. This appears to be an issue with XDebug itself.
Loading

0 comments on commit 706268f

Please sign in to comment.