-
Notifications
You must be signed in to change notification settings - Fork 18
Sucrose Testing Framework
Because the Sucrose Chart Library started off as a clone of NVD3, it uses the chart component pattern introduced by NVD3 as a response to Mike Bostok's blog post on reusable chart components. As such, it is built from individual common components (e.g., axis, menu, header) and rendering models (e.g., line, multibar, pie) that are composed into richer chart objects (e.g., paretoChart, lineChart). There are also a few tooling components like utility, tooltip and scroller. Protected methods defined inside the closure have access to public and private properties and are responsible for rendering the SVG. For details see the Reusable Chart Component Pattern page. I will be referring to the code example on that page in the documentation below.
Unit tests are written to cover all public accessor methods, but since the Sucrose chart objects also have many private rendering methods, the entire component can't be tested with just unit tests. Take for instance the example code in the Reusable Chart Component Pattern page. The public .showTitle() method can be unit tested. Called without parameters it will return the default showTitle value “false”. If called with a parameter option, that value will be persisted in the chart instance (and the chart object is returned for for method chaining). However, the private .formatTitle() rendering function cannot be unit tested.
There are two test types that can be used to provide coverage for these private rendering functions. The easiest is to use JSDOM to create an HTML context then attach the chart object to a new DOM element in that context. When a property in the chart object is set (.showTitle(true)) and its rendering script is executed, the DOM can be inspected to verify that the proper manipulation has taken place. This test by proxy method is almost as fast as a unit test but there are some integration or functional tests that are not possible with this solution.
In order to support various test scenarios that require more complex rendering functionality, it is necessary to set up a virtual browser client and web server within the testing framework:
- testing methods that respond to user interaction like click or hover events,
- how the chart interacts within a browser context (like resizing as a window resizes), or
- if the chart is formatted using CSS.
NPM scripts: /sucrose/package.json
Test folder location: /sucrose/test/
- Data files:
./test/files/ - Text target fixtures:
./test/fixtures/- Instrumented Sucrose source:
./fixtures/build/
- Instrumented Sucrose source:
- Harnesses libraries:
./test/lib/ - Test specifications:
./test/specs/- Data transform tests:
./specs/data/ - DOM tests:
./specs/dom/ - Integration tests:
./specs/int/ - Unit tests:
./specs/unit/
- Data transform tests:
The package.json file found in the root of the Github repo contains a number of code blocks that are related to running tests. The scripts block contains a list of NPM scripts that run the three types of tests (data, unit, dom, integration). Before running any automated tests, it is necessary to generate the instrumented version of the Sucrose library by executing npm run instrument. To run only the unit test suites execute npm run test-unit from the root of your local checkout of the Sucrose repository, while using npm test will run all tests found under the /test/specs/ folder. During development if you only wish to run one test suite, then run the command node ./node_modules/.bin/tape 'test/specs/[type]/[your-test].js'
All tests are developed using the Tape testing library and organized into test suites using Tapes under the ./test/specs/ folder. Common test harness scripts written specifically for Sucrose are under the ./test/lib/ folder. The twine.js script is included in all unit tests to provide custom Tape assertions that all public methods available in the tested chart components are covered. For integration tests, the script dreams.js is for configuring and extending the Nightmare browser automation library, server.js is for bootstrapping a simple node web server, and coverage.js is for setting up the Istanbul code coverage analysis and reporting tools.
Tracking which lines of library code are executed during a full run of testing is useful for making sure there is complete test coverage and also will identify heavily used sections of code appropriate for optimization efforts or sections of dead code not run at all. To run coverage analysis for just the unit tests execute the npm run cover-unit script to run the unit tests along with code coverage analysis and reporting provided by Istanbul’s nyc CLI. Configuration of nyc is defined in the package.json which defines which fixture files are to analyzed and that the output of the Istanbul tool is written to the ./.coverage/ temporary folder. After running code coverage scripts, then execute the npm run cover-rpt script to generate a human readable HTML report in the ./coverage/lcov-report/index.html file. To generate a full coverage report for all test types then execute npm run cover-all.
Because the unit and DOM tests are executing in a node environment, Istanbul is able to share the same temporary coverage data file under ./.coverage/. However, because the integration tests are executed in isolated Nightmare browser contexts, the coverage.js harness script included in all integration tests handles writing the coverage data to the temporary folder. The nyc reporting tool is then able to merge all the temporary files into one unified report because they all share the same instrumented sucrose.js file in the ./test/fixtures/build/ folder.
A continous integration service provides a few processes that all modern open source projects should support. First is a way for any pull requests against the Github repository to have a complete run of all tests pass before they are merged. Second it is helpful to show on the repository home page the status of all tests. The sucrose repository is now set up to use the Travis-ci service. When Travis-ci detects a new PR, it will create a new execution environment and run the npm install script. The .travis.yml file defines three npm scripts to run for each pull request: npm run instrument to run before the tests, npm run cover-all (instead of the default npm test), and then npm run report-coverage which uploads the raw Instanbul code coverage data to the code coverage reporting service codecov.io after all tests are complete. The test pass/fail badge is now displayed on the Sucrose Github repository.
As new code contributions are offered as pull requests against the Github repository, it is import that the overall code coverage does not decrease. Over time the percentage of code in the Sucrose library should increase to 100%. Sucrose is now set up to use the codecov.io coverage reporting service provides the ability to monitor the progress or regression towards complete code coverage by storing the codecov report data from each PR test run. A badge on the Sucrose Github repository shows the current percentage of code coverage.
Tape: test assertions, runner and TAP reporter
Extend-tape: custom test assertions
Tapes: test grouping with before/afterEach
Faucet: TAP summarizer and formatting
Jsdom: virtual HTML DOM
Nightmare: Electorn based browser automation library
Istanbul: code coverage
Nyc: coverage runner
Travis-ci: continuous integration hosting
Codecov.io: coverage reporting hosting