Teaspoon

Teaspoon is a Javascript test runner built on top of Rails. It can run tests in the browser, or headlessly using PhantomJS or with Selenium WebDriver.

Its objective is to be the simplest, while also being the most complete Javascript testing solution for Rails. It takes full advantage of the asset pipeline and ships with support for Jasmine, Mocha, QUnit, and (experiementally) Angular.

Ok, another Javascript test runner, right? Really? Yeah, that's tough, but we're pretty confident Teaspoon is one of the nicest and most full featured you'll find at the moment. And if you disagree, let us know and we'll probably fix whatever it is that you didn't like.

Feedback, ideas and pull requests are always welcome, or you can hit us up on Twitter @modeset_.

If you'd like to use Teaspoon with Guard, check out the guard-teaspoon project.

Developer Notice:

Teabag has been renamed to Teaspoon, thanks to bigtunacan.

This means you should change your gemspec to point to "teaspoon". A deprecation notice was provided with the last release of the gem under the teabag name, and versions will continue to move forward from 0.7.3.

Updating requires that you rename your teabag.rb initializer to teaspoon.rb, and teabag_env.rb to teaspoon_env.rb. Replacing any references to teabag to teaspoon with them. Or if you've not made any changes just clean these files up and run the install generator again.

Screenshots

Running in the console

Running in the console using Tapout

Running in the browser

Table of Contents

1. Installation
2. Usage
3. Writing Specs
4. Fixtures
5. Coverage
6. Suites
7. Configuration
8. Test Frameworks
9. Support Libraries
10. CI Support

Installation

Add it to your Gemfile. In most cases you'll want to restrict it to the :development, :test or :asset groups.

group :development, :test do
  gem "teaspoon"
end

Optionally run the install generator to get the initializer and a basic spec helper.

rails generate teaspoon:install

You can tell the generator which framework you want to use, if you want coffeescript spec helper files, and if you want the env file created (used by the command line interface).

rails generate teaspoon:install --framework=mocha --coffee

You can optionally install phantomjs. If phantomjs is found it will be used, otherwise the gem will be used as a fallback -- which might not be optimal.

Usage

Teaspoon uses the Rails asset pipeline to serve files. This allows you to use = require in your test files, and allows you use things like HAML or RABL/JBuilder within your fixtures. You can run Teaspoon in three ways -- in the browser, via the rake task, and using the command line interface.

Here's a great Quick Start Walkthrough for writing and running your first tests.

Browser

http://localhost:3000/teaspoon

You can focus tests in various ways, and Teaspoon provides an interface to run focus tests by suite, file, and filter.

Rake

rake teaspoon suite=my_fantastic_suite

Command Line Interface

The command line interface requires a teaspoon_env.rb file that you can get by running the generator. This file is used to load the Rails environment so Teaspoon can run within the context of Rails. This file can be in the spec, test, or root path -- but can be specified using the --require option.

bundle exec teaspoon

The CLI provides several ways of focusing tests. You can specify the suite to run, the files to run, and a filter.

bundle exec teaspoon --suite=my_fantastic_suite
bundle exec teaspoon spec/javascripts/calculator_spec.js
bundle exec teaspoon --filter="Calculator should add two digits"

Teaspoon also has support for tapout. Use the tap_y formatter and pipe the results to tapout to use any of the reporters that tapout provides.

bundle exec teaspoon -q --format=tap_y | tapout progress

Get full command line help:

bundle exec teaspoon --help

Note: By default the rake task and command line interface run within the development environment, but you can specify the environment using RAILS_ENV=test rake teaspoon. This is to stay consistent with what you might see in the browser (since that's likely running in development).

Writing Specs

Depending on what framework you use this can be slightly different. There's an expectation that you have a certain level of familiarity with the test framework that you're using. Right now Teaspoon supports Jasmine, Mocha and QUnit.

Since we have the asset pipeline at our fingertips you can feel free to use the = require directive throughout your specs and spec helpers.

Here's a basic spec written in Javascript using Jasmine:

//= require jquery
describe("My great feature", function() {

  it("will change the world", function() {
    expect(true).toBe(true);
    expect(jQuery).toBeDefined();
  });

});

Check out examples of a Mocha Spec, a QUnit Test, and an Angular Test.

Pending Specs

We've normalized declaring that a spec is pending between Mocha and Jasmine. Since Jasmine lacks the concept we've added it in, and since Mocha has several ways to accomplish it we thought it would be worth mentioning what we consider the standard between the two to be. QUnit doesn't support specifying a test as pending.

To mark a spec as pending you can either not provide a function as the second argument to it, or you can use xit and xdescribe. Mocha provides some additional ways to accomplish this, but to keep it consistent we've normalized on what they both support.

describe "My great feature", ->

  it "hasn't been tested yet"

  xit "has a test I can't figure out" ->
    expect("complexity").to.be("easily testable")

  xdescribe "A whole section that I've not gotten to", ->

    it "hasn't been tested yet", ->
      expect(true).to.be(false)

If you're using a specific framework and you want to take advantage of the things that framework provides you're free to do so. This is provided as the standard as the Teaspoon reporters understand the techniques above and have specs for them.

Using Require.js

If you are using require.js to get your files you can set a configuration option for your suite of "use_require".

Teaspoon.setup do |config|
  config.suite do |suite|
    suite.use_require = true
  end
end

Then in your suite spec helper, add require.js to be included, if you have not already. (Note: Teaspoon doesn't include require.js with it, so you will need to provide your own require.js and require the correct path.)

//= require require

Once you've done that, when that suite is executed, Teaspoon will use require.js to get all the specs in the suite (or specific files). In your specs you will need to use require to pull down the dependencies as you would normally. Here is an example with mocha.

define(['Model'] , function (Model) {
  describe('Model' , function () {
    // put your tests here
  });
});

Deferring Execution

Teaspoon allows deferring execution in the cases when you're using AMD or other asynchronous libraries. This is especially useful if you're using CommonJS, etc. You can tell Teaspoon to defer and then execute the runner yourself later -- after loading asychronously. There's a wiki article about how you can setup your specs and spec helper when using RequireJS with Teaspoon.

Teaspoon.defer = true;
setTimeout(Teaspoon.execute, 1000); // defers execution for 1 second

Fixtures

You're free to use your own fixture library (like jasmine-jquery, which we've included as a support library), but Teaspoon ships with a fixture library that works with Jasmine, Mocha, and QUnit with a minimum of effort, has a nice consistent API, and isn't dependent on jQuery.

The fixture path is configurable within Teaspoon, and the views will be rendered by a standard controller. This allows you to use things like RABL/JBuilder if you're building JSON, or HAML if you're building markup. The element that Teaspoon creates is "#teaspoon-fixtures", in case you need to access it directly -- or you can access it via fixture.el after loading fixtures.

Loading Files

Loading fixtures allows you to specify any number of files to load, and if they should be appended to the fixture element, or replace what's currently there.

fixture.load(url[, url, ...], append = false) or fixture(url[, url, ...], append = false)

Setting Manually

If you don't want to load files directly from the server you can provide strings instead of files, otherwise behaves like load.

fixture.set(html[, html, ...], append = false)

Cleaning Up

You shouldn't have to cleanup (we do that for you based on your test framework), but if you need it.

fixture.cleanup()

Preloading Files

Some test cases require stubbing Ajax requests, and in those cases you may want to preload the fixture files -- which caches them for later. You can preload fixtures in your spec helper, or before you start mocking Ajax methods.

fixture.preload(url[, url, ...])

Example Usage

fixture.preload("fixture.html", "fixture.json") # make the actual requests for the files
describe "Using fixtures", ->

  fixture.set("<h2>Another Title</h2>") # create some markup manually (will be in a beforeEach)

  beforeEach ->
    @fixtures = fixture.load("fixture.html", "fixture.json", true) # append these fixtures which were already cached

  it "loads fixtures", ->
    expect($("h1", fixture.el).text()).toBe("Title") # using fixture.el as a jquery scope
    expect($("h2", fixture.el).text()).toBe("Another Title")
    expect(@fixtures[0]).toBe(fixture.el) # the element is available as a return value and through fixture.el
    expect(@fixtures[1]).toEqual(fixture.json[0]) # the json for json fixtures is returned, and available in fixture.json

Check out some example of using fixtures with Mocha, QUnit, and Angular.

Coverage

Teaspoon can use Istanbul to generate code coverage statistics and reports. Install Istanbul and adjust the configuration to always generate coverage reports, or specify by passing --coverage to the command line interface. Check the configuration for more information.

Each suite allows you to specify which files should be ignored when generating coverage reports, which allows you to ignore support libraries and dependencies that you're not testing.

The following example will generate a simple text report and an HTML report with annotated source that you can inspect further.

bundle exec teaspoon --coverage-reports=text,html

An example text report that's output to the console after the tests have completed.

--------------------+-----------+-----------+-----------+-----------+
File                |   % Stmts |% Branches |   % Funcs |   % Lines |
--------------------+-----------+-----------+-----------+-----------+
  phantomjs/        |     93.75 |        75 |     94.12 |     93.65 |
    runner.coffee   |     93.75 |        75 |     94.12 |     93.65 |
--------------------+-----------+-----------+-----------+-----------+
All files           |     93.75 |        75 |     94.12 |     93.65 |
--------------------+-----------+-----------+-----------+-----------+

Suites

Teaspoon uses the concept of suites to group your tests at a high level. These suites are run in isolation from one another, and can have different configurations. You can define suites in the configuration, and for brevity config is the argument passed to the Teaspoon.setup block.

When creating a suite, provide a name (optional) and a block. The following example defines a suite named "my_suite". You can focus run this suite by browsing to /teaspoon/my_suite or running the rake task with suite=my_suite.

config.suite :my_suite do |suite|
  suite.helper = "my_spec_helper.js"
end

There's always a "default" suite defined, and to modify this suite just don't specify a name, or use :default. In this example we're setting the default suite configuration.

config.suite do |suite|
  suite.helper = "other_spec_helper.js"
end

Note: Suites inherit from the default suite, so default configuration will propagate to all other suites.

Manifest Style

Teaspoon is happy to look for files for you, but you can disable this feature and maintain a manifest yourself. Each suite can utilize a different spec helper and you can use these to create your own manifest using the = require directive. This isn't recommended because it limits your abilities to run specific files from the command line interface, but it's available if you want to use it.

Tell the suite that you don't want it to match any files, and then require files in your spec helper.

config.suite do |suite|
  suite.matcher = nil
  suite.helper = "spec_manifest"
end

Suite Configuration Directives

matcher

You can specify a file matcher and all matching files will be loaded when the suite is run. It's important that these files can be served via sprockets / are within an asset path.

Note: Can also be set to nil.

default: "{spec/javascripts,app/assets}/**/*_spec.{js,js.coffee,coffee}"

helper

Each suite can load a different spec helper, which can in turn require additional files. This file is loaded before your specs are loaded, and can be used as a manifest.

default: "spec_helper"

javascripts

These are the core Teaspoon javascripts. It's strongly encouraged to include only the base files here. You can require other support libraries in your spec helper, which allows you to change them without having to restart the server.

Note: To use the CoffeeScript source files use "teaspoon/jasmine" etc.

available: teaspoon-jasmine, teaspoon-mocha, teaspoon-qunit
default: ["teaspoon-jasmine"]

stylesheets

If you want to change how Teaspoon looks, or include your own stylesheets you can do that here. The default is the stylesheet for the HTML reporter.

Note: Spec related CSS can and should be loaded using fixtures.

default: ["teaspoon"]

no_coverage

When running coverage reports, you probably want to exclude libraries that you're not testing. Accepts an array of filenames or regular expressions. The default is to exclude assets from vendors or gems.

default: [%r{/lib/ruby/gems/}, %r{/vendor/assets/}, %r{/support/}, %r{/(.+)_helper.}]

Configuration

The best way to read about the configuration options is to generate the initializer and env, but we've included the info here as well.

Note: Teaspoon.setup may not be available in all environments, so the generator wraps it within a check.

mount_at

This determines where the Teaspoon routes will be mounted. Changing this to "/jasmine" would allow you to browse to http://localhost:3000/jasmine to run your specs.

default: "/teaspoon"

root

This defaults to Rails.root if left nil. If you're testing an engine using a dummy application it can be useful to set this to your engines root.. E.g. Teaspoon::Engine.root

default: nil

asset_paths

These paths are appended to the Rails assets paths (relative to config.root), and by default is an array that you can replace or add to.

default: ["spec/javascripts", "spec/javascripts/stylesheets", "test/javascripts", "test/javascripts/stylesheets"]

fixture_path

Fixtures are rendered through a standard controller. This means you can use things like HAML or RABL/JBuilder, etc. to generate fixtures within this path.

default: "spec/javascripts/fixtures"

Console Runner Specific (Teaspoon Env)

These configuration directives are applicable only when running via the rake task or command line interface and should be set within the teaspoon_env.rb file. You can get this file by running the generator.

driver

Allows you to specify which driver to use when running headlessly. Supports PhantomJS and Selenium Webdriver.

Check this wiki article for information about Using Selenium WebDriver, and this one about Using PhantomJS.

available: phantomjs, selenium
default: "phantomjs"

• CLI: -d, --driver DRIVER
• ENV: DRIVER=selenium

driver_cli_options

An experimental feature to allow you to specify additional CLI options/switches. Currently this is only supported if using the 'phantomjs' driver.

Check this wiki article for information about PhantomJS Command-line Options. Some options may cause Teaspoon to fail to function as expected/may not produce the expected result or may conflict with other options.

default: nil

• CLI: -o, --driver-cli-options OPTIONS_STRING
• ENV: DRIVER_CLI_OPTIONS="--ssl-protocol=any --ssl-certificates-path=/path/to/certs"

server

Specify a server to use with Rack (eg. thin, mongrel). If nil is provided Rack::Server is used.

default: nil

• CLI: --server SERVER
• ENV: SERVER=thin

server_timeout

Timeout for starting the server in seconds. If your server is slow to start you may have to bump this, or you may want to lower this if you know it shouldn't take long to start.

default: 20

• CLI: --server-timeout SECONDS
• ENV: SERVER_TIMEOUT=10

server_port

By default Teaspoon will locate an open port when starting the server, but if you want to run on a specific port you can do so by providing one.

default: nil

• CLI: --server-port PORT
• ENV: SERVER_PORT=31337

fail_fast

If you have several suites it can be useful to make Teaspoon fail directly after any suite contains failures, but in environments like CI this may not be desirable.

default: true

• CLI: --[no-]fail-fast
• ENV: FAIL_FAST=false

formatters

You can specify the formatters that Teaspoon will use when outputting the results.

available: dot, tap, tap_y, swayze_or_oprah
default: "dot"

• CLI: -f, --format FORMATTERS
• ENV: FORMATTERS=dot,swayze_or_oprah

suppress_log

Teaspoon pipes all console[log/debug/error] calls to STDOUT. This is useful to catch places where you've forgotten to remove them, but in verbose applications this may not be desirable.

default: false

• CLI: -q, --[no-]suppress-log
• ENV: SUPPRESS_LOG=true

color

Specify if you want color output by default.

default: true

• CLI: -c, --[no-]color
• ENV: COLOR=false

coverage

Add instrumentation to your code and display coverage information. Requires istanbul.

default: false

• CLI: -C, --coverage
• ENV: COVERAGE=true

coverage_reports

Specify which code coverage reports instanbul should generate.

available: text-summary, text, html, lcov, lcovonly, cobertura
default: nil

• CLI: -R, --coverage-reports REPORTS
• ENV: COVERAGE_REPORTS=text,html

Test Frameworks

Jasmine is used by default unless you specify otherwise. We've been using Jasmine for a long time, and have been pretty happy with it. It lacks a few important things that could be in a test framework, so we've done a little bit of work to make that nicer. Like adding pending spec support.

Mocha came up while we were working on Teaspoon -- we read up about it and thought it was a pretty awesome library with some really great approaches to some of the things that some of us browser types should consider more often, so we included it and added support for it. We encourage you to give it a try. Read more about Using Mocha with Teaspoon.

QUnit We're not sure about how many people use QUnit, but we like jQuery, so we added it. Read more about Using QUnit with Teaspoon.

Angular This is an experimental addition, and feedback is needed. Read more about Using Angular with Teaspoon.

Support Libraries

We know that testing usually requires more than just the test framework, so we've included some of the libraries that we use on a regular basis.

• Sinon.JS Standalone test spies, stubs and mocks for JavaScript. No dependencies, works with any unit testing framework.
• ChaiJS BDD / TDD assertion library for node and the browser that can be delightfully paired with any javascript testing framework.
• expect.js Minimalistic BDD assertion toolkit based on should.js.
• jasmine-jquery.js A set of custom matchers for jQuery, and an API for handling HTML fixtures in your specs.
• angular-scenario.js Angular test setup.

You can require these files in your spec helper by using:

//=require support/sinon
//=require support/chai
//=require support/expect
//=require support/jasmine-jquery
//=require support/angular-scenario

CI Support

Teaspoon works great on CI setups. If you're using TravisCI it just works, but if you're using something else all you should need is to ensure phantomjs is installed.

If you want to generate reports that CI can use you can install istanbul for coverage reports -- and output the report using the cobertura format, which Hudson can read.

Again on hudson compatibile CI setups, you can track spec failure information/rate tracking by using the tap formatter, which can be parsed by hudson.

A good setup:

teaspoon -q --coverage-reports=cobertura --format=tap

Or using Rake/ENV:

SUPPRESS_LOG=true COVERAGE_REPORTS=cobertura FORMATTERS=tap rake

There is also a TeamCity formatter:

teaspoon -q --format=teamcity

Or using Rake/ENV:

SUPPRESS_LOG=true FORMATTERS=teamcity rake

License

Licensed under the MIT License

Copyright 2012 Mode Set

All licenses for the bundled Javascript libraries are included (MIT/BSD).

Make Code Not War