Calabash testing

Calabash-Android lets you run Cucumber features on your Android device or emulator. This page describes the way the Calabash testing framework is used and set up on a Mac for FLOW.

Relevant links

• [Writing feature files in Gherkin syntax] (https://github.com/cucumber/cucumber/wiki/Gherkin)
• [Calabash Github page] (https://github.com/calabash/calabash-android)
• [Cucumber framework] (http://cukes.info/)
• [Cucumber Given-When-Then constructs] (https://github.com/cucumber/cucumber/wiki/Given-When-Then)
• Packing multiple cucumber steps in one
• [Video on how to do behaviour driven testing] (http://skillsmatter.com/podcast/agile-scrum/bdd-as-its-meant-to-be-done)

Setup notes

These setup notes are based on those on the Calabash Github page

Installation

Prerequisites

You need to have Ruby installed. Verify your installation by running ruby -v in a terminal - it should print "ruby 1.8.7" (or higher)

You should have the Android SDK installed and the environment variable ANDROID_HOME should be pointing to it. In addition, the environment variable PATH needs to contain the path to platform-tools directory

The easiest way of doing this is to add:

export ANDROID_HOME=/path/to/android-sdk-macosx
export PATH=$PATH:/path/to/android-sdk-macosx/platform-tools  

to your .bash_profile, which lives in your home directory. If it not there, create it. For example, on my computer, which has the Android SDK in the applications folder, I have:

export ANDROID_HOME=/Users/markwestra/Applications/android-sdk-macosx 
export PATH=$PATH:/Users/markwestra/Applications/android-sdk-macosx/platform-tools  

Installation

Install calabash-android by running

• gem install calabash-android in the terminal
• You might have to run sudo gem install calabash-android if you do not have the right permissions.

Configuration

• Create a folder which you want to use for your Calabash project.

• In the terminal, navigate to the folder, and run calabash-android setup

You will be asked a series of questions about your app and environment. Here is an example (with my answers between double pointy brackets):

When you are through this setup your settings will be saved to .calabash_settings. You can edit this file if you have the need.
What is the package name of the app? You can find the package name in AndroidManifest.xml
<<com.gallatinsystems.survey.device>>
What is the fully qualified name of the main activity?
<<com.gallatinsystems.survey.device.activity.SurveyHomeActivity>>
What is the path to the app?
<</Users/markwestra/1_Akvo/Software/FLOW/FLOW_test/FieldSurvey/bin/FieldSurvey.apk>>
Which api level do you want to use?
It looks like you have the following versions installed:
4, 7, 8, 10, 15
<<10>>
Do you want to specify a keystore for signing the test app?
If now we will be using /Users/jml/.android/debug.keystore
Please answer yes (y) or no (n)
y
Please enter keystore location
<</Users/markwestra/1_Akvo/Software/FLOW/keystore/fieldsurvey-release.keystore>> 
Please enter the password for the keystore
<<PasswordOfKeystore>>
Please enter the alias
<<AliasOfKeystore>>
Please enter the password for the alias
<<PasswordOfAlias>>
Saved your settings to .calabash_settings. You can edit the settings manually or run this setup script again

You can always run calabash-android setup again or change the file manually.

Notice: Make sure that the app you are trying to test is signed with the key store you just selected in the setup. This can be done by selecting "Export Signed Application Package" from the android tools menu in eclipse, which will ask you for the keystore details.

Generate a Cucumber skeleton

To get started with calabash in an empty folder, it might be a good idea to run calabash-android gen. It will create a Cucumber skeleton in the current folder like this:

features
|_support
| |_app_installation_hooks.rb
| |_app_life_cycle_hooks.rb
| |_env.rb
| |_hooks.rb
|_step_definitions
| |_calabash_steps.rb
|_my_first.feature

In this skeleton you find all the predefined steps that comes with calabash. Try to take a look my_first.feature and change it to fit your app.

For FLOW, the directory 'Calabash' is already filled with tests.

Writing a test

The Cucumber features goes in the features library and should have the ".feature" extension.

You can start out by looking at features/test_dummy.feature. You can extend this feature or make your own using some of the predefined steps that comes with Calabash.

This is an example of a test for FLOW, which tests a scenario in the 'adding users' feature:

Feature: User management
Scenario: As a user I can create a user
When I press "Manage Users"
Then I see "Select the current user by clicking. To create a new user, press the Menu button and select Add User. Long-click to edit a user."
Then I select "Add User" from the menu
And I enter "Test User 1" into input field number 1
And I enter "m.t.westra@gmail.com" into input field number 2    
And I press "Save"
Then I see "Test User 1"

Running test

Before running the test, first you need to attach a phone to your computer, or fire up an Android emulator. The emulator can be started by opening the 'tools' folder of 'android-sdk-macosx' in the terminal, and typing ./android. This will open up the Android SDK manager. Click 'Tools' in the menu, and select 'Manage AVDs'. There you can define and start new emulators.

A useful thing to have running is the Dalvik Debug Monitor, which allows you to check what is happening on the emulator. Start it up by opening the 'tools' folder of 'android-sdk-macosx' in the terminal, and typing ./ddms. You will have to start ddms before starting the emulator, otherwise it will not show up.

To run your test, type calabash-android run in the terminal in the folder of your calabash project.

If you run the test for the first time you have to build the test server before running the test (see below). calabash-android run will run calabash-android build if it cannot find a test server.

Building the test server

Calabash will install an instrumentation app along with your app on the device to run the test. Because of some app specific information we need to build the test server based on the input you provided during setup. Please note that you need to rebuild the test server every time you change the app.

You build the test server like this:

calabash-android build

Predefined steps

The predefined steps are located in the features/step_definitions folder. A compiled list of predefined steps with comments is available here