TestScript Imports

TestScript is a FHIR standard resource used for writing tests. A single TestScript resource is interpreted resulting in a single TestReport resource containing the results. While this approach works it does not allow for elements of a TestScript to be reused. Reuse is important when building a large number of tests. Common elements should be code separately and included in tests. This is an application of the DRY concept in software - Don't Repeat Yourself. FHIR resource have two extensibility features, extension and containment, which are used in this proposal.

There is a second approach to reuse possible with TestScript. Tests could be coded and maintained in an external language, not part of the FHIR standard, and then translated into TestScript. This makes TestScript an interchange language. In this environment reuse is not an issue. We are not pursuing this approach.

TestScript features

The following TestScript features are relevant.

Action - operation and assert - the primary building block of TestScript is the action. There are two types of actions: operation and assert. Operation is an interaction with a System Under Test (SUT) like POST or GET. Assert is an evaluation - the if part of an if-then statement. The then part of the statement is to register an error or warning.

Test - a collection of actions executed in the order coded. A script (instance of TestScript) contains one or more tests.

Setup - a collection of actions to be run before tests to initialize the SUT for testing.

Teardown - a collection of actions run after the tests to cleanup the SUT. Teardown includes operations only (no asserts).

Fixture - a container for a resource. Fixtures can be static or dynamic. Static Fixtures are defined with the test and are used to initialize the SUT. Dynamic Fixtures hold the result (response) of an Operation. Dynamic Fixtures include a header and body (resource) - the entire message returned from the SUT. Static Fixtures contain only the resource (body). Operations that include a body (POST) point to a Fixture to define the body contents.

Variable - reference to part of a Fixture. Variables are used to extract content out of a Fixture header or body for inclusion in an operation or assert.

Targets of reuse

In this structure defined in TestScript there are three elements targeted for reuse: Fixtures, Variables, and Tests.

Fixtures and Variables both identify data. They are similar in concept to GETTERS in Java. They define data access and they have a built in assumption that some operation has occurred to create that data. In Java those operations would be object creation and method invocation. In TestScript the operations are Operations.

Tests are reusable because they are hard to create and maintain (DRY principle).

Scripts and Libraries

TestScripts are organized into two groups: scripts and libraries. Scripts are the documented tests a test harness can run against an SUT. They might be visible in a testing user interface. Libraries are collections of components used to build scripts - the elements of reuse.

Focus of reuse - use cases

• Setup - common SUT setup
• Test - entire Test
• Operation - single interaction with SUT
• Assert - one or more if-then's

This list is collapsed into a single element - the Test. Setup is coded as a list of one or more Actions - a Test is a list of one or more actions. A Test can contain a single Operation or Assert. The focus on Test includes all the capabilities of TestScript.

By focusing on Test, Fixture, and Variable, all the key capabilities of TestScript are available for reuse.