Skip to content
This repository has been archived by the owner on Jul 23, 2021. It is now read-only.

testing strategy

Jump to bottom
Daniel Hunt edited this page Nov 6, 2018 · 5 revisions

Testing Strategy

Scope

In-Scope:

  • Functional and non-functional user story testing
  • Unit, integration, and system testing for the developed code
  • Testing for any usability and performance scenarios
  • User Acceptance testing against a simulated environment using VMs

Out-of-Scope:

  • Security testing
  • Penetration testing

Strategy

For each sprint the test strategy is split broadly into development testing, system testing, and user acceptance testing. These will be carried out by various members of the project team at different stages

Progression through the testing stages

Builds will progress through the various stages when they succeed at all previous levels. For example, builds passing all unit and integration will be passed to system testing, etc.

Generally the distribution of the number of tests and the required time for execution should roughly follow the pyramid shown in the diagram below and explained in this blog post by Uncle Bob https://codingjourneyman.com/2014/09/24/the-clean-coder-testing-strategies/

Development Testing

Development testing is primarily managed and maintained by the developers. Developers have a key role in the testing process in addition to the tester; unit and integration testing are the sole responsibility of developers and an important part of the development activity.

  • Automated Unit and Integration tests will be created and monitored around all the features where business logic materialises.
  • Pull requests will require peer review and approval. Other formal and informal peer reviews can be performed throughout the project to ensure coding standards and good practice guidelines are followed.
  • Code will have automated code analysis and unit test code coverage tools to monitor other quality metrics.

System Testing

Most system testing will be story-based against pieces of development as they come through. Where possible there will also be automated end-to-end (E2E) regression tests created that run using the latest code.

One advantage of automating some of these tests is that they can be run frequently eg as part of every build rather than waiting for a deployment of the system. Due to the current lack of a build agent these will likely need to be run manually at regular intervals rather than for each new build.

System testing will also include some unscripted testing, whereby if developers and testers during development identify previously unflagged possible risk areas then these may be investigated. Any discovered bugs would subsequently be reported.

Defect Management

If the story is still open then the developer will be informed of the defect and will be expected to fix it as part of the completion of the story. If bugs are found that appear to be unrelated to any currently open stories/development then a new item will be recorded in the backlog and prioritised accordingly.

User Acceptance Testing

In Sprint review meetings, the project team will demonstrate the user story implementation in the Test environment (this may be synonymous with the Dev environment in the early stages). Business users will be talked through the acceptance criteria for each committed story in the sprint using the latest version of master. Post-review meeting, the business users may perform their user acceptance testing by following SMG guidelines to run the GridFTP solution themselves to evaluate the functionality and usability of the application. Any issue or suggestions identified in the user evaluation period shall be communicated to the development team for further action; most likely story creation and prioritisation.

Automated System Testing

There is an automated system test framework set-up. Here are the steps required to run these:

  • First follow the instructions to setup a development environment as here

  • ssh into machine1 and then run python36 -m venv . from ~ (this creates a python virtual environment that is needed to run the tests)

  • Then use source ./bin/activate to enter the python virtual environment

  • Then cd systemTests and pip install -r requirements.txt to install the required python packages

  • Finally cd test-scenarios and run the system tests using pytest

Deployment Testing

TBD

Specific Validation Goals

This section will contain several high-level test scenarios that when passed will constitute a reasonable indicator of whether this project stream is complete

  • A GridFTP plugin can be created and deployed

  • A GridFTP plugin can GET the permissions of a file

  • A GridFTP plugin can SET the permissions of a file

  • A GridFTP plugin can GET the permissions of a file on input and SET the same permissions when the file is transferred

  • To Be Continued

Clone this wiki locally