test_strategy.rst

Testing Strategy for Tendrl project

Authors:	mkudlej@redhat.com

Test Strategy Identifier

This is master strategy for testing `Tendrl Project`_. In case of any questions please contact `USM QE team`_.

References

• IEEE 829 Test Plan Outline
• ISTQB syllabus
• Requirements defined by developers
• Tendrl documentation
• Tendrl wiki documentation
• UI designs

Introduction

This document describes testing strategy for Tendrl.

Test Items

Install documentation

Tendrl Install Documentation

Note

TODO: Reference documentation snapshots for a particular release. Moreover later, when/if Tendrl Project will have a proper documentation upstream as a ascidoc project, reference that instead.

Packages

Tendrl package repository

Features to be Tested

Tested features are described in testcases. Tested Gluster configurations are linked in testcases and are in :doc:`/configurations/index`. Testcases are split into categories according test type (for example :doc:`/minimal-acceptance/index`) or tested feature (for example :doc:`/web/index`). List of test case categories:

• :doc:`/api/index`
• :doc:`/installation/index`
• :doc:`/minimal-acceptance/index`
• :doc:`/system-integration/index`
• :doc:`/web/index`

Features not to be Tested

All Tendrl package dependencies and Grafana packages are not directly tested.

Approach

Tools used

• Selenium + Webstr - for automated UI tests
• Ansible - for provisioning and configuration machines - USMQE Setup, Tendrl Ansible
• Jenkins - for CI - Tendrl jobs at Centos CI
• KVM, Libvirt, Duffy - infrastructure for deploying machines - Centos CI wiki
• GDeploy - for setting up Gluster clusters

How to test

Manual testing

1. Install clean machines for every testcase (if 2 testcases are completely independent, orthogonal and they are not influenced each other, they can be tested one after another)
2. Read testcase description and related issues.
3. Test it
4. Think about enhancing testcase, make it more general, try to write semi-automated script for testing
5. Change testcase if there are any changes
6. If testcase is related to any issue, please close issue(s) in case of pass. Otherwise file new issue, add it to testcase, add testcase link to that issue.

Automated testing (CI)

There are Jenkins builds which periodically install all required machines, install Tendrl and run automated tests. It is our priority to create automated tests.

Tested configurations

All testcases are tested with basic configuration installed by USMQE Setup playbooks which will use roles from Tendrl Ansible. In addition, Gluster clusters are created and configured by Gdeploy.

Basic configuration for Gluster: 4 nodes with Gluster installed by Gdeploy

All tested configurations are included in related test cases.

Found Issues

If tester finds any issue, it should be documented in issue in related repository. It should include relevant information, see How to file bugs against the Tendrl stack

Responsibilities

Testers:

• Build infrastructure
• Create test cases
• Write automated test cases
• Resolve technical issues
• Track the defects created and make sure they are complete and correct