Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Backwards Compatibility] update instructions in TESTING.md #1030

Merged
merged 1 commit into from
Jan 13, 2022
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 19 additions & 5 deletions TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,19 @@ Overview
- [Unit tests](#unit-tests)
- [Integration tests](#integration-tests)
- [Functional tests](#functional-tests)
- [Backwards Compatibility tests](#backwards-compatibility-tests)
- [Continuous Integration](#continuous-integration)
- [Environment misc](#environment-misc)
- [Misc](#misc)

# General information
OpenSearch Dashboards uses [Jest](https://jestjs.io/) for unit and integration tests and [Selenium](https://www.selenium.dev/) for functional tests.
OpenSearch Dashboards uses [Jest](https://jestjs.io/) for unit and integration tests, [Selenium](https://www.selenium.dev/) for functional tests, and [Cypress](https://www.cypress.io/) for backwards compatibility tests.

In general, we recommend three tiers of tests:
In general, we recommend four tiers of tests:
* Unit tests: Unit tests: small and modular tests that utilize mocks for external dependencies.
* Integration tests: higher-level tests that verify interactions between systems (eg. HTTP APIs, OpenSearch API calls, calling other plugin).
* End-to-end tests (e2e): functional tests that verify behavior in a web browser.
* Backwards Compatibility tests: cypress tests that verify any changes are backwards compatible with previous versions.

# Requirements
* Install the latest NodeJS, [NPM](https://www.npmjs.com/get-npm) and [Yarn](https://classic.yarnpkg.com/en/docs/install/#mac-stable)
Expand Down Expand Up @@ -48,7 +50,20 @@ To debug functional tests:
Say that you would want to debug a test in CI group 1, you can run the following command in your environment:
`node --debug-brk --inspect scripts/functional_tests.js --config test/functional/config.js --include ciGroup1 --debug`

This will print of an address, to which you could open your chrome browser on your instance and navigate to `chrome://inspect/#devices` and inspect the functional test runner `scripts/functional_tests.js`.
This will print off an address, to which you could open your chrome browser on your instance and navigate to `chrome://inspect/#devices` and inspect the functional test runner `scripts/functional_tests.js`.

### Backwards Compatibility tests
To run all the backwards compatibility tests on vanilla OpenSearch Dashboards:

`yarn test:bwc -o [test path to opensearch.tar.gz] -d [test path to opensearch-dashboards.tar.gz]`

To run all the backwards compatibility tests on bundled dashboards, pass the bundle parameter to the test:

`yarn test:bwc -o [test path to opensearch.tar.gz] -d [test path to opensearch-dashboards.tar.gz] -b true`

To run specific versions' backwards compatibility tests, pass the versions to the test:

`yarn test:bwc -o [test path to opensearch.tar.gz] -d [test path to opensearch-dashboards.tar.gz] -v "[test versions]"`

### Additional checks
Make sure you run lint checker before submitting a pull request. To run lint checker:
Expand All @@ -70,5 +85,4 @@ By default the version of OpenSearch Dashboards will pull the snapshot of the sa
Although Jest is the standard for this project, there are a few Mocha tests that still exist. You can run these tests by running:
`yarn test:mocha`

However, these tests will eventually be migrated. Please avoid writing new Mocha tests. For further questions or to check the status please see this [issue](https://github.com/opensearch-project/OpenSearch-Dashboards/issues/215).

However, these tests will eventually be migrated. Please avoid writing new Mocha tests. For further questions or to check the status please see this [issue](https://github.com/opensearch-project/OpenSearch-Dashboards/issues/215).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: We should keep the empty newline character.