Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Outlining the difference between Allure severity levels #750

Merged
merged 7 commits into from
Jul 24, 2020

Conversation

sta1r
Copy link
Contributor

@sta1r sta1r commented Jul 8, 2020

This PR attempts to clarify the difference between the different levels of Severity (MINOR, AVERAGE, MAJOR, CRITICAL or BLOCKER), as per Magento's customised use of this Allure annotation.

Description

When writing MFTF tests, it's often difficult to know what severity level to apply. Currently, the docs don't explain this.

I've learned that Magento has its own custom set of levels that are mapped to Allure's level names. I haven't linked to Allure's site or provided a table indicating the matching of Magento names to Allure names, because I don't think this is very helpful. I've just edited Alex Kolesnyk's explanation to me of the levels vis a vis their impact on conversion rate.

Broken links to devhub.io are replaced with github.com paths.

Affected pages

Contribution checklist

  • Pull request has a meaningful description of its purpose
  • All commits are accompanied by meaningful commit messages
  • All new or changed code is covered with unit/verification tests (if applicable)
  • All automated tests passed successfully (all builds on Travis CI are green)
  • Changes to Framework doesn't have backward incompatible changes for tests or have related Pull Request with fixes to tests

@coveralls
Copy link

coveralls commented Jul 8, 2020

Coverage Status

Coverage remained the same at 56.074% when pulling c996fb4 on sta1r:AM/severity-levels into ef5be1d on magento:develop.

dobooth
dobooth previously approved these changes Jul 13, 2020
@okolesnyk okolesnyk changed the base branch from develop to master July 13, 2020 14:50
@okolesnyk okolesnyk dismissed dobooth’s stale review July 13, 2020 14:50

The base branch was changed.

@okolesnyk okolesnyk changed the base branch from master to develop July 13, 2020 14:51
@okolesnyk okolesnyk self-requested a review July 13, 2020 14:51
okolesnyk
okolesnyk previously approved these changes Jul 13, 2020
@dobooth dobooth added the DOC label Jul 24, 2020
@dobooth dobooth changed the base branch from develop to annual-docs July 24, 2020 14:03
@dobooth dobooth changed the base branch from annual-docs to develop July 24, 2020 14:03
@dobooth dobooth dismissed okolesnyk’s stale review July 24, 2020 14:03

The base branch was changed.

@tomreece tomreece changed the base branch from develop to annual-docs July 24, 2020 14:57
@tomreece tomreece changed the base branch from annual-docs to develop July 24, 2020 14:57
@tomreece tomreece changed the base branch from develop to annual-docs July 24, 2020 15:04
@tomreece tomreece changed the base branch from annual-docs to develop July 24, 2020 15:05
@tomreece tomreece changed the base branch from develop to master July 24, 2020 15:07
@tomreece tomreece changed the base branch from master to develop July 24, 2020 15:07
@okolesnyk okolesnyk force-pushed the AM/severity-levels branch from 194cf0b to f958045 Compare July 24, 2020 15:09
@tomreece tomreece changed the base branch from develop to annual-docs July 24, 2020 15:10
@tomreece tomreece changed the base branch from annual-docs to develop July 24, 2020 15:11
@tomreece tomreece changed the base branch from develop to master July 24, 2020 15:11
@tomreece tomreece changed the base branch from master to develop July 24, 2020 15:12
@tomreece tomreece changed the base branch from develop to master July 24, 2020 15:44
@tomreece tomreece changed the base branch from master to develop July 24, 2020 15:45
Copy link
Contributor

@tomreece tomreece left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved by @dobooth also.

@tomreece tomreece merged commit b8ae00d into magento:develop Jul 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants