You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Expected behavior
Tests in parameterized describe block are not reported as retries. Each describe block is expected to be treated as a separate test item if parameterized (as it is in native 'html' reporter).
Additional context
Reproducible with (at least) playwright 1.34.3/1.36.1 and allure-playwright 2.4.0.
Playwright html reporter:
Allure:
The text was updated successfully, but these errors were encountered:
I can confirm that tests in a parameterised describe block are reported as retries ("allure-playwright": "^2.5.0","@playwright/test": "^1.36.2")
This is very confusing because when you open the Tree of the Launch, for the given file you only see the last parameterised run, so it looks like only those tests ran. However, when you click on a given test and go to retries, you actually see these are not "retries", but are the other parameterised tests. In our case, there's a file with 1 describe block, which contains 4 tests and is parameterised against 3 values (to be fully precise, parameterised values are 5, i.e. we have an array of 5 values, but the last two are skipped, so we run against the 0-2 values). In Allure TestOps launch tree view, this file is reported to have ran 8 tests; but when you click each test and go to retries, there are 2 more tests, which are the other 2 parameterised values. We only see the run against the last parameterised value in the launch tree view for each browser and the runs against the previous 2 parameterised values are only visible in the "retries" section. So, when we ran against 2 browsers, the launch's tree view showed only 8 ran tests in that file, but in reality 24 tests were ran (2 browsers * 3 parameterised values * 4 tests) - the rest of the tests were only visible in the retries section of each test visible in the launch tree view.
Hi guys!
Describe the bug
Tests are unexpectedly reported as retries when wrapped with parameterized describe block.
To Reproduce
Expected behavior
Tests in parameterized describe block are not reported as retries. Each describe block is expected to be treated as a separate test item if parameterized (as it is in native 'html' reporter).
Additional context
Reproducible with (at least) playwright 1.34.3/1.36.1 and allure-playwright 2.4.0.
Playwright html reporter:
Allure:
The text was updated successfully, but these errors were encountered: