Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(connector-iroha): fix flaky openapi-validation.test.ts #2019

Open
petermetz opened this issue May 12, 2022 · 0 comments
Open

test(connector-iroha): fix flaky openapi-validation.test.ts #2019

petermetz opened this issue May 12, 2022 · 0 comments
Labels
bug Something isn't working Flaky-Test-Automation Issues related to test stability (which is a long running issue that can never fully be solved) Iroha Bugs/features related to the Iroha ledger connector plugin P3 Priority 3: Medium Tests Anything related to tests be that automatic or manual, integration or unit, etc.

Comments

@petermetz
Copy link
Contributor

Describe the bug

packages/cactus-plugin-ledger-connector-iroha/src/test/typescript/integration/openapi/openapi-validation.test.ts

2022-05-11T13:36:50.6277195Z # Subtest: ./packages/cactus-plugin-ledger-connector-iroha/src/test/typescript/integration/openapi/openapi-validation.test.ts
2022-05-11T13:36:50.6298058Z     # BEFORE Iroha plugin openapi validation
2022-05-11T13:36:50.6309338Z     ok 1 Pruning didn't throw OK
2022-05-11T13:36:56.2102892Z     # Iroha plugin openapi validation
2022-05-11T13:36:56.2109020Z     not ok 2 Error: IrohaTestLedger#getContainerInfo() no image "ghcr.io/hyperledger/cactus-iroha-all-in-one:2021-08-16--1183"
2022-05-11T13:36:56.2116401Z       ---
2022-05-11T13:36:56.2117491Z         operator: error
2022-05-11T13:36:56.2137191Z         at: bound (/home/runner/work/cactus/cactus/node_modules/tape-promise/node_modules/onetime/index.js:30:12)
2022-05-11T13:36:56.2138115Z         stack: |-
2022-05-11T13:36:56.2140487Z           Error: IrohaTestLedger#getContainerInfo() no image "ghcr.io/hyperledger/cactus-iroha-all-in-one:2021-08-16--1183"
2022-05-11T13:36:56.2144521Z               at IrohaTestLedger.getContainerInfo (/home/runner/work/cactus/cactus/packages/cactus-test-tooling/src/main/typescript/iroha/iroha-test-ledger.ts:66:44)
2022-05-11T13:36:56.2145538Z               at processTicksAndRejections (internal/process/task_queues.js:95:5)
2022-05-11T13:36:56.2148513Z               at async IrohaTestLedger.waitForHealthCheck (/home/runner/work/cactus/cactus/packages/cactus-test-tooling/src/main/typescript/iroha/iroha-test-ledger.ts:66:44)
2022-05-11T13:36:56.2150417Z               at async EventEmitter.<anonymous> (/home/runner/work/cactus/cactus/packages/cactus-test-tooling/src/main/typescript/iroha/iroha-test-ledger.ts:66:44)
2022-05-11T13:36:56.2154007Z       ...
2022-05-11T13:36:56.2916352Z     # AFTER Iroha plugin openapi validation
2022-05-11T13:36:56.2920864Z     Bail out! Error: IrohaTestLedger#getContainerInfo() no image "ghcr.io/hyperledger/cactus-iroha-all-in-one:2021-08-16--1183"
2022-05-11T13:36:56.3563162Z Bail out! Error: IrohaTestLedger#getContainerInfo() no image "ghcr.io/hyperledger/cactus-iroha-all-in-one:2021-08-16--1183"

To Reproduce

It is a flake so you'll have to run the CI an unknown number of times for this to reproduce, randomly.

Expected behavior

Test should be stable.

Logs/Stack traces

2022-05-11T09_59_59_3281829Z.log

Cloud provider or hardware configuration:

GHA CI Runner

Operating system name, version, build:

Ubuntu 20.04

Hyperledger Cactus release version or commit (git rev-parse --short HEAD):

#1819

Hyperledger Cactus Plugins/Connectors Used

Iroha

@petermetz petermetz added bug Something isn't working Iroha Bugs/features related to the Iroha ledger connector plugin Flaky-Test-Automation Issues related to test stability (which is a long running issue that can never fully be solved) Tests Anything related to tests be that automatic or manual, integration or unit, etc. P3 Priority 3: Medium labels May 12, 2022
outSH added a commit to outSH/cactus that referenced this issue Jul 4, 2022
- Remove flaky tests from main CI execution.
- Add separate TAP and jest configuration for running only flaky tests.
- Add test:*:flaky scripts in the main package.json
- Add new step to github CI script to run flaky tests as optional
  (i.e. don't fail if flaky test fails)

Closes: hyperledger-cacti#2112

Related: hyperledger-cacti#2104
Related: hyperledger-cacti#1626
Related: hyperledger-cacti#2019
Related: hyperledger-cacti#1625
Related: hyperledger-cacti#1543
Related: hyperledger-cacti#1598
Related: hyperledger-cacti#1528
Related: hyperledger-cacti#1521
Related: hyperledger-cacti#1495
Related: hyperledger-cacti#1485
Related: hyperledger-cacti#1473
Related: hyperledger-cacti#1471
Related: hyperledger-cacti#1469
Related: hyperledger-cacti#1150

Signed-off-by: Michal Bajer <michal.bajer@fujitsu.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Flaky-Test-Automation Issues related to test stability (which is a long running issue that can never fully be solved) Iroha Bugs/features related to the Iroha ledger connector plugin P3 Priority 3: Medium Tests Anything related to tests be that automatic or manual, integration or unit, etc.
Projects
None yet
Development

No branches or pull requests

1 participant