Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test(connector-corda): fix flaky v4.7 test #1473

Open
petermetz opened this issue Oct 22, 2021 · 2 comments
Open

test(connector-corda): fix flaky v4.7 test #1473

petermetz opened this issue Oct 22, 2021 · 2 comments
Labels
bug Something isn't working Corda dependencies Pull requests that update a dependency file Developer_Experience Flaky-Test-Automation Issues related to test stability (which is a long running issue that can never fully be solved) good-first-issue Good for newcomers good-first-issue-400-expert Hacktoberfest Hacktoberfest participants are welcome to take a stab at issues marked with this label. help wanted Extra attention is needed Tests Anything related to tests be that automatic or manual, integration or unit, etc.

Comments

@petermetz
Copy link
Contributor

petermetz commented Oct 22, 2021 •
edited
Loading

This likely has the same solution as #1469

Describe the bug

We've got a flaky one at:
packages/cactus-plugin-ledger-connector-corda/src/test/typescript/integration/deploy-cordapp-jars-to-nodes-v4.7.test.ts

One of the corda nodes within the container occasionally don't boot up (hangs and then the test waits for an hour before giving up).

To Reproduce

Keep running the test and eventually you'll see it.

Expected behavior

Tests are stable, not flaky.

Logs/Stack traces

The CI logs from the GHA runner as of this morning:
logs_10174.zip

Cloud provider or hardware configuration:

GHA runner

Operating system name, version, build:

Ubuntu 20.04

Hyperledger Cactus release version or commit (git rev-parse --short HEAD):

1.0.0-rc.1

Hyperledger Cactus Plugins/Connectors Used

Corda

Additional context

I'll send a separate PR that skips this test to avoid flakiness in the meantime.
@petermetz petermetz added bug Something isn't working good-first-issue Good for newcomers help wanted Extra attention is needed Corda dependencies Pull requests that update a dependency file Developer_Experience Hacktoberfest Hacktoberfest participants are welcome to take a stab at issues marked with this label. good-first-issue-400-expert Flaky-Test-Automation Issues related to test stability (which is a long running issue that can never fully be solved) Tests Anything related to tests be that automatic or manual, integration or unit, etc. labels Oct 22, 2021
petermetz added a commit to petermetz/cacti that referenced this issue Oct 22, 2021
@petermetz
test: skip corda v4.7 deploy test due to flakiness
d7839b7 
Related to hyperledger-cacti#1473 but does NOT fix it, that is to come at a later point
in time when we have time for a full and proper diagnosis+fix.

Signed-off-by: Peter Somogyvari <peter.somogyvari@accenture.com>
@petermetz petermetz mentioned this issue Oct 22, 2021
Merged
petermetz added a commit that referenced this issue Oct 25, 2021
@petermetz
test: skip corda v4.7 deploy test due to flakiness
305171b 
Related to #1473 but does NOT fix it, that is to come at a later point
in time when we have time for a full and proper diagnosis+fix.

Signed-off-by: Peter Somogyvari <peter.somogyvari@accenture.com>
Leeyoungone pushed a commit to Leeyoungone/cactus that referenced this issue Oct 26, 2021
@petermetz @Leeyoungone
test: skip corda v4.7 deploy test due to flakiness
f454348 
Related to hyperledger-cacti#1473 but does NOT fix it, that is to come at a later point
in time when we have time for a full and proper diagnosis+fix.

Signed-off-by: Peter Somogyvari <peter.somogyvari@accenture.com>
@jagpreetsinghsasan
Copy link
Contributor

Fortunately I didn't face it for 3 runs.

@petermetz
Copy link
Contributor Author

@jagpreetsinghsasan Yeah that's what makes it especially hard to debug, that it works around 9 times out of 10. My best idea so far to make it easier to reproduce is to simulate the constraint hardware resources of the CI environment with a custom dev container for which I have a branch that I've been carrying around (e.g. rebasing onto upstream/main) for just the purpose of trying to reproduce CI-only flaky test issues:

main...petermetz:devcontainer-json-resource-limits-stash

RafaelAPB pushed a commit to RafaelAPB/blockchain-integration-framework that referenced this issue Mar 9, 2022
@petermetz @RafaelAPB
test: skip corda v4.7 deploy test due to flakiness
7048cb6 
Related to hyperledger-cacti#1473 but does NOT fix it, that is to come at a later point
in time when we have time for a full and proper diagnosis+fix.

Signed-off-by: Peter Somogyvari <peter.somogyvari@accenture.com>
outSH added a commit to outSH/cactus that referenced this issue Jul 4, 2022
@outSH
test(cactus): remove flaky tests from main CI execution
c34e857 
- Remove flaky tests from main CI execution.
- Add separate TAP and jest configuration for running only flaky tests.
- Add test:*:flaky scripts in the main package.json
- Add new step to github CI script to run flaky tests as optional
  (i.e. don't fail if flaky test fails)

Closes: hyperledger-cacti#2112

Related: hyperledger-cacti#2104
Related: hyperledger-cacti#1626
Related: hyperledger-cacti#2019
Related: hyperledger-cacti#1625
Related: hyperledger-cacti#1543
Related: hyperledger-cacti#1598
Related: hyperledger-cacti#1528
Related: hyperledger-cacti#1521
Related: hyperledger-cacti#1495
Related: hyperledger-cacti#1485
Related: hyperledger-cacti#1473
Related: hyperledger-cacti#1471
Related: hyperledger-cacti#1469
Related: hyperledger-cacti#1150

Signed-off-by: Michal Bajer <michal.bajer@fujitsu.com>
@outSH outSH mentioned this issue Jul 4, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working Corda dependencies Pull requests that update a dependency file Developer_Experience Flaky-Test-Automation Issues related to test stability (which is a long running issue that can never fully be solved) good-first-issue Good for newcomers good-first-issue-400-expert Hacktoberfest Hacktoberfest participants are welcome to take a stab at issues marked with this label. help wanted Extra attention is needed Tests Anything related to tests be that automatic or manual, integration or unit, etc.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@petermetz @jagpreetsinghsasan