Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test(htlc-eth-besu-erc20): fix flaky refund test case #1150

Closed
petermetz opened this issue Jul 20, 2021 · 1 comment
Closed

test(htlc-eth-besu-erc20): fix flaky refund test case #1150

petermetz opened this issue Jul 20, 2021 · 1 comment
Assignees
@petermetz
Labels
Besu bug Something isn't working Flaky-Test-Automation Issues related to test stability (which is a long running issue that can never fully be solved) Tests Anything related to tests be that automatic or manual, integration or unit, etc.

Comments

@petermetz
Copy link
Contributor

Describe the bug

Most of the time it passes, but this one time it failed here:

packages/cactus-test-plugin-htlc-eth-besu-erc20/src/test/typescript/integration/plugin-htlc-eth-besu-erc20/refund-endpoint.test.ts:572

To Reproduce

Keep running the test long enough and at a certain point it should fail (not very helpful I know)

Expected behavior

Test is consistent and stable, either always fails or always succeeds.

Logs/Stack traces

2021-07-20T17:09:26.2684363Z [2021-07-20T17:09:26.259Z] WARN (PluginKeychainMemory): Never use PluginKeychainMemory in production. It does not support encryption. It stores everything in plain text.
2021-07-20T17:09:59.0844704Z not ok 103 - packages/cactus-test-plugin-htlc-eth-besu-erc20/src/test/typescript/integration/plugin-htlc-eth-besu-erc20/refund-endpoint.test.ts # time=158522.174ms
2021-07-20T17:09:59.0847234Z   ---
2021-07-20T17:09:59.0847778Z   env:
2021-07-20T17:09:59.0848375Z     TS_NODE_COMPILER_OPTIONS: "{}"
2021-07-20T17:09:59.0850364Z   file: packages/cactus-test-plugin-htlc-eth-besu-erc20/src/test/typescript/integration/plugin-htlc-eth-besu-erc20/refund-endpoint.test.ts
2021-07-20T17:09:59.0851887Z   timeout: 3600000
2021-07-20T17:09:59.0852609Z   command: /opt/hostedtoolcache/node/16.2.0/x64/bin/node
2021-07-20T17:09:59.0853261Z   args:
2021-07-20T17:09:59.0853882Z     - -r
2021-07-20T17:09:59.0854926Z     - /home/runner/work/cactus/cactus/node_modules/ts-node/register/index.js
2021-07-20T17:09:59.0855987Z     - --max-old-space-size=4096
2021-07-20T17:09:59.0857973Z     - packages/cactus-test-plugin-htlc-eth-besu-erc20/src/test/typescript/integration/plugin-htlc-eth-besu-erc20/refund-endpoint.test.ts
2021-07-20T17:09:59.0859570Z   stdio:
2021-07-20T17:09:59.0860183Z     - 0
2021-07-20T17:09:59.0861034Z     - pipe
2021-07-20T17:09:59.0861683Z     - 2
2021-07-20T17:09:59.0862287Z   cwd: /home/runner/work/cactus/cactus
2021-07-20T17:09:59.0862886Z   exitCode: 1
2021-07-20T17:09:59.0863358Z   ...
2021-07-20T17:09:59.0867592Z {
2021-07-20T17:09:59.0873270Z     # BEFORE Test refund endpoint
2021-07-20T17:09:59.0899257Z     Detected current process to be running inside a Github Action. Pruning all docker resources...
2021-07-20T17:09:59.0901349Z     [2021-07-20T17:07:34.388Z] INFO (Containers#pruneDockerResources()): Ignoring docker resource cleanup command failure. Error: Command failed with exit code 1: docker rmi
2021-07-20T17:09:59.0902519Z     "docker rmi" requires at least 1 argument.
2021-07-20T17:09:59.0903258Z     See 'docker rmi --help'.
2021-07-20T17:09:59.0903679Z     
2021-07-20T17:09:59.0904196Z     Usage:  docker rmi [OPTIONS] IMAGE [IMAGE...]
2021-07-20T17:09:59.0904684Z     
2021-07-20T17:09:59.0905122Z     Remove one or more images
2021-07-20T17:09:59.0905926Z         at makeError (/home/runner/work/cactus/cactus/node_modules/execa/lib/error.js:60:11)
2021-07-20T17:09:59.0906968Z         at handlePromise (/home/runner/work/cactus/cactus/node_modules/execa/index.js:118:26)
2021-07-20T17:09:59.0908062Z         at processTicksAndRejections (node:internal/process/task_queues:96:5)
2021-07-20T17:09:59.0909981Z         at Function.pruneDockerResources (/home/runner/work/cactus/cactus/packages/cactus-test-tooling/dist/lib/main/typescript/common/containers.js:64:2283)
2021-07-20T17:09:59.0912843Z         at Object.pruneDockerAllIfGithubAction (/home/runner/work/cactus/cactus/packages/cactus-test-tooling/src/main/typescript/github-actions/prune-docker-all-if-github-action.ts:9:1)
2021-07-20T17:09:59.0916265Z         at /home/runner/work/cactus/cactus/packages/cactus-test-plugin-htlc-eth-besu-erc20/src/test/typescript/integration/plugin-htlc-eth-besu-erc20/refund-endpoint.test.ts:64:3 {
2021-07-20T17:09:59.0918674Z       shortMessage: 'Command failed with exit code 1: docker rmi',
2021-07-20T17:09:59.0919578Z       command: 'docker rmi',
2021-07-20T17:09:59.0920287Z       escapedCommand: 'docker rmi',
2021-07-20T17:09:59.0920823Z       exitCode: 1,
2021-07-20T17:09:59.0921283Z       signal: undefined,
2021-07-20T17:09:59.0922032Z       signalDescription: undefined,
2021-07-20T17:09:59.0922677Z       stdout: '',
2021-07-20T17:09:59.0923443Z       stderr: '"docker rmi" requires at least 1 argument.\n' +
2021-07-20T17:09:59.0924216Z         "See 'docker rmi --help'.\n" +
2021-07-20T17:09:59.0924755Z         '\n' +
2021-07-20T17:09:59.0925462Z         'Usage:  docker rmi [OPTIONS] IMAGE [IMAGE...]\n' +
2021-07-20T17:09:59.0926069Z         '\n' +
2021-07-20T17:09:59.0926854Z         'Remove one or more images',
2021-07-20T17:09:59.0927665Z       all: '"docker rmi" requires at least 1 argument.\n' +
2021-07-20T17:09:59.0928451Z         "See 'docker rmi --help'.\n" +
2021-07-20T17:09:59.0929009Z         '\n' +
2021-07-20T17:09:59.0929741Z         'Usage:  docker rmi [OPTIONS] IMAGE [IMAGE...]\n' +
2021-07-20T17:09:59.0930555Z         '\n' +
2021-07-20T17:09:59.0931163Z         'Remove one or more images',
2021-07-20T17:09:59.0931658Z       failed: true,
2021-07-20T17:09:59.0932103Z       timedOut: false,
2021-07-20T17:09:59.0932762Z       isCanceled: false,
2021-07-20T17:09:59.0933203Z       killed: false
2021-07-20T17:09:59.0933750Z     }
2021-07-20T17:09:59.0934176Z     ok 1 Pruning did not throw OK
2021-07-20T17:09:59.0934700Z     # Test refund endpoint
2021-07-20T17:09:59.0935220Z     # Starting Besu Test Ledger
2021-07-20T17:09:59.0936743Z     [2021-07-20T17:08:03.809Z] INFO (PluginKeychainMemory): Created PluginKeychainMemory. KeychainID=2d9b6cc5-208f-4d90-80ec-60cab9c9ba71
2021-07-20T17:09:59.0938031Z     # Deploys HashTimeLock via .json file on initialize function
2021-07-20T17:09:59.0938871Z     ok 2 pluginHtlc.initialize() output is truthy OK
2021-07-20T17:09:59.0939924Z     ok 3 pluginHtlc.initialize() output.transactionReceipt is truthy OK
2021-07-20T17:09:59.0941442Z     ok 4 pluginHtlc.initialize() output.transactionReceipt.contractAddress is truthy OK
2021-07-20T17:09:59.0942795Z     # Deploys TestToken via .json file on deployContract function
2021-07-20T17:09:59.0943564Z     ok 5 deployContract() output is truthy OK
2021-07-20T17:09:59.0944611Z     ok 6 deployContract() output.transactionReceipt is truthy OK
2021-07-20T17:09:59.0945934Z     ok 7 deployContract() output.transactionReceipt.contractAddress is truthy OK
2021-07-20T17:09:59.0947164Z     # Deploys DemoHelpers via .json file on deployContract function
2021-07-20T17:09:59.0947932Z     ok 8 deployContract() output is truthy OK
2021-07-20T17:09:59.0948776Z     ok 9 deployContract() output.transactionReceipt is truthy OK
2021-07-20T17:09:59.0950102Z     ok 10 deployContract() output.transactionReceipt.contractAddress is truthy OK
2021-07-20T17:09:59.0951448Z     # Approve 10 Tokens to HashTimeLockAddress
2021-07-20T17:09:59.0952335Z     ok 11 approve() transactionReceipt.status is true OK
2021-07-20T17:09:59.0953048Z     # Get account balance
2021-07-20T17:09:59.0953582Z     ok 12 balance of account is 100 OK
2021-07-20T17:09:59.0992165Z     # Get current timestamp
2021-07-20T17:09:59.0992841Z     ok 13 callOutput() output.callOutput is truthy OK
2021-07-20T17:09:59.0994070Z     # Get HashTimeLock contract and account allowance
2021-07-20T17:09:59.0994693Z     ok 14 callOutput() is 10 OK
2021-07-20T17:09:59.0995185Z     # Create new contract for HTLC
2021-07-20T17:09:59.0995895Z     ok 15 response status is 200 OK
2021-07-20T17:09:59.0996589Z     # Get account balance
2021-07-20T17:09:59.0997101Z     ok 16 balance of account is 90 OK
2021-07-20T17:09:59.0997653Z     # Get HTLC Id from DemoHelper
2021-07-20T17:09:59.0998165Z     ok 17 result is truthy OK
2021-07-20T17:09:59.0998619Z     # Refund HTLC
2021-07-20T17:09:59.1001853Z     ok 18 response status is 200 OK
2021-07-20T17:09:59.1002688Z     # Get account balance
2021-07-20T17:09:59.1003162Z     ok 19 balance of account is 100 OK
2021-07-20T17:09:59.1003816Z     # Get status of HTLC
2021-07-20T17:09:59.1010939Z     ok 20 response status is 200 OK
2021-07-20T17:09:59.1012049Z     ok 21 the contract status is 2 - Refunded
2021-07-20T17:09:59.1012643Z     # Test invalid refund with invalid time
2021-07-20T17:09:59.1013172Z     # Starting Besu Test Ledger
2021-07-20T17:09:59.1014946Z     [2021-07-20T17:09:26.259Z] INFO (PluginKeychainMemory): Created PluginKeychainMemory. KeychainID=52b71a10-d250-4804-92b2-ded7068ecc7d
2021-07-20T17:09:59.1016172Z     # Deploys HashTimeLock via .json file on initialize function
2021-07-20T17:09:59.1017026Z     ok 22 pluginHtlc.initialize() output is truthy OK
2021-07-20T17:09:59.1018149Z     ok 23 pluginHtlc.initialize() output.transactionReceipt is truthy OK
2021-07-20T17:09:59.1020155Z     ok 24 pluginHtlc.initialize() output.transactionReceipt.contractAddress is truthy OK
2021-07-20T17:09:59.1021501Z     # Deploys TestToken via .json file on deployContract function
2021-07-20T17:09:59.1022272Z     ok 25 deployContract() output is truthy OK
2021-07-20T17:09:59.1023359Z     ok 26 deployContract() output.transactionReceipt is truthy OK
2021-07-20T17:09:59.1024732Z     ok 27 deployContract() output.transactionReceipt.contractAddress is truthy OK
2021-07-20T17:09:59.1026029Z     # Deploys DemoHelpers via .json file on deployContract function
2021-07-20T17:09:59.1026994Z     ok 28 deployContract() output is truthy OK
2021-07-20T17:09:59.1027841Z     ok 29 deployContract() output.transactionReceipt is truthy OK
2021-07-20T17:09:59.1029173Z     ok 30 deployContract() output.transactionReceipt.contractAddress is truthy OK
2021-07-20T17:09:59.1030324Z     # Approve 10 Tokens to HashTimeLockAddress
2021-07-20T17:09:59.1031382Z     ok 31 approve() transactionReceipt.status is true OK
2021-07-20T17:09:59.1032086Z     # Get account balance
2021-07-20T17:09:59.1032614Z     ok 32 balance of account is 100 OK
2021-07-20T17:09:59.1033144Z     # Get current timestamp
2021-07-20T17:09:59.1143777Z     ok 33 callOutput() output.callOutput is truthy OK
2021-07-20T17:09:59.1144859Z     # Get HashTimeLock contract and account allowance
2021-07-20T17:09:59.1145492Z     ok 34 callOutput() is 10 OK
2021-07-20T17:09:59.1146040Z     # Create new contract for HTLC
2021-07-20T17:09:59.1146580Z     ok 35 response status is 200 OK
2021-07-20T17:09:59.1147142Z     # Get HTLC Id from DemoHelper
2021-07-20T17:09:59.1147660Z     ok 36 result is truthy OK
2021-07-20T17:09:59.1148242Z     # Refund HTLC when it is not possible yet
2021-07-20T17:09:59.1148839Z     not ok 37 response status is 400
2021-07-20T17:09:59.1149958Z       ---
2021-07-20T17:09:59.1150379Z         operator: equal
2021-07-20T17:09:59.1150820Z         expected: 400
2021-07-20T17:09:59.1151344Z         actual:   200
2021-07-20T17:09:59.1153551Z         at: <anonymous> (/home/runner/work/cactus/cactus/packages/cactus-test-plugin-htlc-eth-besu-erc20/src/test/typescript/integration/plugin-htlc-eth-besu-erc20/refund-endpoint.test.ts:572:7)
2021-07-20T17:09:59.1155451Z         stack: |-
2021-07-20T17:09:59.1155953Z           Error: response status is 400
2021-07-20T17:09:59.1156842Z               at Test.assert [as _assert] (/home/runner/work/cactus/cactus/node_modules/tape/lib/test.js:260:54)
2021-07-20T17:09:59.1157934Z               at Test.bound [as _assert] (/home/runner/work/cactus/cactus/node_modules/tape/lib/test.js:84:32)
2021-07-20T17:09:59.1159335Z               at Test.strictEqual (/home/runner/work/cactus/cactus/node_modules/tape/lib/test.js:424:10)
2021-07-20T17:09:59.1160388Z               at Test.bound (/home/runner/work/cactus/cactus/node_modules/tape/lib/test.js:84:32)
2021-07-20T17:09:59.1162913Z               at /home/runner/work/cactus/cactus/packages/cactus-test-plugin-htlc-eth-besu-erc20/src/test/typescript/integration/plugin-htlc-eth-besu-erc20/refund-endpoint.test.ts:572:7
2021-07-20T17:09:59.1164843Z               at processTicksAndRejections (node:internal/process/task_queues:96:5)
2021-07-20T17:09:59.1165571Z       ...
2021-07-20T17:09:59.1165992Z     # Get balance of account
2021-07-20T17:09:59.1166878Z     Bail out! response status is 400
2021-07-20T17:09:59.1167376Z }
2021-07-20T17:09:59.1167865Z Bail out! response status is 400
2021-07-20T17:10:06.1434503Z [2021-07-20T17:10:06.142Z] WARN (PluginKeychainMemory): Never use PluginKeychainMemory in production. It does not support encryption. It stores everything in plain text.
2021-07-20T17:10:11.3160840Z (node:174008) [DEP0152] DeprecationWarning: Custom PerformanceEntry accessors are deprecated. Please use the detail property.
2021-07-20T17:10:11.3163274Z (Use `node --trace-deprecation ...` to show where the warning was created)
2021-07-20T17:10:35.5205155Z [2021-07-20T17:10:35.520Z] WARN (PluginKeychainMemory): Never use PluginKeychainMemory in production. It does not support encryption. It stores everything in plain text.
2021-07-20T17:10:45.8042647Z [2021-07-20T17:10:45.802Z] ERROR (WithdrawEndpoint): WithdrawEndpoint#handleRequest() failed to serve request Error: Transaction has been reverted by the EVM:
2021-07-20T17:10:45.8043737Z {
2021-07-20T17:10:45.8045016Z   "blockHash": "0x8809969be8a39b364e0316241d8e2d9bb61e991e1864bfb32420c2df835b7b88",
2021-07-20T17:10:45.8045905Z   "blockNumber": 12,
2021-07-20T17:10:45.8046388Z   "contractAddress": null,
2021-07-20T17:10:45.8046947Z   "cumulativeGasUsed": 25158,
2021-07-20T17:10:45.8047668Z   "from": "0x90f8bf6a479f320ead074411a4b0e7944ea8c9c1",
2021-07-20T17:10:45.8048297Z   "gasUsed": 25158,
2021-07-20T17:10:45.8048837Z   "logs": [],
2021-07-20T17:10:45.8050187Z   "logsBloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
2021-07-20T17:10:45.8051177Z   "status": false,
2021-07-20T17:10:45.8052018Z   "to": "0xe78a0f7e598cc8b0bb87894b0f60dd2a88d6a8ab",
2021-07-20T17:10:45.8053202Z   "transactionHash": "0x11e40b18994d7669975c4202f3aeafe828124f824f475fd63393a8a4d39d59c3",
2021-07-20T17:10:45.8054145Z   "transactionIndex": 0,
2021-07-20T17:10:45.8055368Z   "revertReason": "0x08c379a00000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000000f535741505f4e4f545f4143544956450000000000000000000000000000000000"
2021-07-20T17:10:45.8056339Z }
2021-07-20T17:10:45.8057575Z     at Object.TransactionError (/home/runner/work/cactus/cactus/node_modules/web3-core-helpers/src/errors.js:93:21)
2021-07-20T17:10:45.8060070Z     at Object.TransactionRevertedWithoutReasonError (/home/runner/work/cactus/cactus/node_modules/web3-core-helpers/src/errors.js:105:21)
2021-07-20T17:10:45.8062054Z     at /home/runner/work/cactus/cactus/node_modules/web3-core-method/src/index.js:474:48
2021-07-20T17:10:45.8063331Z     at processTicksAndRejections (node:internal/process/task_queues:96:5) {
2021-07-20T17:10:45.8064346Z   receipt: {
2021-07-20T17:10:45.8065693Z     blockHash: '0x8809969be8a39b364e0316241d8e2d9bb61e991e1864bfb32420c2df835b7b88',
2021-07-20T17:10:45.8069854Z     blockNumber: 12,
2021-07-20T17:10:45.8071347Z     contractAddress: null,
2021-07-20T17:10:45.8072125Z     cumulativeGasUsed: 25158,
2021-07-20T17:10:45.8073237Z     from: '0x90f8bf6a479f320ead074411a4b0e7944ea8c9c1',
2021-07-20T17:10:45.8074024Z     gasUsed: 25158,
2021-07-20T17:10:45.8074419Z     logs: [],
2021-07-20T17:10:45.8077020Z     logsBloom: '0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000',
2021-07-20T17:10:45.8078024Z     status: false,
2021-07-20T17:10:45.8079123Z     to: '0xe78a0f7e598cc8b0bb87894b0f60dd2a88d6a8ab',
2021-07-20T17:10:45.8080528Z     transactionHash: '0x11e40b18994d7669975c4202f3aeafe828124f824f475fd63393a8a4d39d59c3',
2021-07-20T17:10:45.8081460Z     transactionIndex: 0,
2021-07-20T17:10:45.8083653Z     revertReason: '0x08c379a00000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000000f535741505f4e4f545f4143544956450000000000000000000000000000000000'
2021-07-20T17:10:45.8084647Z   }
2021-07-20T17:10:45.8084966Z }

Screenshots

N/A

Cloud provider or hardware configuration:

GHA Runner

Operating system name, version, build:

Ubuntu 20.04

Hyperledger Cactus release version or commit (git rev-parse --short HEAD):

v0.6.0

Hyperledger Cactus Plugins/Connectors Used

Besu
@petermetz petermetz added bug Something isn't working Besu labels Jul 20, 2021
@petermetz petermetz added Flaky-Test-Automation Issues related to test stability (which is a long running issue that can never fully be solved) Tests Anything related to tests be that automatic or manual, integration or unit, etc. labels Nov 17, 2021
@awadhana awadhana mentioned this issue Feb 24, 2022
Closed
outSH added a commit to outSH/cactus that referenced this issue Jul 4, 2022
@outSH
test(cactus): remove flaky tests from main CI execution
c34e857 
- Remove flaky tests from main CI execution.
- Add separate TAP and jest configuration for running only flaky tests.
- Add test:*:flaky scripts in the main package.json
- Add new step to github CI script to run flaky tests as optional
  (i.e. don't fail if flaky test fails)

Closes: hyperledger-cacti#2112

Related: hyperledger-cacti#2104
Related: hyperledger-cacti#1626
Related: hyperledger-cacti#2019
Related: hyperledger-cacti#1625
Related: hyperledger-cacti#1543
Related: hyperledger-cacti#1598
Related: hyperledger-cacti#1528
Related: hyperledger-cacti#1521
Related: hyperledger-cacti#1495
Related: hyperledger-cacti#1485
Related: hyperledger-cacti#1473
Related: hyperledger-cacti#1471
Related: hyperledger-cacti#1469
Related: hyperledger-cacti#1150

Signed-off-by: Michal Bajer <michal.bajer@fujitsu.com>
@outSH outSH mentioned this issue Jul 4, 2022
Closed
@petermetz
Copy link
Contributor Author

Haven't seen this fail in a long while so closing this for now.

@petermetz petermetz closed this as not planned Won't fix, can't repro, duplicate, stale Sep 8, 2024
@petermetz petermetz self-assigned this Sep 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@petermetz petermetz

Labels
Besu bug Something isn't working Flaky-Test-Automation Issues related to test stability (which is a long running issue that can never fully be solved) Tests Anything related to tests be that automatic or manual, integration or unit, etc.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@petermetz