test(tendermint): migrate to local/offline containerized testnets #2128

onur-ozkan · 2024-05-28T11:10:45Z

This PR refactors the tendermint integration tests as follows:

Run a local ATOM testnet network inside a container and use it in the integration tests.
Run a local NUCLEUS testnet network inside a container and use it in the integration tests.
Run a local IBC relayer node inside a container connected to both the ATOM and NUCLEUS local testnets, and use it in the integration tests.
Add test coverage for Tendermint - ETH swap.

The files inside the .docker/container-state directory ensure that the testnet networks start with a fixed state for consistent test results.

Signed-off-by: onur-ozkan <work@onurozkan.dev>

… into offline-tendermint-testnets

Signed-off-by: onur-ozkan <work@onurozkan.dev>

… into offline-tendermint-testnets

Signed-off-by: onur-ozkan <work@onurozkan.dev>

mariocynicys

Awesome! 🎉

First review iter.

mm2src/mm2_main/tests/docker_tests/docker_tests_common.rs

mm2src/mm2_main/tests/docker_tests_main.rs

mm2src/mm2_main/tests/docker_tests/tendermint_tests.rs

mariocynicys · 2024-06-18T16:51:49Z

mm2src/mm2_main/tests/docker_tests/tendermint_tests.rs

-            0.008,
-        ));
+    lazy_static! {
+        // Simple lock used for running the swap tests sequentially.


but why tho?

They use same account seed; the same tendermint account tries to swap from multiple instances. Normally, we have this

komodo-defi-framework/mm2src/coins/tendermint/tendermint_coin.rs

Lines 891 to 919 in b0fd99e

pub(super) async fn seq_safe_send_raw_tx_bytes(

&self,

tx_payload: Any,

fee: Fee,

timeout_height: u64,

memo: String,

) -> Result<(String, Raw), TransactionErr> {

let (tx_id, tx_raw) = loop {

let tx_raw = try_tx_s!(self.any_to_signed_raw_tx(

try_tx_s!(self.priv_key_policy.activated_key_or_err()),

try_tx_s!(self.account_info(&self.account_id).await),

tx_payload.clone(),

fee.clone(),

timeout_height,

memo.clone(),

));

match self.send_raw_tx_bytes(&try_tx_s!(tx_raw.to_bytes())).compat().await {

Ok(tx_id) => break (tx_id, tx_raw),

Err(e) => {

if e.contains(ACCOUNT_SEQUENCE_ERR) {

debug!("Got wrong account sequence, trying again.");

continue;

}

return Err(crate::TransactionErr::Plain(ERRL!("{}", e)));

},

};

};

function which ensures we end up with the correct sequence id so the swap doesn't fail because of sequence mismatch errors, but somehow that didn't help (the node was keep giving the same sequence id instead of the new one, this is either docker testnet specific problem or it's cached for too long from the nucleus node).

Signed-off-by: onur-ozkan <work@onurozkan.dev>

mm2src/mm2_main/tests/docker_tests/tendermint_tests.rs

shamardy

Great Work 🔥 ! Only one non-blocker comment.

btw, test_tendermint_ibc_withdraw and test_tendermint_ibc_withdraw_hd failed in the latest run so they seem a bit unstable, do you think this is acceptable @onur-ozkan?

.docker/container-state/atom-testnet-data/data/application.db/LOG

onur-ozkan · 2024-07-03T06:29:28Z

btw, test_tendermint_ibc_withdraw and test_tendermint_ibc_withdraw_hd failed in the latest run so they seem a bit unstable, do you think this is acceptable @onur-ozkan?

It seems like IBC relayer wasn't ready for connections when those tests were invoked. I will try to handle that properly. Thanks!

Signed-off-by: onur-ozkan <work@onurozkan.dev>

shamardy · 2024-07-05T14:19:33Z

@mariocynicys @dimxy please check if your comments are resolved and if you have any more comments / review iterations.

mariocynicys

please check if your comments are resolved and if you have any more comments / review iterations.

Nope, LGTM

dimxy

LGTM

* dev: feat(nft-swap): add standalone maker contract and proxy support (#2100) feat(ETH): add `gas_limit` coins param to override default values (#2137) feat(tendermint): implement better sequence resolving logic (#2164) ci(artifact): add target for macos on apple silicon (#2163) fix(helpers): extend http to ws address conversion (#2166) fix(makerbot): add "testcoin" to provider options (#2161) fix(hd_wallet): make extended pubkey of hd wallet generic (#2159) fix(docker-tests): implement containers runtime directories (#2162) feat(tendermint): improve the `max` handling for tendermint withdraw (#2155) revert #2158 (comment) (#2160) ci(artifacts): upload build artifacts with in-tree script (#2158) test(tendermint): migrate to local/offline containerized testnets (#2128) use easingthemes/ssh-deploy@v5.0.3 for all builds except windows (#2157) chore(bin): rename mm2 binaries to kdf (#2126)

* dev: (22 commits) chore(release): bump mm2 version to 2.2.0-beta (KomodoPlatform#2188) ci(docker-tests): ignore tendermint IBC tests for now (KomodoPlatform#2185) feat(nft-swap): complete refund methods (KomodoPlatform#2129) chore(release): add changelog entries for v2.1.0-beta (KomodoPlatform#2165) fix(zcoin): don't force low r signing to generate htlc pubkey for zcoin (KomodoPlatform#2184) chore(rust-analyzer): add rust-analyzer into the workspace toolchain (KomodoPlatform#2179) chore: migrate .cargo/config to .cargo/config.toml to avoid deprecation warning (KomodoPlatform#2177) fix(swaps): ensure taker payment spend confirmations (KomodoPlatform#2176) feat(nft-swap): add standalone maker contract and proxy support (KomodoPlatform#2100) feat(ETH): add `gas_limit` coins param to override default values (KomodoPlatform#2137) feat(tendermint): implement better sequence resolving logic (KomodoPlatform#2164) ci(artifact): add target for macos on apple silicon (KomodoPlatform#2163) fix(helpers): extend http to ws address conversion (KomodoPlatform#2166) fix(makerbot): add "testcoin" to provider options (KomodoPlatform#2161) fix(hd_wallet): make extended pubkey of hd wallet generic (KomodoPlatform#2159) fix(docker-tests): implement containers runtime directories (KomodoPlatform#2162) feat(tendermint): improve the `max` handling for tendermint withdraw (KomodoPlatform#2155) revert KomodoPlatform#2158 (comment) (KomodoPlatform#2160) ci(artifacts): upload build artifacts with in-tree script (KomodoPlatform#2158) test(tendermint): migrate to local/offline containerized testnets (KomodoPlatform#2128) ...

create persistent data for tendermint

3d987a9

Signed-off-by: onur-ozkan <work@onurozkan.dev>

onur-ozkan added Refactoring in progress Changes will be made from the author labels May 28, 2024

onur-ozkan added 3 commits May 29, 2024 15:30

migrate tendermint_tests to be docker tests

c7a8f64

Signed-off-by: onur-ozkan <work@onurozkan.dev>

disable certain modules temporarily for development purposes

9ee35e4

Signed-off-by: onur-ozkan <work@onurozkan.dev>

run nucleus docker image

335589f

Signed-off-by: onur-ozkan <work@onurozkan.dev>

onur-ozkan force-pushed the offline-tendermint-testnets branch from 6a8acb7 to 335589f Compare May 29, 2024 14:25

onur-ozkan added 11 commits May 29, 2024 18:16

update nucleus testnet data

16e4f53

Signed-off-by: onur-ozkan <work@onurozkan.dev>

add ticker to nucleusd container

40d2ba5

Signed-off-by: onur-ozkan <work@onurozkan.dev>

dbg

2ec84f4

Signed-off-by: onur-ozkan <work@onurozkan.dev>

wait for specific message from nucleusd container

ef79de9

Signed-off-by: onur-ozkan <work@onurozkan.dev>

fix container runner command

3980ce5

Signed-off-by: onur-ozkan <work@onurozkan.dev>

update nucleusd data

29fd0de

Signed-off-by: onur-ozkan <work@onurozkan.dev>

Merge branch 'dev' of github.com:KomodoPlatform/komodo-defi-framework…

a86b058

… into offline-tendermint-testnets

update atom test seeds

8a7bea9

Signed-off-by: onur-ozkan <work@onurozkan.dev>

add atom testnet state

867e737

Signed-off-by: onur-ozkan <work@onurozkan.dev>

WIP: enable atom local testnet

0903037

Signed-off-by: onur-ozkan <work@onurozkan.dev>

update nucleus testnet data

2b05d67

Signed-off-by: onur-ozkan <work@onurozkan.dev>

onur-ozkan mentioned this pull request Jun 5, 2024

feat(tx-history): handle encoded transaction values #2133

Merged

onur-ozkan added 7 commits June 6, 2024 07:23

Merge branch 'dev' of github.com:KomodoPlatform/komodo-defi-framework…

4bcf052

… into offline-tendermint-testnets

start atom node

282b450

Signed-off-by: onur-ozkan <work@onurozkan.dev>

fix various assertions

e332fd3

Signed-off-by: onur-ozkan <work@onurozkan.dev>

fix balance test

dcb5e81

Signed-off-by: onur-ozkan <work@onurozkan.dev>

update atom testnet conf

0096707

Signed-off-by: onur-ozkan <work@onurozkan.dev>

migrate cosmos tx history test

b3c603a

Signed-off-by: onur-ozkan <work@onurozkan.dev>

update nucleus node data

a19237b

Signed-off-by: onur-ozkan <work@onurozkan.dev>

onur-ozkan force-pushed the offline-tendermint-testnets branch from 56784a2 to a19237b Compare June 6, 2024 05:53

migrate test_tendermint_withdraw_hd

af75d4e

Signed-off-by: onur-ozkan <work@onurozkan.dev>

onur-ozkan force-pushed the offline-tendermint-testnets branch from ae6878a to af75d4e Compare June 6, 2024 14:51

update atom testnet data

5d1a23c

Signed-off-by: onur-ozkan <work@onurozkan.dev>

onur-ozkan force-pushed the offline-tendermint-testnets branch from 52462a1 to e6229f3 Compare June 10, 2024 17:32

fix clippy

4e02b4d

Signed-off-by: onur-ozkan <work@onurozkan.dev>

onur-ozkan force-pushed the offline-tendermint-testnets branch 4 times, most recently from 4fffa84 to 6696109 Compare June 12, 2024 06:35

add nucleus<->eth swap coverage

6696109

Signed-off-by: onur-ozkan <work@onurozkan.dev>

onur-ozkan added under review and removed in progress Changes will be made from the author labels Jun 12, 2024

improve container image usage

8bde022

Signed-off-by: onur-ozkan <work@onurozkan.dev>

mariocynicys reviewed Jun 18, 2024

View reviewed changes

use ticker variables instead of hardcoding them

23f2db5

Signed-off-by: onur-ozkan <work@onurozkan.dev>

dimxy reviewed Jul 2, 2024

View reviewed changes

mm2src/mm2_main/tests/docker_tests/tendermint_tests.rs Outdated Show resolved Hide resolved

shamardy previously approved these changes Jul 2, 2024

View reviewed changes

.docker/container-state/atom-testnet-data/data/application.db/LOG Outdated Show resolved Hide resolved

onur-ozkan dismissed shamardy’s stale review via ecaedab July 3, 2024 14:21

onur-ozkan force-pushed the offline-tendermint-testnets branch from ecaedab to 476c5aa Compare July 3, 2024 14:26

implement wait_until_relayer_container_is_ready

6fd189e

Signed-off-by: onur-ozkan <work@onurozkan.dev>

onur-ozkan force-pushed the offline-tendermint-testnets branch from 476c5aa to 6fd189e Compare July 3, 2024 14:35

onur-ozkan added 3 commits July 3, 2024 21:00

create new ibc channels

edcba4b

Signed-off-by: onur-ozkan <work@onurozkan.dev>

update wait_until_relayer_container_is_ready

c9474a7

Signed-off-by: onur-ozkan <work@onurozkan.dev>

use localhost instead of 0.0.0.0

ef0afd4

Signed-off-by: onur-ozkan <work@onurozkan.dev>

shamardy approved these changes Jul 5, 2024

View reviewed changes

mariocynicys approved these changes Jul 5, 2024

View reviewed changes

dimxy approved these changes Jul 5, 2024

View reviewed changes

shamardy merged commit 8e312a1 into dev Jul 5, 2024
18 of 25 checks passed

shamardy deleted the offline-tendermint-testnets branch July 5, 2024 15:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test(tendermint): migrate to local/offline containerized testnets #2128

test(tendermint): migrate to local/offline containerized testnets #2128

onur-ozkan commented May 28, 2024 •

edited

Loading

mariocynicys left a comment

mariocynicys Jun 18, 2024

onur-ozkan Jun 24, 2024

shamardy left a comment

onur-ozkan commented Jul 3, 2024

shamardy commented Jul 5, 2024

mariocynicys left a comment

dimxy left a comment

	pub(super) async fn seq_safe_send_raw_tx_bytes(
	&self,
	tx_payload: Any,
	fee: Fee,
	timeout_height: u64,
	memo: String,
	) -> Result<(String, Raw), TransactionErr> {
	let (tx_id, tx_raw) = loop {
	let tx_raw = try_tx_s!(self.any_to_signed_raw_tx(
	try_tx_s!(self.priv_key_policy.activated_key_or_err()),
	try_tx_s!(self.account_info(&self.account_id).await),
	tx_payload.clone(),
	fee.clone(),
	timeout_height,
	memo.clone(),
	));

	match self.send_raw_tx_bytes(&try_tx_s!(tx_raw.to_bytes())).compat().await {
	Ok(tx_id) => break (tx_id, tx_raw),
	Err(e) => {
	if e.contains(ACCOUNT_SEQUENCE_ERR) {
	debug!("Got wrong account sequence, trying again.");
	continue;
	}

	return Err(crate::TransactionErr::Plain(ERRL!("{}", e)));
	},
	};
	};

test(tendermint): migrate to local/offline containerized testnets #2128

test(tendermint): migrate to local/offline containerized testnets #2128

Conversation

onur-ozkan commented May 28, 2024 • edited Loading

mariocynicys left a comment

Choose a reason for hiding this comment

mariocynicys Jun 18, 2024

Choose a reason for hiding this comment

onur-ozkan Jun 24, 2024

Choose a reason for hiding this comment

shamardy left a comment

Choose a reason for hiding this comment

onur-ozkan commented Jul 3, 2024

shamardy commented Jul 5, 2024

mariocynicys left a comment

Choose a reason for hiding this comment

dimxy left a comment

Choose a reason for hiding this comment

onur-ozkan commented May 28, 2024 •

edited

Loading