[E2E] Allow testing with live-chain state #1949

SkymanOne · 2023-10-25T13:22:06Z

Summary

Closes #1947

[n] y/n | Does it introduce breaking changes?
[n] y/n | Is it dependent on the specific version of cargo-contract or pallet-contracts?

Allows to run E2E tests against the live chain leveraging its live state

How to test

Due to the inability to run the test on GH workflow, here are instructions on how to run the test locally:

Copied from use-ink/ink-docs#283

Clone chopsticks:

git clone https://github.com/AcalaNetwork/chopsticks

Modify the dev.yml config file in the repo or create one from scratch that you can reference later:

endpoint: ws://127.0.0.1:9944
mock-signature-host: true
block: 1
db: ./db.sqlite

Note: In the example above chopsticks will be mirroring up until block 1 from the substrate-contracts-node. For real world use case you would want to use a different block number and this is the place where you can configure other variables such as a sudo key. Read the chopsticks docs for more info.

Basically produce two empty blocks by submitting 2 system.remark(0) extrinsics on polkadot.js

Run chopsticks locally:

npx @acala-network/chopsticks@latest --config=configs/dev.yml

Go to integration-tests/e2e-call-runtime and modify the test attribute:

...
#[ink_e2e::test(node_url = "ws://127.0.0.1:8000")]
...

Run cargo test -F e2e-tests

Checklist before requesting a review

My code follows the style guidelines of this project
I have added an entry to CHANGELOG.md
I have commented my code, particularly in hard-to-understand areas
I have added tests that prove my fix is effective or that my feature works
Any dependent changes have been merged and published in downstream modules

cmichi · 2023-10-25T13:38:36Z

crates/e2e/src/node_proc.rs

@@ -150,9 +159,6 @@ where
            Err(err) => {
                let err = format!("Failed to connect to node rpc at {url}: {err}");
                tracing::error!("{}", err);
-                proc.kill().map_err(|e| {
-                    format!("Error killing substrate process '{}': {}", proc.id(), e)
-                })?;


Accidental removal?

paritytech-cicd-pr · 2023-10-25T15:12:37Z

🦑 📈 ink! Example Contracts ‒ Changes Report 📉 🦑

These are the results when building the integration-tests/* contracts from this branch with cargo-contract and comparing them to ink! master:

Contract	Upstream Size (kB)	PR Size (kB)	Change
basic-contract-caller	2.967	2.967	➖
basic-contract-caller/other-contract	1.337	1.337	➖
call-builder-return-value	8.735	8.735	➖
call-runtime	1.769	1.769	➖
conditional-compilation	1.209	1.209	➖
contract-storage	7.171	7.171	➖
contract-terminate	1.092	1.092	➖
contract-transfer	1.444	1.444	➖
custom-allocator	7.428	7.428	➖
dns	7.249	7.249	➖
e2e-call-runtime	1.058	1.058	➖
e2e-runtime-only-backend	1.635	1.635	➖
erc1155	13.962	13.962	➖
erc20	6.687	6.687	➖
erc721	9.64	9.64	➖
events	4.763	4.763	➖
flipper	1.393	1.393	➖
incrementer	1.221	1.221	➖
lang-err-integration-tests/call-builder-delegate	2.317	2.317	➖
lang-err-integration-tests/call-builder	4.847	4.847	➖
lang-err-integration-tests/constructors-return-value	1.773	1.773	➖
lang-err-integration-tests/contract-ref	4.328	4.328	➖
lang-err-integration-tests/integration-flipper	1.571	1.571	➖
mapping-integration-tests	7.685	7.685	➖
mother	9.508	9.508	➖
multi-contract-caller	5.924	5.924	➖
multi-contract-caller/accumulator	1.095	1.095	➖
multi-contract-caller/adder	1.669	1.669	➖
multi-contract-caller/subber	1.689	1.689	➖
multisig	21.471	21.471	➖
payment-channel	5.501	5.501	➖
sr25519-verification	0.865	0.865	➖
static-buffer	1.405	1.405	➖
trait-dyn-cross-contract-calls	2.466	2.466	➖
trait-dyn-cross-contract-calls/contracts/incrementer	1.305	1.305	➖
trait-erc20	7.063	7.063	➖
trait-flipper	1.209	1.209	➖
trait-incrementer	1.37	1.37	➖
upgradeable-contracts/delegator	2.908	2.908	➖
upgradeable-contracts/delegator/delegatee	1.369	1.369	➖
upgradeable-contracts/set-code-hash	1.464	1.464	➖
upgradeable-contracts/set-code-hash/updated-incrementer	1.443	1.443	➖
wildcard-selector	2.622	2.622	➖

Link to the run | Last update: Wed Nov 15 22:27:35 CET 2023

codecov-commenter · 2023-10-25T15:13:03Z

Codecov Report

Attention: 7 lines in your changes are missing coverage. Please review.

Comparison is base (64efc07) 53.30% compared to head (200e672) 53.24%.
Report is 1 commits behind head on master.

Files	Patch %	Lines
crates/e2e/macro/src/codegen.rs	0.00%	7 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1949      +/-   ##
==========================================
- Coverage   53.30%   53.24%   -0.07%     
==========================================
  Files         219      219              
  Lines        6838     6844       +6     
==========================================
- Hits         3645     3644       -1     
- Misses       3193     3200       +7

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

cmichi · 2023-10-27T19:29:43Z

CHANGELOG.md

+- [E2E] Allow testing with live-chain state - [#1949](https://github.com/paritytech/ink/pull/1949)
+- Stabilize `call_runtime` ‒ [#1749](https://github.com/paritytech/ink/pull/1749)
+- Schema generation - [#1765](https://github.com/paritytech/ink/pull/1765)
+- Add `set_block_number` to off-chain test api `Engine` - [#1806](https://github.com/paritytech/ink/pull/1806)


Was it intentional to add all those unrelated ones here?

No, just a mistake during merge

cmichi · 2023-10-27T19:33:02Z

On my branch I had added the feature to more subxt deps than you. Did your tests with Chopsticks work with your PR?

SkymanOne · 2023-10-30T11:12:27Z

On my branch I had added the feature to more subxt deps than you. Did your tests with Chopsticks work with your PR?

Yes. I have run tests with WS_PORT=8000 cargo test -F e2e-tests on e2e-call-runtime and flipper. Both succeeded, and logs were produced in chopsticks node.

cmichi · 2023-10-30T14:01:51Z

Yes. I have run tests with WS_PORT=8000 cargo test -F e2e-tests on e2e-call-runtime and flipper. Both succeeded, and logs were produced in chopsticks node.

Did you also run them with -F live-state-test?

SkymanOne · 2023-10-30T14:09:36Z

Yes. I have run tests with WS_PORT=8000 cargo test -F e2e-tests on e2e-call-runtime and flipper. Both succeeded, and logs were produced in chopsticks node.

Did you also run them with -F live-state-test?

Yes, as you can see in git diff in this PR, e2e-call-runtime has live-state-test enabled by default

ascjones

Can you give a brief explanation of the changes and the setup, and how it works to enable testing live-chain state?

crates/e2e/src/node_proc.rs

crates/e2e/macro/src/config.rs

crates/e2e/Cargo.toml

ascjones

LGTM

crates/e2e/macro/src/codegen.rs

crates/e2e/macro/src/config.rs

cmichi and others added 3 commits July 28, 2023 18:56

Support WS_PORT

d310ba4

Enable unstable-light-client for subxt

7e13283

implemented feature and port parsing

fea2435

SkymanOne requested review from cmichi and ascjones as code owners October 25, 2023 13:22

changelog

043b7e8

cmichi reviewed Oct 25, 2023

View reviewed changes

SkymanOne added 3 commits October 25, 2023 15:24

return lines

711a130

Merge branch 'master' into gn/e2e-live-state-test

91ca765

merge other changes

217dd30

cmichi reviewed Oct 27, 2023

View reviewed changes

cleanup changelog

bb5fc05

SkymanOne requested a review from cmichi October 30, 2023 11:15

ascjones reviewed Nov 13, 2023

View reviewed changes

crates/e2e/src/node_proc.rs Outdated Show resolved Hide resolved

SkymanOne added 5 commits November 14, 2023 23:32

support chopsticks url

4a48697

Merge branch 'master' into gn/e2e-live-state-test

4781851

remove field and fmt

b27da1b

fix codegen

9439107

fix codegen for chopsticks

6a4270b

SkymanOne requested a review from ascjones November 15, 2023 00:18

SkymanOne added 2 commits November 15, 2023 09:47

conditionally extract url

8c4838c

fix tests

236e126

ascjones reviewed Nov 15, 2023

View reviewed changes

crates/e2e/macro/src/config.rs Outdated Show resolved Hide resolved

crates/e2e/Cargo.toml Outdated Show resolved Hide resolved

remove feature

4701659

ascjones approved these changes Nov 15, 2023

View reviewed changes

crates/e2e/macro/src/codegen.rs Outdated Show resolved Hide resolved

crates/e2e/macro/src/codegen.rs Outdated Show resolved Hide resolved

crates/e2e/macro/src/config.rs Outdated Show resolved Hide resolved

apply review suggestions

200e672

SkymanOne merged commit fa6dd8e into master Nov 16, 2023
42 checks passed

SkymanOne deleted the gn/e2e-live-state-test branch November 16, 2023 01:50

SkymanOne mentioned this pull request Nov 30, 2023

5.0.0-rc Release #2011

Merged

SkymanOne mentioned this pull request Mar 4, 2024

Release 5.0.0 #2133

Merged

brunopgalvao mentioned this pull request May 22, 2024

Add the "pop up contracts-node" command r0gue-io/pop-cli#185

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[E2E] Allow testing with live-chain state #1949

[E2E] Allow testing with live-chain state #1949

SkymanOne commented Oct 25, 2023 •

edited

Loading

cmichi Oct 25, 2023

SkymanOne Oct 25, 2023

paritytech-cicd-pr commented Oct 25, 2023 •

edited by github-actions bot

Loading

codecov-commenter commented Oct 25, 2023 •

edited

Loading

cmichi Oct 27, 2023

SkymanOne Oct 30, 2023

cmichi commented Oct 27, 2023

SkymanOne commented Oct 30, 2023 •

edited

Loading

cmichi commented Oct 30, 2023

SkymanOne commented Oct 30, 2023

ascjones left a comment

ascjones left a comment

[E2E] Allow testing with live-chain state #1949

[E2E] Allow testing with live-chain state #1949

Conversation

SkymanOne commented Oct 25, 2023 • edited Loading

Summary

How to test

Checklist before requesting a review

cmichi Oct 25, 2023

Choose a reason for hiding this comment

SkymanOne Oct 25, 2023

Choose a reason for hiding this comment

paritytech-cicd-pr commented Oct 25, 2023 • edited by github-actions bot Loading

🦑 📈 ink! Example Contracts ‒ Changes Report 📉 🦑

codecov-commenter commented Oct 25, 2023 • edited Loading

Codecov Report

cmichi Oct 27, 2023

Choose a reason for hiding this comment

SkymanOne Oct 30, 2023

Choose a reason for hiding this comment

cmichi commented Oct 27, 2023

SkymanOne commented Oct 30, 2023 • edited Loading

cmichi commented Oct 30, 2023

SkymanOne commented Oct 30, 2023

ascjones left a comment

Choose a reason for hiding this comment

ascjones left a comment

Choose a reason for hiding this comment

SkymanOne commented Oct 25, 2023 •

edited

Loading

paritytech-cicd-pr commented Oct 25, 2023 •

edited by github-actions bot

Loading

codecov-commenter commented Oct 25, 2023 •

edited

Loading

SkymanOne commented Oct 30, 2023 •

edited

Loading