-
Notifications
You must be signed in to change notification settings - Fork 431
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add [ink::e2e-test]
with a new E2E testing crate à la ink_waterfall
#1234
Comments
I would add that there are potentially benefits to removing some abstractions required for the offchain env in the |
E2E testing framework is good. However there are many things to consider. For example:
|
ink_engine
and replace [ink::test]
with a new E2E testing crate à la ink_waterfall
[ink::e2e-test]
with a new E2E testing crate à la ink_waterfall
I would like to re-iterate customization. What if instead of natively running the Rust, as it does now, and instead of spawning a node, as proposed, an e2e-test setup and ran a full instance of wasmi? This may be best done via modularizing the contracts pallet to support such instances (if not already sufficiently modular). That could then be wrapped by a test, directly feeding TXs, or the pallet, feeding in all TXs from a block. |
This is not true. Substrate contract node startups in a second and uses instant seal. Why you would need to start a browser I don't understand. No wait time here.
I don't see any problem in doing that. In the end it just submits extrinsics.
This is true. However, as of right now this is barely usable anyways as the off chain environment is kind of bogus. The plan is to implement trace based debugging in wasmi. So wasmi emits a trace of the execution on dry-run which can then be downloaded and used to step through the execution.
This is just what we have now with extra steps. You would still need to emulate the rest of the chain. |
Phat Contract runs in an offchain daemon. So instead of spawning up a new substrate node, we will need to spawn a full stack. In addition, the query is served by the daemon instead of substrate rpc, and the contract call extrinsic also has a very different format (e.g. fully e2e encrypted). Recently there are a bunch of nice work in progress worth taking a look as well:
|
Okay but in the end you just have an endpoint where the tests submit their extrinsics against? So you just point the tests to this endpoint instead of letting the tests spawn it.
So even less incentive for us to maintain our own off-chain testing implementation? |
Well, things will get complicated in our specific case (though we understand we are a very special case, and it's impossible to support every cases). Let me give you the example. On our side, to submit a contract call, the contract client (like ContractPromise in polkadot.js or subxt) need to negotiate with the daemon to make a key agreement. Then it encrypts the call data, wraps the call by an extrinsic targeting to our pallet (not pallet-contracts), and submit to our Substrate node. To make a query, it does the similar thing but it sends the encrypted RPC to the daemon. Ideally if there's a common interface like below, and we just provide an alternative implementation, it should work: trait InkClient {
type QueryOption;
type TxOption;
fn query(&self, address: AccountId, data: Data, options: QueryOption);
fn tx(&self, address: AccountId, data: Data, options: TxOption);
} This is actually what we wanted to push the Polkadot.js upstream to do and we didn't make progress. But in the reality, it also brings the maintenance burden to your side.
E2E is of course very useful, but I just want to point out the directions the ecosystem projects have chosen are different. Personally if the unit test framework works well (like with Supercolony's ink_env fork mentioned above), I tend to invest heavily in unit test (faster and interperterable) and write less E2E test. I will still write an E2E test, but maybe in Typescript for flexibility. |
We now have E2E testing available as part of the ink! 4.0 release |
Persisting the results from our discussion on Saturday:
engine
) and replace it with an E2E testing framework.ink_waterfall
. So we want to make the waterfall a product as part of ink!.[ink::test]
syntax, but it will then instead spawn asubstrate-contracts-node
(if one is not already spawned) and execute real transactions viacargo-contract instantiate/call/…
.The reason behind this is that we found that we need a proper E2E testing framework to keep up with competing products. In the past users found it very irritating that they can't test e.g. the call builder in our off-chain testing environment. As we move more towards interoperability (XCM & Co.) the need for E2E testing will need to be prioritized. By making the waterfall part of ink! in this way we can in the future also use this setup to test contracts that interact with parachains.
It will also remove the need of duplicating the API and logic of
pallet-contracts
in ink!.The text was updated successfully, but these errors were encountered: