updating v0.3 to use substrate v0.10 #146

gterzian · 2019-02-20T08:18:57Z

gterzian · 2019-02-20T11:56:56Z

 --> service/src/lib.rs:150:1
    |
150 | / construct_service_factory! {
151 | |     struct Factory {
152 | |         Block = Block,
153 | |         RuntimeApi = RuntimeApi,
...   |
275 | |     }
276 | | }
    | |_^ the trait `sr_primitives::traits::ProvideRuntimeApi` is not implemented for `grandpa::import::GrandpaBlockImport<substrate_client_db::Backend<sr_primitives::generic::Block<sr_primitives::generic::Header<u64, sr_primitives::traits::BlakeTwo256, sr_primitives::generic::DigestItem<primitives::H256, primitives::Ed25519AuthorityId>>, polkadot_primitives::UncheckedExtrinsic>>, client::LocalCallExecutor<substrate_client_db::Backend<sr_primitives::generic::Block<sr_primitives::generic::Header<u64, sr_primitives::traits::BlakeTwo256, sr_primitives::generic::DigestItem<primitives::H256, primitives::Ed25519AuthorityId>>, polkadot_primitives::UncheckedExtrinsic>>, substrate_executor::native_executor::NativeExecutor<polkadot_executor::Executor>>, sr_primitives::generic::Block<sr_primitives::generic::Header<u64, sr_primitives::traits::BlakeTwo256, sr_primitives::generic::DigestItem<primitives::H256, primitives::Ed25519AuthorityId>>, polkadot_primitives::UncheckedExtrinsic>, polkadot_runtime::RuntimeApi, client::Client<substrate_client_db::Backend<sr_primitives::generic::Block<sr_primitives::generic::Header<u64, sr_primitives::traits::BlakeTwo256, sr_primitives::generic::DigestItem<primitives::H256, primitives::Ed25519AuthorityId>>, polkadot_primitives::UncheckedExtrinsic>>, client::LocalCallExecutor<substrate_client_db::Backend<sr_primitives::generic::Block<sr_primitives::generic::Header<u64, sr_primitives::traits::BlakeTwo256, sr_primitives::generic::DigestItem<primitives::H256, primitives::Ed25519AuthorityId>>, polkadot_primitives::UncheckedExtrinsic>>, substrate_executor::native_executor::NativeExecutor<polkadot_executor::Executor>>, sr_primitives::generic::Block<sr_primitives::generic::Header<u64, sr_primitives::traits::BlakeTwo256, sr_primitives::generic::DigestItem<primitives::H256, primitives::Ed25519AuthorityId>>, polkadot_primitives::UncheckedExtrinsic>, polkadot_runtime::RuntimeApi>>`
    |
    = note: required because of the requirements on the impl of `substrate_consensus_common::import_queue::Verifier

Anyone can make-out what this means? @gnunicorn ?

rphmeier · 2019-02-20T16:25:12Z

We used to forward these traits on GrandpaBlockImport but it seems we don't anymore. Maybe we need to do that again upstream -- @sorpaas removed them because they weren't being used.

I think this has to do with the construction of AuraImportQueue -- the impl Verifier for AuraVerifier requires the generic param C there to be C: ProvideRuntimeApi, C::Api: BlockBuilderApi<B>. We really need the AuraImportQueue to import stuff through the GRANDPA worker, but since this is only for the Verifier, we should just be able to pass the client.

gterzian · 2019-02-21T07:19:48Z

@bkchr Would you have any pointers with regards to updating polkadot in the light of paritytech/substrate@53bf81e#diff-2d4fe59416e64524ebe5d49fb1cd82c2

For example, how to implement ProvideInherent for polkadot?

Also, Polkadot had an struct looking like:

InherentData {
			timestamp: self.believed_minimum_timestamp,
			parachains: candidates,
			aura_expected_slot: self.consensus_data.slot,
		};

How could I re-implement this using inherents::InherentData?

Other areas that needs to be updated are propose and propose_with

bkchr · 2019-02-21T09:34:21Z

Polkadot is already using the new inherent data interface in master.

rphmeier · 2019-02-22T15:44:46Z

consensus/src/lib.rs

 {
 	type Error = Error;
 	type Create = Either<
 		CreateProposal<C, TxApi>,
 		future::FutureResult<Block, Error>,
 	>;

-	fn propose(&self, consensus_data: AuraConsensusData) -> Self::Create {
+	fn propose(&self, inherent_data: InherentData, max_duration: Duration) -> Self::Create {


@bkchr can you review this function to make sure the inherent data is backwards compatible with before?

Do we need to produce blocks with the new node and the old runtime? If yes, then this is not compatible. For syncing, this code should not be touched.

Ok, thanks for the clarification. I think we can work around this. What would happen if the old code tried to author on the new runtime?

Create inherent will fail and thus the block production will fail.

rphmeier · 2019-02-22T15:46:43Z

runtime/Cargo.toml

@@ -49,6 +50,7 @@ std = [
 	"polkadot-primitives/std",
 	"parity-codec/std",
 	"parity-codec-derive/std",
+    "substrate-inherents/std",


funky indentation

rphmeier · 2019-02-24T01:07:26Z

network/src/consensus.rs

+		let executor = task_executor.clone();
+
+		// spin up a task in the background that processes all incoming statements
+		// TODO: propagate statements on a timer?


this seems in duplicate with the above comment.

(Also, "no commit of TODO/FIXME without a Link" - I'd vote for removing that ToDo-comment at all)

rphmeier · 2019-02-24T01:08:01Z

network/src/consensus.rs

+		// spin up a task in the background that processes all incoming statements
+		// TODO: propagate statements on a timer?
+		self.network
+			.with_spec(move |spec, ctx| {


this whole block could be moved up a level of indentation if we didn't do a newline on .with_spec

rphmeier · 2019-02-24T01:09:12Z

network/src/consensus.rs

+			Ok(futures::Async::Ready(mut inner)) => {
+				let poll_result = inner.poll();
+				self.inner = Some(inner);
+				poll_result.map_err(|_| NetworkDown)


seems more resilient to return self.poll() here.

rphmeier · 2019-02-24T01:09:25Z

network/src/consensus.rs

@@ -180,14 +191,30 @@ impl<P, E> Network for ConsensusNetwork<P,E> where
 pub struct NetworkDown;

 /// A future that resolves when a collation is received.
-pub struct AwaitingCollation(::futures::sync::oneshot::Receiver<Collation>);
+pub struct AwaitingCollation {
+	outer: ::futures::sync::oneshot::Receiver<::futures::sync::oneshot::Receiver<Collation>>,


I would recommend bringing Receiver into scope.

rphmeier · 2019-02-24T01:09:44Z

network/src/consensus.rs

-		AwaitingCollation(
-			self.network.with_spec(|spec, _| spec.await_collation(relay_parent, parachain))
-		)
+		let (tx, rx) = ::futures::sync::oneshot::channel();


perhaps oneshot being in scope is reasonable.

rphmeier · 2019-02-24T01:10:09Z

network/src/consensus.rs

+			let collation = spec.await_collation(relay_parent, parachain);
+			let _ = tx.send(collation);
+		});
+		AwaitingCollation{outer: rx, inner: None}


style: AwaitingCollation { ... } (note the spaces)

gnunicorn

A few minor fixes to address.

gnunicorn · 2019-02-22T17:09:47Z

consensus/src/lib.rs

-			parachains: candidates,
-			aura_expected_slot: self.consensus_data.slot,
-		};
+		let mut inherent_data = self.inherent_data.take().expect("CreateProposal is not polled after finishing; qed");


Let's please wrap these...

gnunicorn · 2019-02-22T17:11:51Z

network/src/consensus.rs

 				spec.new_consensus(ctx, parent_hash, CurrentConsensus {
 					knowledge,
 					local_session_key,
 				});
-
-				MessageProcessTask {
+				let inner_stream = match rx.try_recv() {


why so complicated rather than a rx.try_recv().expect(...?

gnunicorn · 2019-02-22T17:22:09Z

primitives/Cargo.toml

-sr-version = { git = "https://github.com/paritytech/substrate", branch = "alexander-backports", default-features = false }
-sr-std = { git = "https://github.com/paritytech/substrate", branch = "alexander-backports", default-features = false }
-sr-primitives = { git = "https://github.com/paritytech/substrate", branch = "alexander-backports", default-features = false }
+parity-codec = "3.0"


3.0 still has std on per default but I don't think we want std in here...

Suggested change

parity-codec = "3.0"

parity-codec = { version = "3.0", default-features = false }

gnunicorn · 2019-02-24T08:19:48Z

runtime/src/parachains.rs

@@ -58,7 +58,7 @@ decl_storage! {
 	add_extra_genesis {
 		config(parachains): Vec<(ParaId, Vec<u8>, Vec<u8>)>;
 		config(_phdata): PhantomData<T>;
-		build(|storage: &mut sr_primitives::StorageMap, _: &mut ChildrenStorageMap, config: &GenesisConfig<T>| {
+		build(|storage: &mut sr_primitives::StorageOverlay, _: &mut ChildrenStorageOverlay, config: &GenesisConfig<T>| {


let's wrap.

gnunicorn · 2019-02-25T08:08:18Z

network/src/consensus.rs

+		let executor = task_executor.clone();
+
+		// spin up a task in the background that processes all incoming statements
+		// TODO: propagate statements on a timer?


(Also, "no commit of TODO/FIXME without a Link" - I'd vote for removing that ToDo-comment at all)

primitives/Cargo.toml

runtime/wasm/Cargo.toml

test-parachains/adder/collator/src/main.rs

Co-Authored-By: gterzian <2792687+gterzian@users.noreply.github.com>

gterzian · 2019-02-25T10:29:50Z

@bkchr How can we address https://github.com/paritytech/polkadot/pull/146/files#r259722390 ?

* Rebuild runtime * Remove invalid value from chainspec (#68) * service: use grandpa block import for locally sealed aura blocks (#85) * bump version to v0.3.1 * Update lock file. * limit number of transactions when building blocks (#91) * Update to latest Substrate * Bump to 0.3.2 * Actually bump. * v0.3.2 (#98) * bump substrate version * fix polkadot-collator * point to alexander-backports of substrate * bump version * cli: fix node shutdown (#100) * update to latest substrate, change to v0.3.4 * update to latest substrate, bump version to 0.3.5 * v0.3.6 * try to build on every v0.3 commit and update alexander-backports * bump to v0.3.7 * bump to 0.3.8 * Bump to 0.3.9: network and pruning improvements * Bump to 0.3.10: reduce network bandwidth usage * Use libp2p-kad 0.3.2 (#122) * Bump libp2p-identify to 0.3.1 (#123) * Bump to 0.3.12 (#127) * Update Substrate again (#128) * update substrate and bump version to v0.3.13 * bump version to v0.3.14: fix --reserved-nodes * add a manually curated grandpa module (#136) * updating v0.3 to use substrate v0.10 (#146) * updating to latest substrate v0.10 * better handling of outer poll * nit * fix tests * remove comment * reduce indentation * use self.poll * bring oneshot into scope * spaces * wrap * remove match * wrap * Update primitives/Cargo.toml Co-Authored-By: gterzian <2792687+gterzian@users.noreply.github.com> * Update runtime/wasm/Cargo.toml Co-Authored-By: gterzian <2792687+gterzian@users.noreply.github.com> * Update runtime/wasm/Cargo.toml Co-Authored-By: gterzian <2792687+gterzian@users.noreply.github.com> * Update test-parachains/adder/collator/src/main.rs Co-Authored-By: gterzian <2792687+gterzian@users.noreply.github.com> * indent * add paranthese * config: fix wrong ip for alexander bootnode (#161) * fix curated-grandpa and rebuild wasm (#162) * [v0.3] Integrates new gossip system into Polkadot (#166) * new gossip validation in network * integrate new gossip into service * network: guard validation network future under exit signal (#168) * bump version to v0.3.15: substrate v0.10 * [v0.3] update to substrate master (#175) * update to substrate master * fix test * service: fix telemetry endpoints on alexander chainspec (#169) (#178) * Update v0.3 to latest Substrate master (#177) * update substrate v0.3 to latest master * bump spec version * update to latest master: remove fees module * update runtime blobs * bump version to 0.3.16 * replace sr25519 accountid with anysigner * bump version to v0.3.17 * Some PoC-3 GRANDPA tweaks (#181) * call on_finalise after triggering curated_grandpa change * make grandpa rounds shorter for faster finalization * use authorities when calculating duty roster (#185) * [v0.3] Update to substrate master (#183) * update to latest substrate master * bump version to 0.3.18 * update to latest substrate master * bump spec version * update runtime wasm blobs * remove current_offline_slash from chain spec * update to substrate master: bump version to v0.3.19 (#188) * update to substrate master: bump version to v0.3.19 libp2p network improvements * network: replace NodeIndex with PeerId * network: fix tests * polkadot v0.3.20 (#190) * update to substrate master: bump version to 0.3.20 * runtime: add offchain worker trait * runtime: rebuild wasm blobs * bump spec version (#191) * Fix compilation * Update version to 0.4.0 * Switch to use `polkadot-master` branch from substrate * Remove unused struct * Remove `grandpa::SyncedAuthorities` from `OnSessionChange`

Bumps [derive_more](https://github.com/JelteF/derive_more) from 0.99.7 to 0.99.8. - [Release notes](https://github.com/JelteF/derive_more/releases) - [Changelog](https://github.com/JelteF/derive_more/blob/master/CHANGELOG.md) - [Commits](JelteF/derive_more@v0.99.7...v0.99.8) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>

gterzian added the A3-in_progress Pull request is in progress. No review needed at this stage. label Feb 20, 2019

gterzian force-pushed the v0.3 branch from 251bab6 to 762ac59 Compare February 20, 2019 08:28

rphmeier changed the title ~~updating to substrate v0.10~~ updating v0.3 to use substrate v0.10 Feb 20, 2019

updating to latest substrate v0.10

3798772

gterzian force-pushed the v0.3 branch from 3b0bdee to 3798772 Compare February 21, 2019 13:09

gterzian added A0-please_review Pull request needs code review. and removed A3-in_progress Pull request is in progress. No review needed at this stage. labels Feb 21, 2019

gterzian requested a review from gnunicorn February 21, 2019 13:10

gterzian added 3 commits February 21, 2019 21:19

better handling of outer poll

ca15ae0

nit

77084e0

fix tests

b602d34

gnunicorn requested review from rphmeier and gavofyork February 21, 2019 14:23

rphmeier reviewed Feb 22, 2019

View reviewed changes

rphmeier approved these changes Feb 22, 2019

View reviewed changes

rphmeier reviewed Feb 24, 2019

View reviewed changes

gnunicorn approved these changes Feb 25, 2019

View reviewed changes

gterzian added 3 commits February 25, 2019 18:13

remove comment

f2a505e

reduce indentation

58d0859

use self.poll

e213c80

gterzian and others added 11 commits February 25, 2019 18:17

bring oneshot into scope

aebe865

spaces

db0f7d4

wrap

34b2fc7

remove match

5999e70

wrap

41f733a

Update primitives/Cargo.toml

ed7e75d

Co-Authored-By: gterzian <2792687+gterzian@users.noreply.github.com>

Update runtime/wasm/Cargo.toml

5453bf4

Co-Authored-By: gterzian <2792687+gterzian@users.noreply.github.com>

Update runtime/wasm/Cargo.toml

ba41dad

Co-Authored-By: gterzian <2792687+gterzian@users.noreply.github.com>

Update test-parachains/adder/collator/src/main.rs

5339ef8

Co-Authored-By: gterzian <2792687+gterzian@users.noreply.github.com>

indent

0d9508b

Merge branch 'v0.3' of github.com:gterzian/polkadot into v0.3

6277543

add paranthese

f96e275

arkpar approved these changes Feb 25, 2019

View reviewed changes

rphmeier approved these changes Feb 25, 2019

View reviewed changes

rphmeier added A8-looksgood and removed A0-please_review Pull request needs code review. labels Feb 25, 2019

rphmeier merged commit 461ea31 into paritytech:v0.3 Feb 25, 2019

gterzian deleted the v0.3 branch February 26, 2019 02:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

updating v0.3 to use substrate v0.10 #146

updating v0.3 to use substrate v0.10 #146

gterzian commented Feb 20, 2019

gterzian commented Feb 20, 2019 •

edited

Loading

rphmeier commented Feb 20, 2019 •

edited

Loading

gterzian commented Feb 21, 2019 •

edited

Loading

bkchr commented Feb 21, 2019

rphmeier Feb 22, 2019

bkchr Feb 25, 2019

rphmeier Feb 25, 2019

bkchr Feb 25, 2019

rphmeier Feb 22, 2019

rphmeier Feb 24, 2019

gnunicorn Feb 25, 2019

rphmeier Feb 24, 2019

rphmeier Feb 24, 2019

rphmeier Feb 24, 2019

rphmeier Feb 24, 2019

rphmeier Feb 24, 2019

gnunicorn left a comment

gnunicorn Feb 22, 2019

gnunicorn Feb 22, 2019

gnunicorn Feb 22, 2019

gnunicorn Feb 24, 2019

gnunicorn Feb 25, 2019

gterzian commented Feb 25, 2019

	parity-codec = "3.0"
	parity-codec = { version = "3.0", default-features = false }

updating v0.3 to use substrate v0.10 #146

updating v0.3 to use substrate v0.10 #146

Conversation

gterzian commented Feb 20, 2019

gterzian commented Feb 20, 2019 • edited Loading

rphmeier commented Feb 20, 2019 • edited Loading

gterzian commented Feb 21, 2019 • edited Loading

bkchr commented Feb 21, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gnunicorn left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gterzian commented Feb 25, 2019

gterzian commented Feb 20, 2019 •

edited

Loading

rphmeier commented Feb 20, 2019 •

edited

Loading

gterzian commented Feb 21, 2019 •

edited

Loading