MEDIUM_1 v1.1: Upgrade the Joystream runtime to the latest Substrate version.

[shamil-gadelshin]

We need to upgrade our runtime to the latest Substrate version.

Comments

• It makes sense to use "polkadot-*" branches to apply updates similar to Polkadot and other major Substrate-based networks: https://github.com/AcalaNetwork/Acala/blob/399a1f61e2d0bf4d601fd624446d90487eaf6bd2/modules/asset-registry/Cargo.toml#L12
• the current version is polkadot-v0.9.17
• some of the changes were implemented during the upgrade up to the Substrate 3.0 version for the olympia_audit2 branch.
• we don't plan to convert our pallets to the new macro-syntax: we need to rename Trait to Config only in every pallet and change benchmarking syntax
• all tests from the Substrate node templates should be copied. They help to verify upgrade correctness.

Current plan

1. Upgrade pallets

• change dependencies' version
• rename Trait to Config
• change references to the Parity's pallets (eg.: frame_system::Trait to frame_system::Config)
• change benchmarking syntax

1. Change runtime

• change dependencies' version
• change Parity's pallets (like frame_system) - some of the changes are implemented in olympia_audit2 branch

1. Change node
2. Change chain-spec-builder
3. (Optional). Upgrade pallets to the new syntax version: upgrade old macro syntax to the new one

Rough work time estimations for a single person

1. Upgrade pallets (14 crates): (1-4 hours) x 14 crates = 2-7 days
2. Change runtime: 2-7 days
3. Change node: 2-7 days
4. Change chain-spec-builder: 1-2 days
   Subtotal(without optional syntax upgrade for pallets): 7-23 days
5. (Optional). Upgrade pallets to the new syntax version: (4-8) hours * 14 crates = 7-14 days
   Total: 14-37 days

The *->2.0.0 upgrade took 2 weeks with reviews. The 2.0.1->3.0.0 upgrade took 1 month with reviews.

┆Issue is synchronized with this Asana task by Unito

[bedeho]

It is my understanding that some recent Substrate version frees us from the burden of having a types library?

• Am I correct, if so, would we be targeting a version where this is unlocked?
• If so, what are implications of trying to take advantage of this in other apps?

[mnaamani]

It is my understanding that some recent Substrate version frees us from the burden of having a types library?

* Am I correct, if so, would we be targeting a version where this is unlocked?

* If so, what are implications of trying to take advantage of this in other apps?

The latest version of polkadot depends on substrate https://github.com/paritytech/substrate?branch=polkadot-v0.9.23#6cbe1772bf258793fa9845daa8f43ea0cadee596 Seems like a good target to use ourselves. I'll check if it supports this capability, but I suspect its the newer version of polkadot-js that unlocks this possibility.

I was considering building on olympia_audit2 branch and updating it with latest olympia, ephesus, and audit3 work, but I think the work already done by the team would be duplicated. Instead I will create carthage based off audit3 and update that to substrate v3. Once #3814 is completed we can update carthage. The work already done in olympia_audit2 will serve as reference how to update to v3 and hopefully save time.

[mnaamani]

This might be a faster approach that build on all the work already done by shamil

• Merge current master into olympia_audit2, resolve conflicts, apply any v3 fixes
• Update to newer substrate v3 release
• Merge olympia_audit2 into new ephesus-v3 (based off ephesus) and apply any fixes
• Audit3: Merge Ephesus #3814 would be updated from ephesus-v3 with fixes
• Merge 3814 into auditv3 and apply v3 fixes

[mnaamani]

Started to make progress on first two steps. Working with olympia_audit2 branch tried updating to latest release. I'm finding I'm having to derive the TypeInfo trait on many of the runtime types https://github.com/paritytech/scale-info Maybe this has something todo with supporting clients (like polkadot-js) to be able to derive more detailed type information... will investigate further.

[mnaamani]

I have branched off current master and created rhodes-substrate-v3 and opened #3875.

I will try to keep track of any changes to our runtime types or api in a separate issue for reference #3876

[mnaamani]

Just found this handy guide: Upgrading-2.0-to-3.0.md

[bedeho]

Done.