From 33cbdf7c6a9e72a3975697d98d2883ba75ca2cc3 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Mon, 23 Jan 2023 11:30:13 +0100 Subject: [PATCH 01/76] WIP Signed-off-by: Cyrill Leutwiler --- .../datastructures/overview.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md index 42fda8ace9..37a7341dc7 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md @@ -4,15 +4,16 @@ slug: /datastructures/overview --- The `ink_storage` crate acts as the standard storage library for ink! smart contracts. At -the moment it only provides a single low-level primitive for interacting with storage, -the [`Mapping`](https://docs.rs/ink_storage/4.0.0-beta/ink_storage/struct.Mapping.html). +the moment it provides two primitives for interacting with storage, +[`Mapping`](https://docs.rs/ink_storage/4.0.0-beta/ink_storage/struct.Mapping.html) +and [`Lazy`](https://docs.rs/ink_storage/4.0.0-beta/ink_storage/struct.Lazy.html). -The `Mapping` is a mapping of key-value pairs directly to the contract storage. Its main advantage -is to be simple and lightweight. As such, it does not provide any high-level -functionality, such as iteration or automatic clean-up. Smart contract authors will need -to implement any high level functionality themselves. +The `Mapping` is a mapping of key-value pairs directly to the contract storage. It is very +similar to a traditional hash table and its main advantage is being simple and lightweight. +Hence, it does not provide any high-level functionality, such as iteration or automatic +clean-up. Smart contract authors will need to implement any high level functionality themselves. -## Eager Loading +## Eager Loading vs. Lazy Loading When executing a contract, all the fields of the `#[ink(storage)]` struct will be pulled from storage, regardless of whether or not they are used during the message execution. From bac0ba06ce8b4634faf71f610710f71cb8fa351d Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Mon, 23 Jan 2023 15:56:19 +0100 Subject: [PATCH 02/76] eager loading best explained in storage layout section Signed-off-by: Cyrill Leutwiler --- .../datastructures/overview.md | 55 +++++-------------- .../datastructures/spread-storage-layout.md | 35 ++++++++++++ 2 files changed, 49 insertions(+), 41 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md index 37a7341dc7..909d9907d6 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md @@ -3,47 +3,20 @@ title: Overview slug: /datastructures/overview --- -The `ink_storage` crate acts as the standard storage library for ink! smart contracts. At -the moment it provides two primitives for interacting with storage, +The `ink_storage` crate acts as the standard storage library for ink! smart contracts. +At the moment it provides two primitives for interacting with storage, [`Mapping`](https://docs.rs/ink_storage/4.0.0-beta/ink_storage/struct.Mapping.html) and [`Lazy`](https://docs.rs/ink_storage/4.0.0-beta/ink_storage/struct.Lazy.html). -The `Mapping` is a mapping of key-value pairs directly to the contract storage. It is very -similar to a traditional hash table and its main advantage is being simple and lightweight. -Hence, it does not provide any high-level functionality, such as iteration or automatic -clean-up. Smart contract authors will need to implement any high level functionality themselves. - -## Eager Loading vs. Lazy Loading - -When executing a contract, all the fields of the `#[ink(storage)]` struct will be pulled -from storage, regardless of whether or not they are used during the message execution. - -Smart contract authors should be aware of this behaviour since it could potentially -affect their contract performance. For example, consider the following storage struct: - -```rust -#[ink(storage)] -pub struct EagerLoading { - a: i32, - b: ink_prelude::vec::Vec, -} - -impl EagerLoading { - #[ink(message)] - pub fn read_a(&self) { - let a = self.a; - } -} -``` - -In `EagerLoading::read_a()` we only read the `a` storage item. However, the `b` storage -item will still be loaded from storage. As a reminder, this means accessing the -underlying database and SCALE decoding the value. This can incur high costs, especially -as the number of elements in `b` grows. - -:::note - -Eager loading does **not** apply to `Mapping` fields, though, as key lookups in mappings -are done directly from contract storage. - -::: +`Mapping` is a mapping of key-value pairs directly to the contract storage. It is very +similar to traditional hash tables and comparable to the `mapping` type Solidity offers. +As a core ingredient to the ink! language, its main advantage is being simple and +lightweight. However, it does not provide any high-level functionality, such as iteration +or automatic clean-up. Smart contract authors will need to implement any high level +functionality themselves. + +`Lazy` allows smart contract developers fine grained manual control over the storage +layout of individual types. Conceivably, it may be desirable to change certain aspects +on how your contract deals with it's storage variables. You can find out more about this +in the section about the ink! +[Storage Layout](https://use.ink/datastructures/spread-storage-layout). diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/spread-storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/spread-storage-layout.md index 0bdadb7d3f..1895ed2d29 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/spread-storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/spread-storage-layout.md @@ -49,3 +49,38 @@ persisted to storage in a spread layout.
Storage Organization: Spreading
+ +## Eager Loading vs. Lazy Loading + +When executing a contract, all the fields of the `#[ink(storage)]` struct will be pulled +from storage, regardless of whether or not they are used during the message execution. + +Smart contract authors should be aware of this behaviour since it could potentially +affect their contract performance. For example, consider the following storage struct: + +```rust +#[ink(storage)] +pub struct EagerLoading { + a: i32, + b: ink_prelude::vec::Vec, +} + +impl EagerLoading { + #[ink(message)] + pub fn read_a(&self) { + let a = self.a; + } +} +``` + +In `EagerLoading::read_a()` we only read the `a` storage item. However, the `b` storage +item will still be loaded from storage. As a reminder, this means accessing the +underlying database and SCALE decoding the value. This can incur high costs, especially +as the number of elements in `b` grows. + +:::note + +Eager loading does **not** apply to `Mapping` fields, though, as key lookups in mappings +are done directly from contract storage. + +::: From fc0a0b9eb8a032c41647edac4bbd84847dd47c46 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Mon, 23 Jan 2023 15:59:41 +0100 Subject: [PATCH 03/76] rename to storage-layout Signed-off-by: Cyrill Leutwiler --- .../version-4.0.0-alpha.1/datastructures/overview.md | 2 +- .../datastructures/spread-storage-layout.md | 2 +- versioned_sidebars/version-4.0.0-alpha.1-sidebars.json | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md index 909d9907d6..75bfed6fb8 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md @@ -19,4 +19,4 @@ functionality themselves. layout of individual types. Conceivably, it may be desirable to change certain aspects on how your contract deals with it's storage variables. You can find out more about this in the section about the ink! -[Storage Layout](https://use.ink/datastructures/spread-storage-layout). +[Storage Layout](https://use.ink/datastructures/storage-layout). diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/spread-storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/spread-storage-layout.md index 1895ed2d29..571f10b5cc 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/spread-storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/spread-storage-layout.md @@ -1,6 +1,6 @@ --- title: Spread Storage Layout -slug: /datastructures/spread-storage-layout +slug: /datastructures/storage-layout --- :::caution diff --git a/versioned_sidebars/version-4.0.0-alpha.1-sidebars.json b/versioned_sidebars/version-4.0.0-alpha.1-sidebars.json index 91b1f64902..107812b5bb 100644 --- a/versioned_sidebars/version-4.0.0-alpha.1-sidebars.json +++ b/versioned_sidebars/version-4.0.0-alpha.1-sidebars.json @@ -64,7 +64,7 @@ "Storage & Data Structures": [ "datastructures/overview", "datastructures/mapping", - "datastructures/spread-storage-layout", + "datastructures/storage-layout", "datastructures/custom", "datastructures/storage-in-metadata" ], @@ -86,4 +86,4 @@ "brand-assets/cargo-contract" ] } -} +} \ No newline at end of file From e83dcc3d1824b854ccfb6dfe5d3302804d117c54 Mon Sep 17 00:00:00 2001 From: xermicus Date: Mon, 23 Jan 2023 17:12:59 +0100 Subject: [PATCH 04/76] WIP update mappings Signed-off-by: xermicus --- .../datastructures/mapping.md | 94 ++++++++++--------- 1 file changed, 52 insertions(+), 42 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md index 4b759dd94d..e6d893372c 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md @@ -3,76 +3,86 @@ title: Working with Mapping slug: /datastructures/mapping --- -:::caution -TODO +In this section we demonstrate how to work with ink! [`Mapping`](https://docs.rs/ink_storage/4.0.0-beta/ink_storage/struct.Mapping.html). -Beware, this page is no longer up to date for 4.0! -::: - -In this section we want to demonstrate how to work with ink! [`Mapping`](https://docs.rs/ink_storage/4.0.0-beta/ink_storage/struct.Mapping.html). - -Here is an example of a mapping from a user to a number: +Here is an example of a mapping from a user to a `Balance`: ```rust #[ink(storage)] -#[derive(SpreadAllocate)] pub struct MyContract { - // Store a mapping from AccountIds to a u32 - map: ink_storage::Mapping, + /// Assign a balance to every account. + balances: ink::storage::Mapping, } ``` This means that for a given key, you can store a unique instance of a value type. In this -case, each "user" gets their own number. +case, each "user" gets credited their own balance. -## Initializing a Mapping +## A simple working example -In order to correctly initialize a `Mapping` we need two things: -1. An implementation of the [`SpreadAllocate`](https://docs.rs/ink_storage/4.0.0-beta/ink_storage/traits/trait.SpreadAllocate.html) trait on our storage struct -2. The [`ink_lang::utils::initalize_contract`](https://docs.rs/ink_lang/4.0.0-beta/ink_lang/utils/fn.initialize_contract.html) initializer - -Not initializing storage before you use it is a common mistake that can break your smart -contract. If you do not initialize your `Mapping`'s correctly you may end up with -different `Mapping`'s operating on the same set of storage entries 😱. +The following example constitues a contract where anyone can deposit and withdraw balances. +This functionality is realized using the `Mapping`. ```rust #![cfg_attr(not(feature = "std"), no_std)] #[ink::contract] mod mycontract { - use ink_storage::traits::SpreadAllocate; + use ink::storage::Mapping; #[ink(storage)] - #[derive(SpreadAllocate)] pub struct MyContract { - // Store a mapping from AccountIds to a u32 - map: ink_storage::Mapping, + /// Assign a balance to every account ID + balances: Mapping, } impl MyContract { - #[ink(constructor)] - pub fn new(count: u32) -> Self { - // This call is required in order to correctly initialize the - // `Mapping`s of our contract. - ink_lang::utils::initialize_contract(|contract: &mut Self| { - let caller = Self::env().caller(); - contract.map.insert(&caller, &count); - }) + /// Constructor that initializes the contract with an empty mapping. + #[ink(constructor, payable)] + pub fn new() -> Self { + let balances = Mapping::default(); + Self { balances } } - #[ink(constructor)] - pub fn default() -> Self { - // Even though we're not explicitly initializing the `Mapping`, - // we still need to call this - ink_lang::utils::initialize_contract(|_| {}) + /// Retreive the balance of the caller. + #[ink(message)] + pub fn get_balance(&self) -> Option { + let caller = self.env().caller(); + self.balances.get(caller) } - // Grab the number at the caller's AccountID, if it exists - #[ink(message)] - pub fn get(&self) -> u32 { - let caller = Self::env().caller(); - self.map.get(&caller).unwrap_or_default() + /// Credit more money to the contract. + #[ink(message, payable)] + pub fn transfer(&mut self) { + let caller = self.env().caller(); + let balance = self.balances.get(caller).unwrap_or(0); + let endowment = self.env().transferred_value(); + self.balances.insert(caller, &(balance + endowment)); + } + + /// Withdraw any balance from the contract. + pub fn withdraw(&mut self) { + let caller = self.env().caller(); + let balance = self.balances.get(caller).unwrap(); + self.balances.remove(caller); + self.env().transfer(caller, balance).unwrap() } } } + ``` + +## Considerations when using the `Mapping` type + +### Updating values + +The attentive reader has noticed that accessing mapping values via the `get()` function will +result in an owned value (a local copy), as opposed to a direct reference into the storage. +Changes to this value will not be reflected in the contracts storage "automatically". To +avoid this common pitfall, the value must be inserted again at the same key after it was modified. + +### Loading Behaviour + +Each mapping value lives under it's own storage key. Briefly, this means that mappings are +lazyly loaded in ink!. In other words, if your message does only access a single key of a +mapping, it will not load the whole mapping but only the value being accessed. From daf100ae7a0bf2bff57f0a6fd2a28800611eb64d Mon Sep 17 00:00:00 2001 From: xermicus Date: Mon, 23 Jan 2023 17:19:43 +0100 Subject: [PATCH 05/76] update mappings Signed-off-by: xermicus --- .../version-4.0.0-alpha.1/datastructures/mapping.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md index e6d893372c..814f1d44a4 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md @@ -20,8 +20,8 @@ case, each "user" gets credited their own balance. ## A simple working example -The following example constitues a contract where anyone can deposit and withdraw balances. -This functionality is realized using the `Mapping`. +The following example contract utilizes a `Mapping` so that anyone can deposit and withdraw +balance for their own account: ```rust #![cfg_attr(not(feature = "std"), no_std)] @@ -37,7 +37,7 @@ mod mycontract { } impl MyContract { - /// Constructor that initializes the contract with an empty mapping. + /// Constructor to initialize the contract with an empty mapping. #[ink(constructor, payable)] pub fn new() -> Self { let balances = Mapping::default(); @@ -60,7 +60,7 @@ mod mycontract { self.balances.insert(caller, &(balance + endowment)); } - /// Withdraw any balance from the contract. + /// Withdraw all your balance from the contract. pub fn withdraw(&mut self) { let caller = self.env().caller(); let balance = self.balances.get(caller).unwrap(); @@ -78,10 +78,10 @@ mod mycontract { The attentive reader has noticed that accessing mapping values via the `get()` function will result in an owned value (a local copy), as opposed to a direct reference into the storage. -Changes to this value will not be reflected in the contracts storage "automatically". To +Changes to this value won't be reflected in the contracts storage "automatically". To avoid this common pitfall, the value must be inserted again at the same key after it was modified. -### Loading Behaviour +### Storage loading behaviour Each mapping value lives under it's own storage key. Briefly, this means that mappings are lazyly loaded in ink!. In other words, if your message does only access a single key of a From a985a0b3f6d4aa491132358075cd535063f68a31 Mon Sep 17 00:00:00 2001 From: xermicus Date: Mon, 23 Jan 2023 19:09:40 +0100 Subject: [PATCH 06/76] WIP storage layout Signed-off-by: xermicus --- static/img/kv.svg | 563 +++++++++++++----- ...ad-storage-layout.md => storage-layout.md} | 28 +- 2 files changed, 437 insertions(+), 154 deletions(-) rename versioned_docs/version-4.0.0-alpha.1/datastructures/{spread-storage-layout.md => storage-layout.md} (70%) diff --git a/static/img/kv.svg b/static/img/kv.svg index d9ce02507b..eeed8f35b4 100644 --- a/static/img/kv.svg +++ b/static/img/kv.svg @@ -1,7 +1,50 @@ - - - - diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/spread-storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md similarity index 70% rename from versioned_docs/version-4.0.0-alpha.1/datastructures/spread-storage-layout.md rename to versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index 571f10b5cc..f286c87108 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/spread-storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -1,15 +1,12 @@ --- -title: Spread Storage Layout +title: Storage Layout slug: /datastructures/storage-layout --- -:::caution -TODO +Let's dive a bit deeper into the concepts behind ink! storage, so you can get a better +understanding of some of its implications. -Beware, this page is no longer up to date for 4.0! -::: - -### Storage Organization +## Storage Organization The following schema depicts the storage which is exposed to ink! by the contracts pallet: @@ -18,18 +15,20 @@ to ink! by the contracts pallet: Storage Organization: Layout -ink!'s storage operates by storing and loading entries into and from a single storage -cell. At the moment there is no way to customize this behaviour. Depending on the data -we're dealing with, this can end up being good or bad. +ink!'s storage API operates by storing and loading entries into and from a single storage +cell. Since the storage API does not care about the values at all - a value is just an +arbitrary byte sequence after all - smart contract authors are given some flexibility in +regards on how they want to organize the storage layout of their contracts. -For example, if we have a somewhat small `ink_prelude::vec::Vec` loading all the elements -at the same time can be advantegous - especially if we expect our message to interact +## Packed vs Non-Packed layout +For example, if we have a somewhat small contract storage consisting of only a few tiny +fields, loading this vector the same time can be advantegous - especially if we expect our message to interact with most of them in a single call. On the other hand, this can be problematic if we're loading a large `Vec` and only operating on a few elements. -### Spreading +### Eager Loading vs. Lazy Loading ink! spreads information to as many cells as possible. For example if you have the following `#[ink(storage)]` struct every field will live in its own single storage cell. @@ -84,3 +83,6 @@ Eager loading does **not** apply to `Mapping` fields, though, as key lookups in are done directly from contract storage. ::: + +## Considerations + From bbf76109c22694f76ba4fb21d283494faebd640c Mon Sep 17 00:00:00 2001 From: xermicus Date: Tue, 24 Jan 2023 13:38:08 +0100 Subject: [PATCH 07/76] explain packed layout Signed-off-by: xermicus --- .../datastructures/storage-layout.md | 28 +++++++++++++------ 1 file changed, 19 insertions(+), 9 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index f286c87108..c728da0389 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -15,18 +15,28 @@ to ink! by the contracts pallet: Storage Organization: Layout -ink!'s storage API operates by storing and loading entries into and from a single storage -cell. Since the storage API does not care about the values at all - a value is just an -arbitrary byte sequence after all - smart contract authors are given some flexibility in +The storage API operates by storing and loading entries into and from a single storage +cell. Since it does not care about the values at all - a value is just an arbitrary +byte sequence after all - smart contract authors are given some flexibility in regards on how they want to organize the storage layout of their contracts. ## Packed vs Non-Packed layout -For example, if we have a somewhat small contract storage consisting of only a few tiny -fields, loading this vector the same time can be advantegous - especially if we expect our message to interact -with most of them in a single call. - -On the other hand, this can be problematic if we're loading a large `Vec` and only -operating on a few elements. +Types that can be stored as a whole under a single storage cell are considered +[`Packed`](https://paritytech.github.io/ink/ink/storage/traits/trait.Packed.html). +By default, ink! tries to store all storage struct fields under a single storage cell. + +Consequentially, with a `Packed` storage layout, any message interacting with the contract +storage will always need to operate on the whole storage struct. + +For example, if we have a somewhat small contract storage struct consisting of only a few +tiny fields, reading (decoding) and writing (encoding) the whole storage struct inside +every message is not problematic. It may even be advantegous - especially if we expect most +messages to interact with most of the storage fields. + +On the other hand, this can get problematic if we're storing a large `Vec` on the +contract storage but provide messages that do not need to read and write from this `Vec`. +In that scenario, each and every contract message bears the runtime gas costs dealing +with that `Vec`, regardless whether they access it or not, resulting in extra gas costs. ### Eager Loading vs. Lazy Loading From ee9fa226d67a043744ffcb191db14f50bea08c13 Mon Sep 17 00:00:00 2001 From: xermicus Date: Tue, 24 Jan 2023 14:24:17 +0100 Subject: [PATCH 08/76] WIP Signed-off-by: xermicus --- .../datastructures/storage-layout.md | 34 ++++--------------- 1 file changed, 6 insertions(+), 28 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index c728da0389..73ceb4479c 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -21,6 +21,7 @@ byte sequence after all - smart contract authors are given some flexibility in regards on how they want to organize the storage layout of their contracts. ## Packed vs Non-Packed layout + Types that can be stored as a whole under a single storage cell are considered [`Packed`](https://paritytech.github.io/ink/ink/storage/traits/trait.Packed.html). By default, ink! tries to store all storage struct fields under a single storage cell. @@ -29,43 +30,18 @@ Consequentially, with a `Packed` storage layout, any message interacting with th storage will always need to operate on the whole storage struct. For example, if we have a somewhat small contract storage struct consisting of only a few -tiny fields, reading (decoding) and writing (encoding) the whole storage struct inside -every message is not problematic. It may even be advantegous - especially if we expect most -messages to interact with most of the storage fields. +tiny fields, pulling everything from the storage inside every message is not +problematic. It may even be advantegous - especially if we expect most messages to +interact with most of the storage fields. On the other hand, this can get problematic if we're storing a large `Vec` on the contract storage but provide messages that do not need to read and write from this `Vec`. In that scenario, each and every contract message bears the runtime gas costs dealing with that `Vec`, regardless whether they access it or not, resulting in extra gas costs. -### Eager Loading vs. Lazy Loading - -ink! spreads information to as many cells as possible. For example if you have the -following `#[ink(storage)]` struct every field will live in its own single storage cell. -Note that for `b` all 32 bytes will share the same cell! - -```rust -#[ink(storage)] -pub struct Spread { - a: i32, - b: [u8; 32], -} -``` - -The following schema depicts the storage layout for a vector with three elements, -persisted to storage in a spread layout. - -
- Storage Organization: Spreading -
- ## Eager Loading vs. Lazy Loading -When executing a contract, all the fields of the `#[ink(storage)]` struct will be pulled -from storage, regardless of whether or not they are used during the message execution. -Smart contract authors should be aware of this behaviour since it could potentially -affect their contract performance. For example, consider the following storage struct: ```rust #[ink(storage)] @@ -82,6 +58,8 @@ impl EagerLoading { } ``` +## Manual Key + In `EagerLoading::read_a()` we only read the `a` storage item. However, the `b` storage item will still be loaded from storage. As a reminder, this means accessing the underlying database and SCALE decoding the value. This can incur high costs, especially From 1ef577b12def9216d4cfd193ff9933bcda7d646d Mon Sep 17 00:00:00 2001 From: xermicus Date: Tue, 24 Jan 2023 16:57:17 +0100 Subject: [PATCH 09/76] explain lazy Signed-off-by: xermicus --- .../datastructures/storage-layout.md | 83 ++++++++++++++----- 1 file changed, 60 insertions(+), 23 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index 73ceb4479c..f852c48235 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -3,8 +3,8 @@ title: Storage Layout slug: /datastructures/storage-layout --- -Let's dive a bit deeper into the concepts behind ink! storage, so you can get a better -understanding of some of its implications. +Let's dive deeper into the concepts behind ink! storage to get a better understanding +of some of its implications and limitations. ## Storage Organization @@ -25,7 +25,6 @@ regards on how they want to organize the storage layout of their contracts. Types that can be stored as a whole under a single storage cell are considered [`Packed`](https://paritytech.github.io/ink/ink/storage/traits/trait.Packed.html). By default, ink! tries to store all storage struct fields under a single storage cell. - Consequentially, with a `Packed` storage layout, any message interacting with the contract storage will always need to operate on the whole storage struct. @@ -36,41 +35,79 @@ interact with most of the storage fields. On the other hand, this can get problematic if we're storing a large `Vec` on the contract storage but provide messages that do not need to read and write from this `Vec`. -In that scenario, each and every contract message bears the runtime gas costs dealing +In that scenario, each and every contract message bears runtime overhead by dealing with that `Vec`, regardless whether they access it or not, resulting in extra gas costs. +To solve this problem we need to turn our storage into a `Non-Packed` layout somehow. ## Eager Loading vs. Lazy Loading +With the [`Lazy`](https://paritytech.github.io/ink/ink/storage/struct.Lazy.html) primitive, +ink! provides means of breaking the storage up into smaller pieces, which can be loaded +on demand. Wrapping any storage field inside a `Lazy` struct makes the storage +`Non-Packed`, preventing it being eagerly loaded by arbitrary storage operations. - +The following example demonstrates how we can solve the problem introduced in the above +section. You'll notice that for the lazily loaded storage field, we now work with getters +and setters to access and modify the underlying storage value: ```rust -#[ink(storage)] -pub struct EagerLoading { - a: i32, - b: ink_prelude::vec::Vec, -} +#![cfg_attr(not(feature = "std"), no_std)] + +#[ink::contract] +mod mycontract { + use ink::prelude::vec::Vec; + use ink::storage::Lazy; + + /// Simple example of lazily loaded contract storage. + #[derive(Default)] + #[ink(storage)] + pub struct MyContract { + tiny_value: Balance, + /// This vector might get large and expensive to work with. + /// We want to enforce a `Non-Packed` storage layout. + large_vec: Lazy>, + } -impl EagerLoading { - #[ink(message)] - pub fn read_a(&self) { - let a = self.a; + impl MyContract { + #[ink(constructor)] + pub fn new() -> Self { + Self::default() + } + + /// Because `large_vec` is loaded lazily, this message is always cheap. + #[ink(message)] + pub fn get_balance(&self) -> Balance { + self.tiny_value + } + + /// Modify `large_vec` via dedicated `get()` and `set()` storage operators. + #[ink(message)] + pub fn add_balance(&mut self, balance: Balance) { + let mut large_vec = self.large_vec.get_or_default(); + large_vec.push(balance); + self.large_vec.set(&large_vec); + } } } ``` -## Manual Key - -In `EagerLoading::read_a()` we only read the `a` storage item. However, the `b` storage -item will still be loaded from storage. As a reminder, this means accessing the -underlying database and SCALE decoding the value. This can incur high costs, especially -as the number of elements in `b` grows. - :::note -Eager loading does **not** apply to `Mapping` fields, though, as key lookups in mappings -are done directly from contract storage. +`Vec` are always loaded as a whole, meaning that all elements of the `Vec` live under a +single storage key. Wrapping the `Vec` inside `Lazy`, like the provided example above does, +has no influence on its elements. If you are dealing with sparse arrays on contract +storage, consider using a `Mapping` instead. ::: ## Considerations +It might be worthwhile to think about the desired storage layout of your contract. While +using a `Packed` layout will keep your contracts overall code size smaller, it can cause +unnecessarily high gas costs. Thus we consider it a good practice to break up large +or complex storage layouts into reasonably sized distinct storage cells. + +:::note + +ink! `Mappings` are always `Non-Packed` and loaded lazily, one key-value pair at the time. + +::: From 270f826a8aee087540ce6deb68503fdf914b34d6 Mon Sep 17 00:00:00 2001 From: xermicus Date: Tue, 24 Jan 2023 17:03:23 +0100 Subject: [PATCH 10/76] update example Signed-off-by: xermicus --- .../datastructures/storage-layout.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index f852c48235..54af471c4f 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -57,7 +57,6 @@ mod mycontract { use ink::prelude::vec::Vec; use ink::storage::Lazy; - /// Simple example of lazily loaded contract storage. #[derive(Default)] #[ink(storage)] pub struct MyContract { @@ -73,21 +72,22 @@ mod mycontract { Self::default() } - /// Because `large_vec` is loaded lazily, this message is always cheap. + /// Because `large_vec` is loaded lazyly, this message is always cheap. #[ink(message)] pub fn get_balance(&self) -> Balance { self.tiny_value } - /// Modify `large_vec` via dedicated `get()` and `set()` storage operators. + /// Lazy fields like `large_vec` provide `get()` and `set()` storage operators. #[ink(message)] - pub fn add_balance(&mut self, balance: Balance) { - let mut large_vec = self.large_vec.get_or_default(); - large_vec.push(balance); - self.large_vec.set(&large_vec); + pub fn add_balance(&mut self, value: Balance) { + let mut balances = self.large_vec.get_or_default(); + balances.push(value); + self.large_vec.set(&balances); } } } +} ``` :::note From 07398fbdcbaca911a89a1aafc24973c8741aee5a Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Tue, 24 Jan 2023 18:28:05 +0100 Subject: [PATCH 11/76] WIP custom storage types Signed-off-by: Cyrill Leutwiler --- .../datastructures/custom.md | 71 ++++++++++--------- .../datastructures/storage-layout.md | 2 +- 2 files changed, 38 insertions(+), 35 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md index 3b7eec23f7..9a1bacb9df 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md @@ -3,48 +3,51 @@ title: Custom Data Structures slug: /datastructures/custom-datastructure --- -:::caution -TODO - -Beware, this page is no longer up to date for 4.0! -::: - -While the `ink_storage` crate provides tons of useful utilities and data structures to organize and manipulate the contract's storage contract authors are not limited by its capabilities. By implementing the core `SpreadLayout`/`PackedLayout` traits (and the `StorageLayout` trait for supporting the metadata generated for the `.contract` bundle) users are able to define their very own custom storage data structures with their own set of requirement and features that work along the `ink_storage` data structures as long as they fulfill the mere requirements stated by those two traits. - -A basic example of a custom struct is shown below: - -``` rust -struct Inner { - value: bool +While the `ink_storage` crate provides useful utilities and data structures to organize and +manipulate the contract's storage contract authors are not limited by its capabilities. + +## Using custom types on storage +Any custom type wanting to be compatible with ink! storage must implement the +[`Storable`](https://docs.rs/ink_storage_traits/4.0.0-beta/ink_storage_traits/trait.Storable.html) +trait, so it can be SCALE +[`encoded`](https://docs.rs/parity-scale-codec/3.2.2/parity_scale_codec/trait.Encode.html) +and +[`decoded`](https://docs.rs/parity-scale-codec/3.2.2/parity_scale_codec/trait.Decode.html). +Additionaly, the traits +[`StorageLayout`](https://docs.rs/ink_storage/latest/ink_storage/traits/trait.StorageLayout.html) +and [`TypeInfo`](https://docs.rs/scale-info/2.3.1/scale_info/trait.TypeInfo.html) +are required as well. But don't worry, usually these traits can just be derived: + +```rust +/// A custom type on our contract storage +#[derive(scale::Decode, scale::Encode, Debug)] +#[cfg_attr( + feature = "std", + derive(scale_info::TypeInfo, ink::storage::traits::StorageLayout) +)] +pub struct Inner { + value: bool, } #[ink(storage)] -pub struct MyContract { - inner: Inner +pub struct MyContractStorage { + inner: Inner, } ``` -Compiling the above will result in errors. While having an inner struct which holds only a boolean might not be the best idea, it serves well to illustrate how to implement the trait: - -``` rust -impl SpreadLayout for Inner { - const FOOTPRINT: u64 = 1; +Even better: there is a macro `#[ink::storage_item]`, which derives all necessary traits for you. Unless you need to implement any behaviour, the above code example can be +simplified as follows: - fn pull_spread(ptr: &mut KeyPtr) -> Self { - Self { - value: SpreadLayout::pull_spread(ptr), - } - } - - fn push_spread(&self, ptr: &mut KeyPtr) { - SpreadLayout::push_spread(&self.value, ptr); - } - - fn clear_spread(&self, ptr: &mut KeyPtr) { - SpreadLayout::clear_spread(&self.value, ptr); - } +```rust +/// A custom type on our contract storage +#[ink::storage_item] +pub struct Inner { + value: bool, } +#[ink(storage)] +pub struct SparseArray { + inner: Inner, +} ``` -You can check what each method does in the [trait's docs](https://docs.rs/ink_storage/4.0.0-beta/ink_storage/traits/trait.SpreadLayout.html). Check how some data structures are implemented, such as [Mapping](https://docs.rs/ink_storage/4.0.0-beta/src/ink_storage/lazy/mapping.rs.html#113). diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index 54af471c4f..6791745493 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -90,7 +90,7 @@ mod mycontract { } ``` -:::note +:::caution `Vec` are always loaded as a whole, meaning that all elements of the `Vec` live under a single storage key. Wrapping the `Vec` inside `Lazy`, like the provided example above does, From 16d86dbf9f26806f891cd7a83bb4940222f522a9 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Tue, 24 Jan 2023 22:49:02 +0100 Subject: [PATCH 12/76] custom storage types Signed-off-by: Cyrill Leutwiler --- .../datastructures/custom.md | 68 ++++++++++++++++++- .../datastructures/mapping.md | 13 ++-- .../datastructures/storage-layout.md | 4 +- 3 files changed, 76 insertions(+), 9 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md index 9a1bacb9df..5f2abafd93 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md @@ -4,7 +4,7 @@ slug: /datastructures/custom-datastructure --- While the `ink_storage` crate provides useful utilities and data structures to organize and -manipulate the contract's storage contract authors are not limited by its capabilities. +manipulate the contract's storagem, contract authors are not limited by its capabilities. ## Using custom types on storage Any custom type wanting to be compatible with ink! storage must implement the @@ -35,8 +35,8 @@ pub struct MyContractStorage { } ``` -Even better: there is a macro `#[ink::storage_item]`, which derives all necessary traits for you. Unless you need to implement any behaviour, the above code example can be -simplified as follows: +Even better: there is a macro `#[ink::storage_item]`, which derives all necessary traits for you. If there is no need to implement any special behaviour, the above code example +can be simplified further as follows: ```rust /// A custom type on our contract storage @@ -51,3 +51,65 @@ pub struct SparseArray { } ``` +## Generic storage structs + +It is possible to use generic data types in your storage, as long as any generic type +satisfies the required storage trait bounds. + +Let's say you want a mapping where accessing a non-existant key should just return +it's default value, akin to how mappings work in Solidity. Additionally, you want to know +how many values there are in the mapping (its length). This could be implemented as a +thin wrapper around the ink! `Mapping` as follows: + + +```rust +/// Values for this map need to implement the `Default` trait. +#[ink::storage_item] +pub struct DefaultMap { + values: Mapping, + length: u32, +} + +impl DefaultMap { + /// Accessing non-existant keys will return the default value. + pub fn get(&self, key: &K) -> V { + self.values.get(key).unwrap_or_default() + } + + /// Inserting into the map increases it's length by one. + pub fn set(&mut self, key: I, value: &E) + where + I: scale::EncodeLike, + E: scale::EncodeLike + Storable, + { + if self.values.insert(key, value).is_none() { + self.length += 1 + } + } + + /// Removing a value from the map decreases it's length by one. + pub fn remove(&mut self, key: &K) { + if self.values.take(key).is_some() { + self.length -= 1 + } + } + + /// Return how many values the mapping contains + pub fn len(&self) -> u32 { + self.length + } +} + +/// `DefaultMap` is compatible with contract storage. +#[ink(storage)] +pub struct MyContract { + my_map: DefaultMap, +} +``` + +:::caution + +Generic data types may substantially increase your contracts overall code size. + +::: + diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md index 814f1d44a4..b18794cea8 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md @@ -76,13 +76,18 @@ mod mycontract { ### Updating values -The attentive reader has noticed that accessing mapping values via the `get()` function will -result in an owned value (a local copy), as opposed to a direct reference into the storage. -Changes to this value won't be reflected in the contracts storage "automatically". To -avoid this common pitfall, the value must be inserted again at the same key after it was modified. +The attentive reader may have noticed that accessing mapping values via the `get()` +function will result in an owned value (a local copy), as opposed to a direct reference +into the storage. Changes to this value won't be reflected in the contracts storage +"automatically". To avoid this common pitfall, the value must be inserted again at the same +key after it was modified. ### Storage loading behaviour Each mapping value lives under it's own storage key. Briefly, this means that mappings are lazyly loaded in ink!. In other words, if your message does only access a single key of a mapping, it will not load the whole mapping but only the value being accessed. + +Furthermore, this implies that it is not possible to iterate over the contents of a map. +Circumventing this feature by storing populated keys inside a `Vec` is not adviseable +as this might result in very high gas costs. diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index 6791745493..a6ad6e4a3d 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -42,8 +42,8 @@ To solve this problem we need to turn our storage into a `Non-Packed` layout som ## Eager Loading vs. Lazy Loading With the [`Lazy`](https://paritytech.github.io/ink/ink/storage/struct.Lazy.html) primitive, ink! provides means of breaking the storage up into smaller pieces, which can be loaded -on demand. Wrapping any storage field inside a `Lazy` struct makes the storage -`Non-Packed`, preventing it being eagerly loaded by arbitrary storage operations. +on demand. Wrapping any storage field inside a `Lazy` struct makes the storage also +`Non-Packed`, preventing it from being eagerly loaded during arbitrary storage operations. The following example demonstrates how we can solve the problem introduced in the above section. You'll notice that for the lazily loaded storage field, we now work with getters From 037c35be258aa70afd3d5b7805cf876aa3217c1c Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Tue, 24 Jan 2023 23:03:08 +0100 Subject: [PATCH 13/76] WIP metadata storage docs Signed-off-by: Cyrill Leutwiler --- .../datastructures/custom.md | 7 +++++-- .../datastructures/storage-in-metadata.md | 18 +++++++++++++++--- 2 files changed, 20 insertions(+), 5 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md index 5f2abafd93..4c7c25fad5 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md @@ -51,10 +51,11 @@ pub struct SparseArray { } ``` -## Generic storage structs +## Generic storage fields It is possible to use generic data types in your storage, as long as any generic type -satisfies the required storage trait bounds. +satisfies the required storage trait bounds. In fact, we already witnessed this in the +previous sections about the `Mapping`. Let's say you want a mapping where accessing a non-existant key should just return it's default value, akin to how mappings work in Solidity. Additionally, you want to know @@ -64,6 +65,8 @@ thin wrapper around the ink! `Mapping` as follows: ```rust /// Values for this map need to implement the `Default` trait. +/// Naturally, they also must be compatible with contract storage. +/// Note that the underlying `Mapping` type only supports `Packed` values. #[ink::storage_item] pub struct DefaultMap { values: Mapping, diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md index e48ff955f9..e255ea0863 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md @@ -3,6 +3,18 @@ title: Metadata Format slug: /datastructures/storage-in-metadata --- -:::caution -TODO: Add documentation on the `storage ` field in the metadata. -::: +The storage layout of a contract is reflected inside it's metadata. It allows third-party +tooling to work with contract and can also help to better understand the storage layout of ] +a given contract. + +The following snippet is an example of how that might look like: + +```json +TODO +``` + +## Storage key generation +TODO +- explain hashing algorithm used in ink +- explain manualkey vs. autokey +- explain how keys are calculated for mappings \ No newline at end of file From 009a0d1d09abd62dd75c9f81cef6a4559831587d Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Wed, 25 Jan 2023 16:43:34 +0100 Subject: [PATCH 14/76] update storage layout in metadata Signed-off-by: Cyrill Leutwiler --- static/img/kv.svg | 35 +++++-- .../datastructures/storage-in-metadata.md | 91 +++++++++++++++++-- .../datastructures/storage-layout.md | 37 +++++++- 3 files changed, 139 insertions(+), 24 deletions(-) diff --git a/static/img/kv.svg b/static/img/kv.svg index eeed8f35b4..22a136c609 100644 --- a/static/img/kv.svg +++ b/static/img/kv.svg @@ -21,16 +21,22 @@ inkscape:deskcolor="#d1d1d1" showgrid="false" inkscape:zoom="0.81726684" - inkscape:cx="814.29953" - inkscape:cy="408.06746" - inkscape:window-width="2560" - inkscape:window-height="1369" + inkscape:cx="815.52312" + inkscape:cy="406.84387" + inkscape:window-width="1920" + inkscape:window-height="1019" inkscape:window-x="0" inkscape:window-y="0" inkscape:window-maximized="1" inkscape:current-layer="Layer_1" /> + Key / Value Database- Arbitrary key length- SCALE encoded values- Values are any byte sequence + id="tspan613" /> + diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md index e255ea0863..b4624dc30f 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md @@ -3,18 +3,89 @@ title: Metadata Format slug: /datastructures/storage-in-metadata --- -The storage layout of a contract is reflected inside it's metadata. It allows third-party -tooling to work with contract and can also help to better understand the storage layout of ] -a given contract. +The storage layout of a contract is reflected inside the metadata. It allows third-party +tooling to work with contract stroage and can also help to better understand the storage +layout of any given contract. -The following snippet is an example of how that might look like: +Given a contract with the following storage: + +```rust +#[ink(storage)] +pub struct MyContract { + balance: Balance, + block: BlockNumber, + something: Lazy, +} +``` + +The storage will be reflected inside the metadata as like follows: ```json -TODO +"root": { + "layout": { + "struct": { + "fields": [ + { + "layout": { + "leaf": { + "key": "0x00000000", + "ty": 0 + } + }, + "name": "balance" + }, + { + "layout": { + "leaf": { + "key": "0x00000000", + "ty": 1 + } + }, + "name": "block" + }, + { + "layout": { + "root": { + "layout": { + "leaf": { + "key": "0xb1f4904e", + "ty": 2 + } + }, + "root_key": "0xb1f4904e" + } + }, + "name": "something" + } + ], + "name": "MyContract" + } + }, + "root_key": "0x00000000" +} ``` -## Storage key generation -TODO -- explain hashing algorithm used in ink -- explain manualkey vs. autokey -- explain how keys are calculated for mappings \ No newline at end of file +We observe that the storage is layed out as a tree, where tangible storage values end up +inside a `leaf`. Because of `Packed` encoding, leafs can share the same storage key, and +in order to reach them you'd need fetch and decode the whole storage cell under this key. + +A `root_key` is meant to either be used to directly access a `Packed` storage field +or to serve as the base key for calculting the actual keys needed to access values in +`Non-Packed` fields (such as Mappings). + +## Storage key calculation for mappings + +Root storage keys are always 4 bytes in size. However, the storage API of the contracts +pallet supports keys of arbitrary length. In order to reach a mapping value, the storage +key of said value is calculated at runtime. + +The formula to calculate a storage key `S` used to access a mapping value under the key `K` +for a mapping with base key `B` can be expressed as follows: + +``` +S = B + scale::encode(K) +``` + +In words, find the base (root) key of the mapping and concatenate it with the +SCALE encoded key of the mapped value to obtain the actual storage key used to +access the mapped value. diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index a6ad6e4a3d..b9860bb3c4 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -3,6 +3,7 @@ title: Storage Layout slug: /datastructures/storage-layout --- +Smart contract authors are given some flexibility in regards on how they want to organize the storage layout of their contracts. Let's dive deeper into the concepts behind ink! storage to get a better understanding of some of its implications and limitations. @@ -15,10 +16,10 @@ to ink! by the contracts pallet: Storage Organization: Layout -The storage API operates by storing and loading entries into and from a single storage -cell. Since it does not care about the values at all - a value is just an arbitrary -byte sequence after all - smart contract authors are given some flexibility in -regards on how they want to organize the storage layout of their contracts. +Storage data is always encoded with the +[`SCALE`](https://docs.substrate.io/reference/scale-codec/) codec. +The storage API operates by storing and loading entries into and from a single storages cells, where each storage cell is accessed under it's own dedicated storage key. To some +extent, the storage API works similar to a traditional key-value database. ## Packed vs Non-Packed layout @@ -99,6 +100,34 @@ storage, consider using a `Mapping` instead. ::: +## Manual vs. Automatic Key generation + +Per default, keys are calculated automatically for you, thanks to the +[`AutoKey`](https://docs.rs/ink_storage_traits/4.0.0-beta.1/ink_storage_traits/struct.AutoKey.html) +primitive. They'll be generated at compile time and ruled out for conflicts. However, for `Non-Packed` types like `Lazy` or the `Mapping`, the +[`ManualKey`](https://docs.rs/ink_storage_traits/4.0.0-beta.1/ink_storage_traits/struct.ManualKey.html) +primitive allows manual control over the storage key of a field like so: + +```rust +#[ink(storage)] +pub struct MyContract { + /// The storage key for this field is always `0x0000007f` + inner: Lazy>, +} +``` + +This may be advantegous: Your storage key will always stay the same, regardless of +the version of your contract or ink! itself (note that the key calcualtion algorithm may +change with future ink! versions). + +:::caution + +Using `ManualKey` instead of `AutoKey` might be especially desireable for upgradable +contracts, as using `AutoKey` might result in a different storage key for the same field +in a newer version of the contract. Which will break the contract after an upgrade! + +::: + ## Considerations It might be worthwhile to think about the desired storage layout of your contract. While From fc05550bf5f6e4d033c20f89e450b6c1ae82b993 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Wed, 25 Jan 2023 19:44:45 +0100 Subject: [PATCH 15/76] typos Signed-off-by: Cyrill Leutwiler --- .../version-4.0.0-alpha.1/datastructures/mapping.md | 4 ++-- .../version-4.0.0-alpha.1/datastructures/storage-layout.md | 7 +++---- 2 files changed, 5 insertions(+), 6 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md index b18794cea8..544ed61a4b 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md @@ -11,7 +11,7 @@ Here is an example of a mapping from a user to a `Balance`: #[ink(storage)] pub struct MyContract { /// Assign a balance to every account. - balances: ink::storage::Mapping, + map: balances: ink::storage::Mapping } ``` @@ -85,7 +85,7 @@ key after it was modified. ### Storage loading behaviour Each mapping value lives under it's own storage key. Briefly, this means that mappings are -lazyly loaded in ink!. In other words, if your message does only access a single key of a +lazily loaded in ink!. In other words, if your message does only access a single key of a mapping, it will not load the whole mapping but only the value being accessed. Furthermore, this implies that it is not possible to iterate over the contents of a map. diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index b9860bb3c4..32982ecff3 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -73,7 +73,7 @@ mod mycontract { Self::default() } - /// Because `large_vec` is loaded lazyly, this message is always cheap. + /// Because `large_vec` is loaded lazily, this message is always cheap. #[ink(message)] pub fn get_balance(&self) -> Balance { self.tiny_value @@ -88,15 +88,14 @@ mod mycontract { } } } -} ``` :::caution `Vec` are always loaded as a whole, meaning that all elements of the `Vec` live under a single storage key. Wrapping the `Vec` inside `Lazy`, like the provided example above does, -has no influence on its elements. If you are dealing with sparse arrays on contract -storage, consider using a `Mapping` instead. +has no influence on its elements. If you are dealing with large or sparse arrays on +contract storage, consider using a `Mapping` instead. ::: From c873ff072045842906c886bdea610c0414599d58 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Thu, 26 Jan 2023 11:35:35 +0100 Subject: [PATCH 16/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Michael Müller --- versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md | 1 + 1 file changed, 1 insertion(+) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md index 4c7c25fad5..6b29ec520c 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md @@ -113,6 +113,7 @@ pub struct MyContract { :::caution Generic data types may substantially increase your contracts overall code size. +The reason for this is [Rust's monomorphization](https://rustwasm.github.io/twiggy/concepts/generic-functions-and-monomorphization.html). ::: From d39bf54baf702c6e95728888123ea18216ea5338 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Thu, 26 Jan 2023 11:36:04 +0100 Subject: [PATCH 17/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Michael Müller --- .../version-4.0.0-alpha.1/datastructures/storage-layout.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index 32982ecff3..1c3b97e38b 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -43,7 +43,8 @@ To solve this problem we need to turn our storage into a `Non-Packed` layout som ## Eager Loading vs. Lazy Loading With the [`Lazy`](https://paritytech.github.io/ink/ink/storage/struct.Lazy.html) primitive, ink! provides means of breaking the storage up into smaller pieces, which can be loaded -on demand. Wrapping any storage field inside a `Lazy` struct makes the storage also +on demand. Wrapping any storage field inside a `Lazy` struct makes the storage +struct in which that field appears also `Non-Packed`, preventing it from being eagerly loaded during arbitrary storage operations. The following example demonstrates how we can solve the problem introduced in the above From a2d48804eb083322afda6c7d5ee0726199cca61d Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Thu, 26 Jan 2023 11:37:00 +0100 Subject: [PATCH 18/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Michael Müller --- .../version-4.0.0-alpha.1/datastructures/storage-layout.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index 1c3b97e38b..271574c487 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -38,7 +38,7 @@ On the other hand, this can get problematic if we're storing a large `Vec` on th contract storage but provide messages that do not need to read and write from this `Vec`. In that scenario, each and every contract message bears runtime overhead by dealing with that `Vec`, regardless whether they access it or not, resulting in extra gas costs. -To solve this problem we need to turn our storage into a `Non-Packed` layout somehow. +To solve this problem we need to turn our storage into a non-packed layout somehow. ## Eager Loading vs. Lazy Loading With the [`Lazy`](https://paritytech.github.io/ink/ink/storage/struct.Lazy.html) primitive, From c5ee9dbddbb423d49395b5eea70a764e8aa4dad7 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Fri, 27 Jan 2023 10:42:55 +0100 Subject: [PATCH 19/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md Co-authored-by: Hernando Castano --- .../version-4.0.0-alpha.1/datastructures/overview.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md index 75bfed6fb8..e1249ce737 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md @@ -5,8 +5,8 @@ slug: /datastructures/overview The `ink_storage` crate acts as the standard storage library for ink! smart contracts. At the moment it provides two primitives for interacting with storage, -[`Mapping`](https://docs.rs/ink_storage/4.0.0-beta/ink_storage/struct.Mapping.html) -and [`Lazy`](https://docs.rs/ink_storage/4.0.0-beta/ink_storage/struct.Lazy.html). +[`Mapping`](https://docs.rs/ink_storage/4.0.0-beta.1/ink_storage/struct.Mapping.html) +and [`Lazy`](https://docs.rs/ink_storage/4.0.0-beta.1/ink_storage/struct.Lazy.html). `Mapping` is a mapping of key-value pairs directly to the contract storage. It is very similar to traditional hash tables and comparable to the `mapping` type Solidity offers. From 14d4b8a25663d21df79b48b2a50c4f0ab240e1ff Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Fri, 27 Jan 2023 10:43:08 +0100 Subject: [PATCH 20/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md Co-authored-by: Hernando Castano --- versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md index e1249ce737..304da22fb1 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md @@ -19,4 +19,4 @@ functionality themselves. layout of individual types. Conceivably, it may be desirable to change certain aspects on how your contract deals with it's storage variables. You can find out more about this in the section about the ink! -[Storage Layout](https://use.ink/datastructures/storage-layout). +[Storage Layout](https://use.ink/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout). From d23196db24f3f1d6e1e3a4a64bd8d33b5f519f2a Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Fri, 27 Jan 2023 10:43:18 +0100 Subject: [PATCH 21/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md Co-authored-by: Hernando Castano --- versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md index 544ed61a4b..a067fd4f76 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md @@ -3,7 +3,7 @@ title: Working with Mapping slug: /datastructures/mapping --- -In this section we demonstrate how to work with ink! [`Mapping`](https://docs.rs/ink_storage/4.0.0-beta/ink_storage/struct.Mapping.html). +In this section we demonstrate how to work with ink! [`Mapping`](https://docs.rs/ink_storage/4.0.0-beta.1/ink_storage/struct.Mapping.html). Here is an example of a mapping from a user to a `Balance`: From 26ffbb34ffbef41621fa5665d28525f5f0f7dcd0 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Fri, 27 Jan 2023 10:43:36 +0100 Subject: [PATCH 22/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md Co-authored-by: Hernando Castano --- versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md index a067fd4f76..610604b67c 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md @@ -18,7 +18,7 @@ pub struct MyContract { This means that for a given key, you can store a unique instance of a value type. In this case, each "user" gets credited their own balance. -## A simple working example +## Example: Using a `Mapping` The following example contract utilizes a `Mapping` so that anyone can deposit and withdraw balance for their own account: From b75865efa81e93c588105c6da3bae8dd21141c98 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Fri, 27 Jan 2023 10:43:53 +0100 Subject: [PATCH 23/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md Co-authored-by: Hernando Castano --- .../version-4.0.0-alpha.1/datastructures/mapping.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md index 610604b67c..3235a5d900 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md @@ -76,8 +76,8 @@ mod mycontract { ### Updating values -The attentive reader may have noticed that accessing mapping values via the `get()` -function will result in an owned value (a local copy), as opposed to a direct reference +The attentive reader may have noticed that accessing mapping values via the `Mapping::get()` +method will result in an owned value (a local copy), as opposed to a direct reference into the storage. Changes to this value won't be reflected in the contracts storage "automatically". To avoid this common pitfall, the value must be inserted again at the same key after it was modified. From 601cb6912d84295af1ea8dbd2f942c8055d9f7e4 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Fri, 27 Jan 2023 10:44:06 +0100 Subject: [PATCH 24/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md Co-authored-by: Hernando Castano --- .../version-4.0.0-alpha.1/datastructures/storage-layout.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index 271574c487..f310064f09 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -100,7 +100,7 @@ contract storage, consider using a `Mapping` instead. ::: -## Manual vs. Automatic Key generation +## Manual vs. Automatic Key Generation Per default, keys are calculated automatically for you, thanks to the [`AutoKey`](https://docs.rs/ink_storage_traits/4.0.0-beta.1/ink_storage_traits/struct.AutoKey.html) From 278b18d382156169415b98c01211c5135c0638c2 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Fri, 27 Jan 2023 10:44:35 +0100 Subject: [PATCH 25/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md Co-authored-by: Hernando Castano --- .../version-4.0.0-alpha.1/datastructures/storage-layout.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index f310064f09..38b0cd2558 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -124,7 +124,7 @@ change with future ink! versions). Using `ManualKey` instead of `AutoKey` might be especially desireable for upgradable contracts, as using `AutoKey` might result in a different storage key for the same field -in a newer version of the contract. Which will break the contract after an upgrade! +in a newer version of the contract. This may break your contract after an upgrade 😱! ::: From 37dd49aaebfa244ba98955fa0d4835c990607f6c Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Fri, 27 Jan 2023 10:44:57 +0100 Subject: [PATCH 26/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md Co-authored-by: Hernando Castano --- .../version-4.0.0-alpha.1/datastructures/storage-layout.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index 38b0cd2558..f94e689312 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -120,7 +120,7 @@ This may be advantegous: Your storage key will always stay the same, regardless the version of your contract or ink! itself (note that the key calcualtion algorithm may change with future ink! versions). -:::caution +:::tip Using `ManualKey` instead of `AutoKey` might be especially desireable for upgradable contracts, as using `AutoKey` might result in a different storage key for the same field From 855c005495ec7957e5aed58462d793c6ee1d5a72 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Fri, 27 Jan 2023 10:45:08 +0100 Subject: [PATCH 27/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md Co-authored-by: Hernando Castano --- .../version-4.0.0-alpha.1/datastructures/storage-layout.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index f94e689312..0b6b945eeb 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -117,7 +117,7 @@ pub struct MyContract { ``` This may be advantegous: Your storage key will always stay the same, regardless of -the version of your contract or ink! itself (note that the key calcualtion algorithm may +the version of your contract or ink! itself (note that the key calculation algorithm may change with future ink! versions). :::tip From 3889490592629518ed4ef8b6052b11bdb71a99ef Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Fri, 27 Jan 2023 10:46:00 +0100 Subject: [PATCH 28/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md Co-authored-by: Hernando Castano --- versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md index 3235a5d900..18bcd3dc98 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md @@ -84,7 +84,7 @@ key after it was modified. ### Storage loading behaviour -Each mapping value lives under it's own storage key. Briefly, this means that mappings are +Each `Mapping` value lives under it's own storage key. Briefly, this means that `Mapping`s are lazily loaded in ink!. In other words, if your message does only access a single key of a mapping, it will not load the whole mapping but only the value being accessed. From 55ca0a9e9ca10b6a19495dfb49a6932db73c10ef Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Fri, 27 Jan 2023 10:46:24 +0100 Subject: [PATCH 29/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md Co-authored-by: Hernando Castano --- versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md index 18bcd3dc98..f4cf67bc47 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md @@ -85,7 +85,7 @@ key after it was modified. ### Storage loading behaviour Each `Mapping` value lives under it's own storage key. Briefly, this means that `Mapping`s are -lazily loaded in ink!. In other words, if your message does only access a single key of a +lazily loaded in ink!. In other words, if your message only accesses a single key of a mapping, it will not load the whole mapping but only the value being accessed. Furthermore, this implies that it is not possible to iterate over the contents of a map. From 0d06b29cdf9b962f7999ffd90c0b42f76bcf2c02 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Fri, 27 Jan 2023 10:46:53 +0100 Subject: [PATCH 30/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md Co-authored-by: Hernando Castano --- versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md index f4cf67bc47..0fc32c729c 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md @@ -89,5 +89,5 @@ lazily loaded in ink!. In other words, if your message only accesses a single ke mapping, it will not load the whole mapping but only the value being accessed. Furthermore, this implies that it is not possible to iterate over the contents of a map. -Circumventing this feature by storing populated keys inside a `Vec` is not adviseable +Circumventing this feature by storing populated keys inside an `ink_prelude::Vec` is not advisable as this might result in very high gas costs. From 3be20413703331644cdc709545332ca4e99f60d7 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Fri, 27 Jan 2023 10:48:23 +0100 Subject: [PATCH 31/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md Co-authored-by: Hernando Castano --- .../version-4.0.0-alpha.1/datastructures/storage-layout.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index 0b6b945eeb..d86d895e8b 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -23,7 +23,7 @@ extent, the storage API works similar to a traditional key-value database. ## Packed vs Non-Packed layout -Types that can be stored as a whole under a single storage cell are considered +Types that can be stored entirely under a single storage cell are considered [`Packed`](https://paritytech.github.io/ink/ink/storage/traits/trait.Packed.html). By default, ink! tries to store all storage struct fields under a single storage cell. Consequentially, with a `Packed` storage layout, any message interacting with the contract From de68341e0079ede955e8529329464dadeb546c09 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Fri, 27 Jan 2023 10:48:46 +0100 Subject: [PATCH 32/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md Co-authored-by: Hernando Castano --- .../version-4.0.0-alpha.1/datastructures/storage-layout.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index d86d895e8b..6967901d0c 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -27,7 +27,7 @@ Types that can be stored entirely under a single storage cell are considered [`Packed`](https://paritytech.github.io/ink/ink/storage/traits/trait.Packed.html). By default, ink! tries to store all storage struct fields under a single storage cell. Consequentially, with a `Packed` storage layout, any message interacting with the contract -storage will always need to operate on the whole storage struct. +storage will always need to operate on the entire contract storage struct. For example, if we have a somewhat small contract storage struct consisting of only a few tiny fields, pulling everything from the storage inside every message is not From ed67c093ea699b4ee2ca2bd6effd086a839b7cb7 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Fri, 27 Jan 2023 10:49:05 +0100 Subject: [PATCH 33/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md Co-authored-by: Hernando Castano --- .../version-4.0.0-alpha.1/datastructures/storage-layout.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index 6967901d0c..eed6ee2761 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -34,7 +34,7 @@ tiny fields, pulling everything from the storage inside every message is not problematic. It may even be advantegous - especially if we expect most messages to interact with most of the storage fields. -On the other hand, this can get problematic if we're storing a large `Vec` on the +On the other hand, this can get problematic if we're storing a large `Vec` in the contract storage but provide messages that do not need to read and write from this `Vec`. In that scenario, each and every contract message bears runtime overhead by dealing with that `Vec`, regardless whether they access it or not, resulting in extra gas costs. From 6a86095e001fa94bd29f9e7eef45cbb68cdea24e Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Fri, 27 Jan 2023 10:49:26 +0100 Subject: [PATCH 34/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md Co-authored-by: Hernando Castano --- .../version-4.0.0-alpha.1/datastructures/storage-layout.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index eed6ee2761..a1f8f167a5 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -37,7 +37,7 @@ interact with most of the storage fields. On the other hand, this can get problematic if we're storing a large `Vec` in the contract storage but provide messages that do not need to read and write from this `Vec`. In that scenario, each and every contract message bears runtime overhead by dealing -with that `Vec`, regardless whether they access it or not, resulting in extra gas costs. +with that `Vec`, regardless whether they access it or not. This results in extra gas costs. To solve this problem we need to turn our storage into a non-packed layout somehow. ## Eager Loading vs. Lazy Loading From 328216104f54adc04d05409fd3c3fb969c997906 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Fri, 27 Jan 2023 10:50:33 +0100 Subject: [PATCH 35/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md Co-authored-by: Hernando Castano --- .../version-4.0.0-alpha.1/datastructures/storage-layout.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index a1f8f167a5..7f60154643 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -93,7 +93,7 @@ mod mycontract { :::caution -`Vec` are always loaded as a whole, meaning that all elements of the `Vec` live under a +`Vec`'s are always loaded in their entirety. This is because all elements of the `Vec` live under a single storage key. Wrapping the `Vec` inside `Lazy`, like the provided example above does, has no influence on its elements. If you are dealing with large or sparse arrays on contract storage, consider using a `Mapping` instead. From a551e164792dc13f971a8452f02fbe2b093a63ae Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Fri, 27 Jan 2023 10:52:30 +0100 Subject: [PATCH 36/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md Co-authored-by: Hernando Castano --- .../version-4.0.0-alpha.1/datastructures/storage-layout.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index 7f60154643..2b0b095028 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -102,7 +102,7 @@ contract storage, consider using a `Mapping` instead. ## Manual vs. Automatic Key Generation -Per default, keys are calculated automatically for you, thanks to the +By default, keys are calculated automatically for you, thanks to the [`AutoKey`](https://docs.rs/ink_storage_traits/4.0.0-beta.1/ink_storage_traits/struct.AutoKey.html) primitive. They'll be generated at compile time and ruled out for conflicts. However, for `Non-Packed` types like `Lazy` or the `Mapping`, the [`ManualKey`](https://docs.rs/ink_storage_traits/4.0.0-beta.1/ink_storage_traits/struct.ManualKey.html) From 8dbce95e69ac956520cc35dccb0d9f02091072b7 Mon Sep 17 00:00:00 2001 From: xermicus Date: Sun, 29 Jan 2023 13:20:38 +0100 Subject: [PATCH 37/76] rework mapping overview Signed-off-by: xermicus --- .../version-4.0.0-alpha.1/datastructures/overview.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md index 304da22fb1..7993834a72 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md @@ -11,9 +11,10 @@ and [`Lazy`](https://docs.rs/ink_storage/4.0.0-beta.1/ink_storage/struct.Lazy.ht `Mapping` is a mapping of key-value pairs directly to the contract storage. It is very similar to traditional hash tables and comparable to the `mapping` type Solidity offers. As a core ingredient to the ink! language, its main advantage is being simple and -lightweight. However, it does not provide any high-level functionality, such as iteration -or automatic clean-up. Smart contract authors will need to implement any high level -functionality themselves. +lightweight: It favors being efficient in terms of gas costs and code size +over providing a lot of advanced high-level functionality found in other implementations +like the `HashMap` type from the Rust `std` library. +Overall, the ink! `Mapping` will be solid choice for most contracts. `Lazy` allows smart contract developers fine grained manual control over the storage layout of individual types. Conceivably, it may be desirable to change certain aspects From 055a411f8ebbcda150d3912646dcac0819377148 Mon Sep 17 00:00:00 2001 From: xermicus Date: Sun, 29 Jan 2023 13:44:22 +0100 Subject: [PATCH 38/76] rework lazy overview Signed-off-by: xermicus --- .../version-4.0.0-alpha.1/datastructures/overview.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md index 7993834a72..fa92edebaa 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md @@ -16,8 +16,9 @@ over providing a lot of advanced high-level functionality found in other impleme like the `HashMap` type from the Rust `std` library. Overall, the ink! `Mapping` will be solid choice for most contracts. -`Lazy` allows smart contract developers fine grained manual control over the storage -layout of individual types. Conceivably, it may be desirable to change certain aspects -on how your contract deals with it's storage variables. You can find out more about this -in the section about the ink! +`Lazy` is a wrapper type that can be used over any other storage compatible type. This allows smart contract developers fine grained manual control over the layout of the +contract storage. For example, it can be used to prevent the contract from eagerly +loading large storage fields. +Conceivably, it may be desirable to change certain aspects on how your contract deals with +it's storage variables. You can find out more about this in the section about the ink! [Storage Layout](https://use.ink/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout). From 5685d84602df01944611e834948c772e5ba3a3e7 Mon Sep 17 00:00:00 2001 From: xermicus Date: Sun, 29 Jan 2023 13:58:32 +0100 Subject: [PATCH 39/76] add mapping example for local variable Signed-off-by: xermicus --- .../datastructures/mapping.md | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md index 0fc32c729c..c3f50b2d14 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md @@ -80,7 +80,23 @@ The attentive reader may have noticed that accessing mapping values via the `Map method will result in an owned value (a local copy), as opposed to a direct reference into the storage. Changes to this value won't be reflected in the contracts storage "automatically". To avoid this common pitfall, the value must be inserted again at the same -key after it was modified. +key after it was modified. The `transfer` function from above example illustrates this: + +```rust +pub fn transfer(&mut self) { + let caller = self.env().caller(); + // `balance` is a local value and not a reference to the value on storage! + let balance = self.balances.get(caller).unwrap_or(0); + let endowment = self.env().transferred_value(); + // The following line of code would have no effect to the balance of the + // caller stored in contract storage: + // + // balance += endowment; + // + // Instead, we use the `insert` function to write it back like so: + self.balances.insert(caller, &(balance + endowment)); +} +``` ### Storage loading behaviour From 76bc8ae7d4e25ba5166091790cb352a31a76df15 Mon Sep 17 00:00:00 2001 From: xermicus Date: Sun, 29 Jan 2023 14:12:07 +0100 Subject: [PATCH 40/76] mapping storage access example Signed-off-by: xermicus --- .../datastructures/mapping.md | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md index c3f50b2d14..e1c6e33825 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md @@ -104,6 +104,19 @@ Each `Mapping` value lives under it's own storage key. Briefly, this means that lazily loaded in ink!. In other words, if your message only accesses a single key of a mapping, it will not load the whole mapping but only the value being accessed. +```rust +// This causes only a single storage access and the decoding of a single "MyValue" struct, +// no matter ho many elements there are inside the mapping. +let foo: MyValue = my_mapping.get(0)?; + +for n in 0..5 { + // This causes a storage access and a decoding operation for each loop iteration. + // It is not possible to "fetch" all key/value pairs directly at once. + let bar: MyValue = my_mapping.get(n)?; +} + +``` + Furthermore, this implies that it is not possible to iterate over the contents of a map. -Circumventing this feature by storing populated keys inside an `ink_prelude::Vec` is not advisable -as this might result in very high gas costs. +Circumventing this restriction by storing populated keys inside an `ink_prelude::Vec` is not +advisable as this might result in very high gas costs. From 852607a750b565952eda8e48a44a8f5c3532ca4b Mon Sep 17 00:00:00 2001 From: xermicus Date: Sun, 29 Jan 2023 14:27:07 +0100 Subject: [PATCH 41/76] explain why iteration over mapping might be expensive Signed-off-by: xermicus --- .../version-4.0.0-alpha.1/datastructures/mapping.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md index e1c6e33825..4d4dfefac5 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md @@ -114,9 +114,10 @@ for n in 0..5 { // It is not possible to "fetch" all key/value pairs directly at once. let bar: MyValue = my_mapping.get(n)?; } - ``` -Furthermore, this implies that it is not possible to iterate over the contents of a map. -Circumventing this restriction by storing populated keys inside an `ink_prelude::Vec` is not -advisable as this might result in very high gas costs. +Furthermore, it follows that mapping values do not have a contiguos storage layout and it is +therefore not possible to iterate over the contents of a map. +Circumventing this restriction by storing populated keys inside an `ink_prelude::Vec` might +not always be advisable: As accessing a storage cell is relatively expensive, this might +result in very high gas costs for large mappings. From d1bf3a8f25b7a607fc16facc86e00c3208810f9c Mon Sep 17 00:00:00 2001 From: xermicus Date: Sun, 29 Jan 2023 14:32:24 +0100 Subject: [PATCH 42/76] small fixes Signed-off-by: xermicus --- .../datastructures/storage-layout.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index 2b0b095028..39746f6dd3 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -3,7 +3,8 @@ title: Storage Layout slug: /datastructures/storage-layout --- -Smart contract authors are given some flexibility in regards on how they want to organize the storage layout of their contracts. +Smart contract authors are given some flexibility in regards on how they want to organize +the storage layout of their contracts. Let's dive deeper into the concepts behind ink! storage to get a better understanding of some of its implications and limitations. @@ -17,14 +18,15 @@ to ink! by the contracts pallet: Storage data is always encoded with the -[`SCALE`](https://docs.substrate.io/reference/scale-codec/) codec. -The storage API operates by storing and loading entries into and from a single storages cells, where each storage cell is accessed under it's own dedicated storage key. To some +[`SCALE`](https://docs.substrate.io/reference/scale-codec/) codec. +The storage API operates by storing and loading entries into and from a single storages +cells, where each storage cell is accessed under it's own dedicated storage key. To some extent, the storage API works similar to a traditional key-value database. ## Packed vs Non-Packed layout Types that can be stored entirely under a single storage cell are considered -[`Packed`](https://paritytech.github.io/ink/ink/storage/traits/trait.Packed.html). +[`Packed`](https://docs.rs/ink_storage_traits/4.0.0-beta.1/ink_storage_traits/trait.Packed.html). By default, ink! tries to store all storage struct fields under a single storage cell. Consequentially, with a `Packed` storage layout, any message interacting with the contract storage will always need to operate on the entire contract storage struct. From 9832d476a453f81e789eab638f349de040600934 Mon Sep 17 00:00:00 2001 From: xermicus Date: Sun, 29 Jan 2023 17:30:01 +0100 Subject: [PATCH 43/76] more fixes Signed-off-by: xermicus --- .../datastructures/storage-layout.md | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index 39746f6dd3..d0e4cdb3ba 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -43,9 +43,10 @@ with that `Vec`, regardless whether they access it or not. This results in extra To solve this problem we need to turn our storage into a non-packed layout somehow. ## Eager Loading vs. Lazy Loading -With the [`Lazy`](https://paritytech.github.io/ink/ink/storage/struct.Lazy.html) primitive, ink! provides means of breaking the storage up into smaller pieces, which can be loaded -on demand. Wrapping any storage field inside a `Lazy` struct makes the storage +on demand, with the +[`Lazy`](https://paritytech.github.io/ink/ink/storage/struct.Lazy.html) primitive. +Wrapping any storage field inside a `Lazy` struct makes the storage struct in which that field appears also `Non-Packed`, preventing it from being eagerly loaded during arbitrary storage operations. @@ -95,10 +96,10 @@ mod mycontract { :::caution -`Vec`'s are always loaded in their entirety. This is because all elements of the `Vec` live under a -single storage key. Wrapping the `Vec` inside `Lazy`, like the provided example above does, -has no influence on its elements. If you are dealing with large or sparse arrays on -contract storage, consider using a `Mapping` instead. +`Vec`'s are always loaded in their entirety. This is because all elements of the `Vec` live +under a single storage key. Wrapping the `Vec` inside `Lazy`, like the provided example +above does, has no influence on its inner layout. If you are dealing with large or sparse +arrays on contract storage, consider using a `Mapping` instead. ::: @@ -106,7 +107,8 @@ contract storage, consider using a `Mapping` instead. By default, keys are calculated automatically for you, thanks to the [`AutoKey`](https://docs.rs/ink_storage_traits/4.0.0-beta.1/ink_storage_traits/struct.AutoKey.html) -primitive. They'll be generated at compile time and ruled out for conflicts. However, for `Non-Packed` types like `Lazy` or the `Mapping`, the +primitive. They'll be generated at compile time and ruled out for conflicts. +However, for `Non-Packed` types like `Lazy` or the `Mapping`, the [`ManualKey`](https://docs.rs/ink_storage_traits/4.0.0-beta.1/ink_storage_traits/struct.ManualKey.html) primitive allows manual control over the storage key of a field like so: From 54954a567658f01def3866977014ff8edb710319 Mon Sep 17 00:00:00 2001 From: xermicus Date: Mon, 30 Jan 2023 18:06:07 +0100 Subject: [PATCH 44/76] wip transparent hashing Signed-off-by: xermicus --- .../datastructures/storage-in-metadata.md | 116 +++++++++++++++++- 1 file changed, 114 insertions(+), 2 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md index b4624dc30f..7398b6a915 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md @@ -73,9 +73,9 @@ A `root_key` is meant to either be used to directly access a `Packed` storage fi or to serve as the base key for calculting the actual keys needed to access values in `Non-Packed` fields (such as Mappings). -## Storage key calculation for mappings +## Storage key calculation for the ink! `Mapping` -Root storage keys are always 4 bytes in size. However, the storage API of the contracts +Base storage keys are always 4 bytes in size. However, the storage API of the contracts pallet supports keys of arbitrary length. In order to reach a mapping value, the storage key of said value is calculated at runtime. @@ -89,3 +89,115 @@ S = B + scale::encode(K) In words, find the base (root) key of the mapping and concatenate it with the SCALE encoded key of the mapped value to obtain the actual storage key used to access the mapped value. + +## Accessing storage items outside the contract + +There are two ways to query for storage fields of smart contracts on-chain. This section +explains how to do that with [`polkadot-js`](https://polkadot.js.org/apps/). + +### With the `contractsApi` runtime call API + +The straight forward way to query a contracts storage is via a +[`runtime API`](https://polkadot.js.org/apps/#/runtime) call, using the `contractsApi` +endpoint provided by the contracts pallet. The endpoint provides a `getStorage` method, +which just expects a contract address and a storage key as arguments. + +For example, to access the root storage struct under the key `0x00000000` of a contrect, +just specify the contracts address and the storage key `0x00000000` as-is. The API call +will return the scale-encoded root storage struct of the contract. + +### With the `childState` RPC call API + +Under the hood, each contract gets it's own +[child trie](https://paritytech.github.io/substrate/master/frame_support/storage/child/index.html), where its storage items are actually stored. + +Additionally, the contracts pallet uses the +[`Blake2 128 Concat`](https://paritytech.github.io/substrate/master/frame_support/struct.Blake2_128Concat.html) +[`Transparent hashing algorithm`](https://docs.substrate.io/build/runtime-storage/#transparent-hashing-algorithms) +to calculate storage keys for any stored item inside the child trie. +You'll need to account for that as well. + +With that in mind, to directly access storage items of any on-chain contract using a +childState [`RPC call`](https://polkadot.js.org/apps/#/rpc), you'll need the following: +- The child trie ID of the contract, represented as a `PrefixedStorageKey` +- The hashed storage key of the storage field + +#### Finding the contracts child trie ID +The chield trie ID is the `Blake2_256` hash of the contracts instantiation nonce +concatenated to it's `AccountId`. You can find it in [`polkadot-js chainstate query interface`](https://polkadot.js.org/apps/#/chainstate), where you need to execute the +`contracts_contractInfoOf` state query. + +It can also be calculate manually according to the following code snippet. Note that you +need to know the instantiation nonce of the contract! + +```rust +use sp_core::crypto::Ss58Codec; +use parity_scale_codec::Encode; + +// Given our contract ID is 5DjcHxSfjAgCTSF9mp6wQBJWBgj9h8uh57c7TNx1mL5hdQp4 +let account: AccountId32 = + Ss58Codec::from_string("5DjcHxSfjAgCTSF9mp6wQBJWBgj9h8uh57c7TNx1mL5hdQp4").unwrap(); +// Given our instnatiation nonce was 1 +let nonce: u64 = 1; + +// The child trie ID can be calculated as follows: +let trie_id = (&account, nonce).using_encoded(Blake2_256::hash); +``` + +#### Calcualte the `PrefixedStorageKey` from the child trie ID +A [`PrefixedStorageKey`](https://docs.rs/sp-storage/10.0.0/sp_storage/struct.PrefixedStorageKey.html) +can be construct using the +[`ChildInfo`](https://docs.rs/sp-storage/10.0.0/sp_storage/enum.ChildInfo.html) primitive as follows: + +```rust +use sp_core::storage::ChildInfo; +let prefixed_storage_key = ChildInfo::new_default(&trie_id).into_prefixed_storage_key(); +``` + +#### Calculate the storage key using transparent hashing +Finally, we need the hashed storage key of the storage item we are wanting to access. +The algorithm is simple: `Blake2_128` hash the storage key and then concatenate the unhashed +key to the hash. Given you want to access the storage item under the `0x00000000`, +it will look like this in code: + +```rust +use frame_support::Blake2_128Concat; + +// The base key is 0x00000000 +let storage_key = Blake2_128Concat::hash(&[0, 0, 0, 0]); +``` + +#### A full example + +```rust +use frame_support::{sp_runtime::AccountId32, Blake2_128Concat, Blake2_256, StorageHasher}; +use parity_scale_codec::Encode; +use sp_core::{crypto::Ss58Codec, storage::ChildInfo}; +use std::ops::Deref; + +fn main() { + // Find the child storage trie ID + let account_id = "5DjcHxSfjAgCTSF9mp6wQBJWBgj9h8uh57c7TNx1mL5hdQp4"; + let account: AccountId32 = Ss58Codec::from_string(account_id).unwrap(); + let instantiation_nonce: u64 = 1; + let trie_id = (account, instantiation_nonce).using_encoded(Blake2_256::hash); + assert_eq!( + hex::encode(trie_id), + "2fa252b7f996d28fd5d8b11098c09e56295eaf564299c6974421aa5ed887803b" + ); + + // Calculate the PrefixedStorageKey based on the trie ID + let prefixed_storage_key = ChildInfo::new_default(&trie_id).into_prefixed_storage_key(); + assert_eq!( + hex::encode(prefixed_storage_key.deref()), + "3a6368696c645f73746f726167653a64656661756c743a2fa252b7f996d28fd5d8b11098c09e56295eaf564299c6974421aa5ed887803b" + ); + + // Calculate the storage key using transparent hashing + let storage_key = Blake2_128Concat::hash(&[0, 0, 0, 0]); + assert_eq!( + hex::encode(&storage_key), + "11d2df4e979aa105cf552e9544ebd2b500000000" + ); +} +``` From e19ead947433a18779b9dc9e252a035a4b16ab31 Mon Sep 17 00:00:00 2001 From: xermicus Date: Mon, 30 Jan 2023 21:08:51 +0100 Subject: [PATCH 45/76] fix layout of storage in metadata section Signed-off-by: xermicus --- .../datastructures/storage-in-metadata.md | 60 +++++++++++-------- 1 file changed, 34 insertions(+), 26 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md index 7398b6a915..5efe55bdb0 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md @@ -73,14 +73,14 @@ A `root_key` is meant to either be used to directly access a `Packed` storage fi or to serve as the base key for calculting the actual keys needed to access values in `Non-Packed` fields (such as Mappings). -## Storage key calculation for the ink! `Mapping` +## Storage key calculation for ink! `Mapping` values Base storage keys are always 4 bytes in size. However, the storage API of the contracts pallet supports keys of arbitrary length. In order to reach a mapping value, the storage key of said value is calculated at runtime. -The formula to calculate a storage key `S` used to access a mapping value under the key `K` -for a mapping with base key `B` can be expressed as follows: +The formula to calculate the base storage key `S` used to access a mapping value under the +key `K` for a mapping with base key `B` can be expressed as follows: ``` S = B + scale::encode(K) @@ -90,12 +90,10 @@ In words, find the base (root) key of the mapping and concatenate it with the SCALE encoded key of the mapped value to obtain the actual storage key used to access the mapped value. -## Accessing storage items outside the contract +## Accessing storage items with the `contractsApi` runtime call API -There are two ways to query for storage fields of smart contracts on-chain. This section -explains how to do that with [`polkadot-js`](https://polkadot.js.org/apps/). - -### With the `contractsApi` runtime call API +There are two ways to query for storage fields of smart contracts from outside a contract. +Both methods are accessible via the [`polkadot-js`](https://polkadot.js.org/apps/) web UI. The straight forward way to query a contracts storage is via a [`runtime API`](https://polkadot.js.org/apps/#/runtime) call, using the `contractsApi` @@ -106,7 +104,7 @@ For example, to access the root storage struct under the key `0x00000000` of a c just specify the contracts address and the storage key `0x00000000` as-is. The API call will return the scale-encoded root storage struct of the contract. -### With the `childState` RPC call API +## Accessing storage items with the `childState` RPC call API Under the hood, each contract gets it's own [child trie](https://paritytech.github.io/substrate/master/frame_support/storage/child/index.html), where its storage items are actually stored. @@ -122,13 +120,15 @@ childState [`RPC call`](https://polkadot.js.org/apps/#/rpc), you'll need the fol - The child trie ID of the contract, represented as a `PrefixedStorageKey` - The hashed storage key of the storage field -#### Finding the contracts child trie ID -The chield trie ID is the `Blake2_256` hash of the contracts instantiation nonce +### Finding the contracts child trie ID + +The child trie ID is the `Blake2_256` hash of the contracts instantiation nonce concatenated to it's `AccountId`. You can find it in [`polkadot-js chainstate query interface`](https://polkadot.js.org/apps/#/chainstate), where you need to execute the `contracts_contractInfoOf` state query. -It can also be calculate manually according to the following code snippet. Note that you -need to know the instantiation nonce of the contract! +It can also be calculate manually according to the following code snippet. The +instantiation note of the contract must be still be know. You can get it using the +`contracts_nonce` chain state query in polkadot-js UI. ```rust use sp_core::crypto::Ss58Codec; @@ -144,7 +144,7 @@ let nonce: u64 = 1; let trie_id = (&account, nonce).using_encoded(Blake2_256::hash); ``` -#### Calcualte the `PrefixedStorageKey` from the child trie ID +### Calcualte the `PrefixedStorageKey` from the child trie ID A [`PrefixedStorageKey`](https://docs.rs/sp-storage/10.0.0/sp_storage/struct.PrefixedStorageKey.html) can be construct using the [`ChildInfo`](https://docs.rs/sp-storage/10.0.0/sp_storage/enum.ChildInfo.html) primitive as follows: @@ -154,8 +154,9 @@ use sp_core::storage::ChildInfo; let prefixed_storage_key = ChildInfo::new_default(&trie_id).into_prefixed_storage_key(); ``` -#### Calculate the storage key using transparent hashing -Finally, we need the hashed storage key of the storage item we are wanting to access. +### Calculate the storage key using transparent hashing + +Finally, we calculate the hashed storage key of the storage item we are wanting to access. The algorithm is simple: `Blake2_128` hash the storage key and then concatenate the unhashed key to the hash. Given you want to access the storage item under the `0x00000000`, it will look like this in code: @@ -167,7 +168,18 @@ use frame_support::Blake2_128Concat; let storage_key = Blake2_128Concat::hash(&[0, 0, 0, 0]); ``` -#### A full example +### A full example + +Let's recap the last few paragraphs into a full example. Given: + +* A contract at address `5DjcHxSfjAgCTSF9mp6wQBJWBgj9h8uh57c7TNx1mL5hdQp4` +* With instantiation nonce of `1` +* The root storage struct is to be found at base key `0x00000000` + +The following rust program demonstrates how to calculate the `PrefixedStorageKey` of the +contracts child trie, as well as the hashed key for the storage struct, which can then be +used with the `chilstate` RPC endpoint function `getStorage` in polkadot-js to receive +the root storage struct of the contract: ```rust use frame_support::{sp_runtime::AccountId32, Blake2_128Concat, Blake2_256, StorageHasher}; @@ -179,7 +191,7 @@ fn main() { // Find the child storage trie ID let account_id = "5DjcHxSfjAgCTSF9mp6wQBJWBgj9h8uh57c7TNx1mL5hdQp4"; let account: AccountId32 = Ss58Codec::from_string(account_id).unwrap(); - let instantiation_nonce: u64 = 1; + let instantiation_nonce = 1u64; let trie_id = (account, instantiation_nonce).using_encoded(Blake2_256::hash); assert_eq!( hex::encode(trie_id), @@ -188,16 +200,12 @@ fn main() { // Calculate the PrefixedStorageKey based on the trie ID let prefixed_storage_key = ChildInfo::new_default(&trie_id).into_prefixed_storage_key(); - assert_eq!( - hex::encode(prefixed_storage_key.deref()), - "3a6368696c645f73746f726167653a64656661756c743a2fa252b7f996d28fd5d8b11098c09e56295eaf564299c6974421aa5ed887803b" - ); + println!("0x{}", hex::encode(prefixed_storage_key.deref())); + // 0x3a6368696c645f73746f726167653a64656661756c743a2fa252b7f996d28fd5d8b11098c09e56295eaf564299c6974421aa5ed887803b // Calculate the storage key using transparent hashing let storage_key = Blake2_128Concat::hash(&[0, 0, 0, 0]); - assert_eq!( - hex::encode(&storage_key), - "11d2df4e979aa105cf552e9544ebd2b500000000" - ); + println!("0x{}", hex::encode(&storage_key)); + // 0x11d2df4e979aa105cf552e9544ebd2b500000000 } ``` From 69544a3acb4701a36aa0254a95a3c787d63b9a10 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Mon, 30 Jan 2023 21:12:58 +0100 Subject: [PATCH 46/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md Co-authored-by: Green Baneling --- versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md index 4d4dfefac5..15a06c1769 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md @@ -11,7 +11,7 @@ Here is an example of a mapping from a user to a `Balance`: #[ink(storage)] pub struct MyContract { /// Assign a balance to every account. - map: balances: ink::storage::Mapping + balances: ink::storage::Mapping, } ``` From a4edead6cf78be330113eee8be2b681e45ec22bc Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Mon, 30 Jan 2023 21:13:18 +0100 Subject: [PATCH 47/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md Co-authored-by: Green Baneling --- .../version-4.0.0-alpha.1/datastructures/storage-in-metadata.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md index 5efe55bdb0..a165be1174 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md @@ -4,7 +4,7 @@ slug: /datastructures/storage-in-metadata --- The storage layout of a contract is reflected inside the metadata. It allows third-party -tooling to work with contract stroage and can also help to better understand the storage +tooling to work with contract storage and can also help to better understand the storage layout of any given contract. Given a contract with the following storage: From 125b4e04936b59dbb6b67644d65f1265479ff85b Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Mon, 30 Jan 2023 21:13:32 +0100 Subject: [PATCH 48/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md Co-authored-by: Green Baneling --- versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md index fa92edebaa..b851ce4a4d 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md @@ -13,7 +13,7 @@ similar to traditional hash tables and comparable to the `mapping` type Solidity As a core ingredient to the ink! language, its main advantage is being simple and lightweight: It favors being efficient in terms of gas costs and code size over providing a lot of advanced high-level functionality found in other implementations -like the `HashMap` type from the Rust `std` library. +like the `ink::prelude::collections::HashMap` type from the standard Rust. Overall, the ink! `Mapping` will be solid choice for most contracts. `Lazy` is a wrapper type that can be used over any other storage compatible type. This allows smart contract developers fine grained manual control over the layout of the From de865b3de35ef3f7260bb39edeb9fb2e5a555897 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Mon, 30 Jan 2023 21:14:10 +0100 Subject: [PATCH 49/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md Co-authored-by: Green Baneling --- .../version-4.0.0-alpha.1/datastructures/overview.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md index b851ce4a4d..72afe00559 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md @@ -16,9 +16,11 @@ over providing a lot of advanced high-level functionality found in other impleme like the `ink::prelude::collections::HashMap` type from the standard Rust. Overall, the ink! `Mapping` will be solid choice for most contracts. -`Lazy` is a wrapper type that can be used over any other storage compatible type. This allows smart contract developers fine grained manual control over the layout of the -contract storage. For example, it can be used to prevent the contract from eagerly -loading large storage fields. +`Lazy` is a wrapper type that can be used over any other storage compatible type. +This allows smart contract developers fine grained manual control over the layout of +the contract storage by assigning a separate storage cell for the field. For example, +it can be used to prevent the contract from eagerly loading large storage fields +during each contract call. Conceivably, it may be desirable to change certain aspects on how your contract deals with it's storage variables. You can find out more about this in the section about the ink! [Storage Layout](https://use.ink/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout). From 475082c06ed6595784227929e6cd94e6fd6e9a21 Mon Sep 17 00:00:00 2001 From: xermicus Date: Mon, 30 Jan 2023 21:25:31 +0100 Subject: [PATCH 50/76] smol fix Signed-off-by: xermicus --- .../version-4.0.0-alpha.1/datastructures/overview.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md index 72afe00559..81d931a82a 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md @@ -12,9 +12,10 @@ and [`Lazy`](https://docs.rs/ink_storage/4.0.0-beta.1/ink_storage/struct.Lazy.ht similar to traditional hash tables and comparable to the `mapping` type Solidity offers. As a core ingredient to the ink! language, its main advantage is being simple and lightweight: It favors being efficient in terms of gas costs and code size -over providing a lot of advanced high-level functionality found in other implementations -like the `ink::prelude::collections::HashMap` type from the standard Rust. -Overall, the ink! `Mapping` will be solid choice for most contracts. +over providing a lot of high-level functionality found in other implementations +like the `HashMap` type from the Rust `std` library. +Overall, the ink! `Mapping` will be solid choice for most contracts. Moreover, smart +contracts developers can implement advanced features themselves. `Lazy` is a wrapper type that can be used over any other storage compatible type. This allows smart contract developers fine grained manual control over the layout of From 0c5904453b5b457b003aaa58b973cf75bcd494a8 Mon Sep 17 00:00:00 2001 From: xermicus Date: Mon, 30 Jan 2023 21:28:11 +0100 Subject: [PATCH 51/76] explain mapping loading behavior first Signed-off-by: xermicus --- .../datastructures/mapping.md | 48 +++++++++---------- 1 file changed, 24 insertions(+), 24 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md index 15a06c1769..9b1c6f20d9 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md @@ -74,30 +74,6 @@ mod mycontract { ## Considerations when using the `Mapping` type -### Updating values - -The attentive reader may have noticed that accessing mapping values via the `Mapping::get()` -method will result in an owned value (a local copy), as opposed to a direct reference -into the storage. Changes to this value won't be reflected in the contracts storage -"automatically". To avoid this common pitfall, the value must be inserted again at the same -key after it was modified. The `transfer` function from above example illustrates this: - -```rust -pub fn transfer(&mut self) { - let caller = self.env().caller(); - // `balance` is a local value and not a reference to the value on storage! - let balance = self.balances.get(caller).unwrap_or(0); - let endowment = self.env().transferred_value(); - // The following line of code would have no effect to the balance of the - // caller stored in contract storage: - // - // balance += endowment; - // - // Instead, we use the `insert` function to write it back like so: - self.balances.insert(caller, &(balance + endowment)); -} -``` - ### Storage loading behaviour Each `Mapping` value lives under it's own storage key. Briefly, this means that `Mapping`s are @@ -121,3 +97,27 @@ therefore not possible to iterate over the contents of a map. Circumventing this restriction by storing populated keys inside an `ink_prelude::Vec` might not always be advisable: As accessing a storage cell is relatively expensive, this might result in very high gas costs for large mappings. + +### Updating values + +The attentive reader may have noticed that accessing mapping values via the `Mapping::get()` +method will result in an owned value (a local copy), as opposed to a direct reference +into the storage. Changes to this value won't be reflected in the contracts storage +"automatically". To avoid this common pitfall, the value must be inserted again at the same +key after it was modified. The `transfer` function from above example illustrates this: + +```rust +pub fn transfer(&mut self) { + let caller = self.env().caller(); + // `balance` is a local value and not a reference to the value on storage! + let balance = self.balances.get(caller).unwrap_or(0); + let endowment = self.env().transferred_value(); + // The following line of code would have no effect to the balance of the + // caller stored in contract storage: + // + // balance += endowment; + // + // Instead, we use the `insert` function to write it back like so: + self.balances.insert(caller, &(balance + endowment)); +} +``` From 9448a2efc1c8b166fdf6a6ecde61eb60bc9a5538 Mon Sep 17 00:00:00 2001 From: xermicus Date: Mon, 30 Jan 2023 21:34:27 +0100 Subject: [PATCH 52/76] mention other datastructures under collections prelude Signed-off-by: xermicus --- .../version-4.0.0-alpha.1/datastructures/mapping.md | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md index 9b1c6f20d9..30807275ed 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md @@ -76,8 +76,8 @@ mod mycontract { ### Storage loading behaviour -Each `Mapping` value lives under it's own storage key. Briefly, this means that `Mapping`s are -lazily loaded in ink!. In other words, if your message only accesses a single key of a +Each `Mapping` value lives under it's own storage key. Briefly, this means that `Mapping`s +are lazily loaded in ink!. In other words, if your message only accesses a single key of a mapping, it will not load the whole mapping but only the value being accessed. ```rust @@ -98,6 +98,14 @@ Circumventing this restriction by storing populated keys inside an `ink_prelude: not always be advisable: As accessing a storage cell is relatively expensive, this might result in very high gas costs for large mappings. +:::note + +There are many additional datastructures accessible under `ink::prelude::collections`, such +such as `HashMap` or `BTreeMap` (to name a few). Note that these datastructures all exhibit +`Packed` storage loading behavior, as opposed to the ink! `Mapping`! + +::: + ### Updating values The attentive reader may have noticed that accessing mapping values via the `Mapping::get()` From 7c0826f2e723560f72a7a7bfbd5b632c49f825e5 Mon Sep 17 00:00:00 2001 From: xermicus Date: Mon, 30 Jan 2023 21:41:50 +0100 Subject: [PATCH 53/76] mention that the ink mapping can store a lot of values Signed-off-by: xermicus --- .../datastructures/mapping.md | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md index 30807275ed..4a2e16b296 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md @@ -74,6 +74,16 @@ mod mycontract { ## Considerations when using the `Mapping` type +One of the main purposes of the ink! `Mapping` is to allow storing a lot of values. + +:::note + +There are many additional datastructures accessible under `ink::prelude::collections`, such +such as `HashMap` or `BTreeMap` (to name a few). Note that these datastructures all exhibit +`Packed` storage loading behavior, as opposed to the ink! `Mapping`! + +::: + ### Storage loading behaviour Each `Mapping` value lives under it's own storage key. Briefly, this means that `Mapping`s @@ -98,13 +108,6 @@ Circumventing this restriction by storing populated keys inside an `ink_prelude: not always be advisable: As accessing a storage cell is relatively expensive, this might result in very high gas costs for large mappings. -:::note - -There are many additional datastructures accessible under `ink::prelude::collections`, such -such as `HashMap` or `BTreeMap` (to name a few). Note that these datastructures all exhibit -`Packed` storage loading behavior, as opposed to the ink! `Mapping`! - -::: ### Updating values From ad310b3edb66855580b3cdd1b2b76840c117cdae Mon Sep 17 00:00:00 2001 From: xermicus Date: Mon, 30 Jan 2023 21:47:13 +0100 Subject: [PATCH 54/76] mention pitfall of the contract trapping when decoding values too large Signed-off-by: xermicus --- .../datastructures/storage-layout.md | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index d0e4cdb3ba..64139fcf49 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -42,8 +42,19 @@ In that scenario, each and every contract message bears runtime overhead by deal with that `Vec`, regardless whether they access it or not. This results in extra gas costs. To solve this problem we need to turn our storage into a non-packed layout somehow. +:::caution + +If any type exhibiting `Packed` layout gets large enough (an ever growing `Vec` migth be +a prime candidate for this), it will break your contract. This is because for encoding and +decoding storage items, there is a buffer with only limited capacity (around 16KB in the +default configuration) available. This means any contract trying to decode more than that +will trap! If you are unsure about the potential size a datastructure might get, consider +using an ink! `Mapping`, which can store an arbitrary number of elements, instead. + +::: + ## Eager Loading vs. Lazy Loading -ink! provides means of breaking the storage up into smaller pieces, which can be loaded +ink! provides means of breaking the storage up into ismaller pieces, which can be loaded on demand, with the [`Lazy`](https://paritytech.github.io/ink/ink/storage/struct.Lazy.html) primitive. Wrapping any storage field inside a `Lazy` struct makes the storage From 213762dcf86ffc0629bc062f36f9fb5b75cec73e Mon Sep 17 00:00:00 2001 From: xermicus Date: Mon, 30 Jan 2023 22:09:26 +0100 Subject: [PATCH 55/76] refer to ink::prelude::vec::Vec Signed-off-by: xermicus --- .../datastructures/storage-layout.md | 33 ++++++++++--------- 1 file changed, 18 insertions(+), 15 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index 64139fcf49..cf1cb4a555 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -36,20 +36,22 @@ tiny fields, pulling everything from the storage inside every message is not problematic. It may even be advantegous - especially if we expect most messages to interact with most of the storage fields. -On the other hand, this can get problematic if we're storing a large `Vec` in the -contract storage but provide messages that do not need to read and write from this `Vec`. -In that scenario, each and every contract message bears runtime overhead by dealing -with that `Vec`, regardless whether they access it or not. This results in extra gas costs. -To solve this problem we need to turn our storage into a non-packed layout somehow. +On the other hand, this can get problematic if we're storing a large `ink::prelude::vec::Vec` +in the contract storage but provide messages that do not need to read and write from this +`ink::prelude::vec::Vec`. In that scenario, each and every contract message bears runtime +overhead by dealing with that `ink::prelude::vec::Vec`, regardless whether they access it or +not. This results in extra gas costs. To solve this problem we need to turn our storage +into a non-packed layout somehow. :::caution -If any type exhibiting `Packed` layout gets large enough (an ever growing `Vec` migth be -a prime candidate for this), it will break your contract. This is because for encoding and -decoding storage items, there is a buffer with only limited capacity (around 16KB in the -default configuration) available. This means any contract trying to decode more than that -will trap! If you are unsure about the potential size a datastructure might get, consider -using an ink! `Mapping`, which can store an arbitrary number of elements, instead. +If any type exhibiting `Packed` layout gets large enough (an ever growing +`ink::prelude::vec::Vec` migth be a prime candidate for this), it will break your contract. +This is because for encoding and decoding storage items, there is a buffer with only limited +capacity (around 16KB in the default configuration) available. This means any contract +trying to decode more than that will trap! If you are unsure about the potential size a +datastructure might get, consider using an ink! `Mapping`, which can store an arbitrary +number of elements, instead. ::: @@ -107,10 +109,11 @@ mod mycontract { :::caution -`Vec`'s are always loaded in their entirety. This is because all elements of the `Vec` live -under a single storage key. Wrapping the `Vec` inside `Lazy`, like the provided example -above does, has no influence on its inner layout. If you are dealing with large or sparse -arrays on contract storage, consider using a `Mapping` instead. +`ink::prelude::vec::Vec`'s are always loaded in their entirety. This is because all elements +of the `ink::prelude::vec::Vec` live under a single storage key. Wrapping the +`ink::prelude::vec::Vec` inside `Lazy`, like the +provided example above does, has no influence on its inner layout. If you are dealing with +large or sparse arrays on contract storage, consider using a `Mapping` instead. ::: From c61c021d4feb10b22bcaba43e2f98f3e17e7ed68 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Tue, 31 Jan 2023 23:45:01 +0100 Subject: [PATCH 56/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Michael Müller --- versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md index 6b29ec520c..c28f267518 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md @@ -4,7 +4,7 @@ slug: /datastructures/custom-datastructure --- While the `ink_storage` crate provides useful utilities and data structures to organize and -manipulate the contract's storagem, contract authors are not limited by its capabilities. +manipulate the contract's storage, contract authors are not limited by its capabilities. ## Using custom types on storage Any custom type wanting to be compatible with ink! storage must implement the From 9b07b1fcc127ea9f50a7a2b84031fb3555cde5d4 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Tue, 31 Jan 2023 23:46:15 +0100 Subject: [PATCH 57/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Michael Müller --- .../version-4.0.0-alpha.1/datastructures/storage-layout.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index cf1cb4a555..d44d1088cb 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -122,7 +122,7 @@ large or sparse arrays on contract storage, consider using a `Mapping` instead. By default, keys are calculated automatically for you, thanks to the [`AutoKey`](https://docs.rs/ink_storage_traits/4.0.0-beta.1/ink_storage_traits/struct.AutoKey.html) primitive. They'll be generated at compile time and ruled out for conflicts. -However, for `Non-Packed` types like `Lazy` or the `Mapping`, the +However, for non-`Packed` types like `Lazy` or the `Mapping`, the [`ManualKey`](https://docs.rs/ink_storage_traits/4.0.0-beta.1/ink_storage_traits/struct.ManualKey.html) primitive allows manual control over the storage key of a field like so: From f8fca53603beedffa01170319e7c4516d29296b7 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Tue, 31 Jan 2023 23:46:26 +0100 Subject: [PATCH 58/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Michael Müller --- .../version-4.0.0-alpha.1/datastructures/storage-layout.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index d44d1088cb..cb6ea751ec 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -80,7 +80,7 @@ mod mycontract { pub struct MyContract { tiny_value: Balance, /// This vector might get large and expensive to work with. - /// We want to enforce a `Non-Packed` storage layout. + /// We want to enforce a non-`Packed` storage layout. large_vec: Lazy>, } From 8f6fbe43029613df6ebd013ea39e3c16d1fc16f6 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Tue, 31 Jan 2023 23:46:36 +0100 Subject: [PATCH 59/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Michael Müller --- .../version-4.0.0-alpha.1/datastructures/storage-in-metadata.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md index a165be1174..d425d2dcd6 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md @@ -106,7 +106,7 @@ will return the scale-encoded root storage struct of the contract. ## Accessing storage items with the `childState` RPC call API -Under the hood, each contract gets it's own +Under the hood, each contract gets its own [child trie](https://paritytech.github.io/substrate/master/frame_support/storage/child/index.html), where its storage items are actually stored. Additionally, the contracts pallet uses the From 9464011cc1c4d32fe40b154b631b2a3de150c04b Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Tue, 31 Jan 2023 23:46:49 +0100 Subject: [PATCH 60/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Michael Müller --- .../version-4.0.0-alpha.1/datastructures/storage-layout.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index cb6ea751ec..b6b60d8655 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -61,7 +61,7 @@ on demand, with the [`Lazy`](https://paritytech.github.io/ink/ink/storage/struct.Lazy.html) primitive. Wrapping any storage field inside a `Lazy` struct makes the storage struct in which that field appears also -`Non-Packed`, preventing it from being eagerly loaded during arbitrary storage operations. +non-`Packed`, preventing it from being eagerly loaded during arbitrary storage operations. The following example demonstrates how we can solve the problem introduced in the above section. You'll notice that for the lazily loaded storage field, we now work with getters From 78a0e29da9670b807ffdb04def3c63ee0c9ab1a1 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Tue, 31 Jan 2023 23:47:02 +0100 Subject: [PATCH 61/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Michael Müller --- .../version-4.0.0-alpha.1/datastructures/storage-in-metadata.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md index d425d2dcd6..384641eab2 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md @@ -100,7 +100,7 @@ The straight forward way to query a contracts storage is via a endpoint provided by the contracts pallet. The endpoint provides a `getStorage` method, which just expects a contract address and a storage key as arguments. -For example, to access the root storage struct under the key `0x00000000` of a contrect, +For example, to access the root storage struct under the key `0x00000000` of a contract, just specify the contracts address and the storage key `0x00000000` as-is. The API call will return the scale-encoded root storage struct of the contract. From 4cdf3618830dd71279f4ab803145b9804372a059 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Tue, 31 Jan 2023 23:47:16 +0100 Subject: [PATCH 62/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Michael Müller --- .../version-4.0.0-alpha.1/datastructures/storage-layout.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index b6b60d8655..d358fd0331 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -155,6 +155,6 @@ or complex storage layouts into reasonably sized distinct storage cells. :::note -ink! `Mappings` are always `Non-Packed` and loaded lazily, one key-value pair at the time. +ink! `Mapping`s are always non-`Packed` and loaded lazily, one key-value pair at the time. ::: From a5998c44b502dcf88bc1c55f5d78a0c44316f345 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Tue, 31 Jan 2023 23:47:28 +0100 Subject: [PATCH 63/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Michael Müller --- .../version-4.0.0-alpha.1/datastructures/storage-in-metadata.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md index 384641eab2..ba65d66a91 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md @@ -71,7 +71,7 @@ in order to reach them you'd need fetch and decode the whole storage cell under A `root_key` is meant to either be used to directly access a `Packed` storage field or to serve as the base key for calculting the actual keys needed to access values in -`Non-Packed` fields (such as Mappings). +non-`Packed` fields (such as `Mapping`s). ## Storage key calculation for ink! `Mapping` values From 12f28847ab0a1f09a91001fa69fffbe50f6f7075 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Tue, 31 Jan 2023 23:47:44 +0100 Subject: [PATCH 64/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Michael Müller --- .../version-4.0.0-alpha.1/datastructures/storage-layout.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index d358fd0331..d70ea2d19e 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -20,7 +20,7 @@ to ink! by the contracts pallet: Storage data is always encoded with the [`SCALE`](https://docs.substrate.io/reference/scale-codec/) codec. The storage API operates by storing and loading entries into and from a single storages -cells, where each storage cell is accessed under it's own dedicated storage key. To some +cells, where each storage cell is accessed under its own dedicated storage key. To some extent, the storage API works similar to a traditional key-value database. ## Packed vs Non-Packed layout From f6fb88537092a65682430f7d467c62ab6daa1e06 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Tue, 31 Jan 2023 23:48:25 +0100 Subject: [PATCH 65/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Michael Müller --- .../version-4.0.0-alpha.1/datastructures/storage-in-metadata.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md index ba65d66a91..9381ed57c4 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md @@ -65,7 +65,7 @@ The storage will be reflected inside the metadata as like follows: } ``` -We observe that the storage is layed out as a tree, where tangible storage values end up +We observe that the storage layout is represented as a tree, where tangible storage values end up inside a `leaf`. Because of `Packed` encoding, leafs can share the same storage key, and in order to reach them you'd need fetch and decode the whole storage cell under this key. From 3447c9476232c49150bba50367645dbd7f78e907 Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Tue, 31 Jan 2023 23:48:40 +0100 Subject: [PATCH 66/76] Update versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Michael Müller --- .../version-4.0.0-alpha.1/datastructures/storage-in-metadata.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md index 9381ed57c4..add3b3df32 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md @@ -101,7 +101,7 @@ endpoint provided by the contracts pallet. The endpoint provides a `getStorage` which just expects a contract address and a storage key as arguments. For example, to access the root storage struct under the key `0x00000000` of a contract, -just specify the contracts address and the storage key `0x00000000` as-is. The API call +just specify the contract's address and the storage key `0x00000000` as-is. The API call will return the scale-encoded root storage struct of the contract. ## Accessing storage items with the `childState` RPC call API From 53acaeae34517d9b82cf39aae81d9743ce18767e Mon Sep 17 00:00:00 2001 From: Cyrill Leutwiler Date: Wed, 1 Feb 2023 11:25:16 +0100 Subject: [PATCH 67/76] Apply suggestions from code review Co-authored-by: Hernando Castano --- .../datastructures/custom.md | 25 ++++++++++--------- .../datastructures/storage-in-metadata.md | 14 +++++------ 2 files changed, 20 insertions(+), 19 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md index c28f267518..5c22f46596 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md @@ -3,8 +3,9 @@ title: Custom Data Structures slug: /datastructures/custom-datastructure --- -While the `ink_storage` crate provides useful utilities and data structures to organize and -manipulate the contract's storage, contract authors are not limited by its capabilities. +The `ink_storage` crate provides useful utilities and data structures to organize and +manipulate the contract's storage. However, contract authors should know that they can +also create their own custom data structures. ## Using custom types on storage Any custom type wanting to be compatible with ink! storage must implement the @@ -13,14 +14,14 @@ trait, so it can be SCALE [`encoded`](https://docs.rs/parity-scale-codec/3.2.2/parity_scale_codec/trait.Encode.html) and [`decoded`](https://docs.rs/parity-scale-codec/3.2.2/parity_scale_codec/trait.Decode.html). -Additionaly, the traits +Additionally, the traits [`StorageLayout`](https://docs.rs/ink_storage/latest/ink_storage/traits/trait.StorageLayout.html) and [`TypeInfo`](https://docs.rs/scale-info/2.3.1/scale_info/trait.TypeInfo.html) are required as well. But don't worry, usually these traits can just be derived: ```rust -/// A custom type on our contract storage -#[derive(scale::Decode, scale::Encode, Debug)] +/// A custom type that we can use in our contract storage +#[derive(scale::Decode, scale::Encode)] #[cfg_attr( feature = "std", derive(scale_info::TypeInfo, ink::storage::traits::StorageLayout) @@ -39,14 +40,14 @@ Even better: there is a macro `#[ink::storage_item]`, which derives all necessar can be simplified further as follows: ```rust -/// A custom type on our contract storage +/// A custom type that we can use in our contract storage #[ink::storage_item] pub struct Inner { value: bool, } #[ink(storage)] -pub struct SparseArray { +pub struct Outer { inner: Inner, } ``` @@ -57,7 +58,7 @@ It is possible to use generic data types in your storage, as long as any generic satisfies the required storage trait bounds. In fact, we already witnessed this in the previous sections about the `Mapping`. -Let's say you want a mapping where accessing a non-existant key should just return +Let's say you want a mapping where accessing a non-existent key should just return it's default value, akin to how mappings work in Solidity. Additionally, you want to know how many values there are in the mapping (its length). This could be implemented as a thin wrapper around the ink! `Mapping` as follows: @@ -74,13 +75,13 @@ pub struct DefaultMap { } impl DefaultMap { - /// Accessing non-existant keys will return the default value. + /// Accessing non-existent keys will return the default value. pub fn get(&self, key: &K) -> V { self.values.get(key).unwrap_or_default() } - /// Inserting into the map increases it's length by one. - pub fn set(&mut self, key: I, value: &E) + /// Inserting into the map increases its length by one. + pub fn set(&mut self, key: I, value: &U) where I: scale::EncodeLike, E: scale::EncodeLike + Storable, @@ -90,7 +91,7 @@ impl DefaultMap { } } - /// Removing a value from the map decreases it's length by one. + /// Removing a value from the map decreases its length by one. pub fn remove(&mut self, key: &K) { if self.values.take(key).is_some() { self.length -= 1 diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md index add3b3df32..338936b527 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md @@ -14,7 +14,7 @@ Given a contract with the following storage: pub struct MyContract { balance: Balance, block: BlockNumber, - something: Lazy, + lazy: Lazy, } ``` @@ -55,7 +55,7 @@ The storage will be reflected inside the metadata as like follows: "root_key": "0xb1f4904e" } }, - "name": "something" + "name": "lazy" } ], "name": "MyContract" @@ -70,7 +70,7 @@ inside a `leaf`. Because of `Packed` encoding, leafs can share the same storage in order to reach them you'd need fetch and decode the whole storage cell under this key. A `root_key` is meant to either be used to directly access a `Packed` storage field -or to serve as the base key for calculting the actual keys needed to access values in +or to serve as the base key for calculating the actual keys needed to access values in non-`Packed` fields (such as `Mapping`s). ## Storage key calculation for ink! `Mapping` values @@ -127,7 +127,7 @@ concatenated to it's `AccountId`. You can find it in [`polkadot-js chainstate qu `contracts_contractInfoOf` state query. It can also be calculate manually according to the following code snippet. The -instantiation note of the contract must be still be know. You can get it using the +instantiation note of the contract must be still be known. You can get it using the `contracts_nonce` chain state query in polkadot-js UI. ```rust @@ -137,14 +137,14 @@ use parity_scale_codec::Encode; // Given our contract ID is 5DjcHxSfjAgCTSF9mp6wQBJWBgj9h8uh57c7TNx1mL5hdQp4 let account: AccountId32 = Ss58Codec::from_string("5DjcHxSfjAgCTSF9mp6wQBJWBgj9h8uh57c7TNx1mL5hdQp4").unwrap(); -// Given our instnatiation nonce was 1 +// Given our instantiation nonce was 1 let nonce: u64 = 1; // The child trie ID can be calculated as follows: let trie_id = (&account, nonce).using_encoded(Blake2_256::hash); ``` -### Calcualte the `PrefixedStorageKey` from the child trie ID +### Calculate the `PrefixedStorageKey` from the child trie ID A [`PrefixedStorageKey`](https://docs.rs/sp-storage/10.0.0/sp_storage/struct.PrefixedStorageKey.html) can be construct using the [`ChildInfo`](https://docs.rs/sp-storage/10.0.0/sp_storage/enum.ChildInfo.html) primitive as follows: @@ -176,7 +176,7 @@ Let's recap the last few paragraphs into a full example. Given: * With instantiation nonce of `1` * The root storage struct is to be found at base key `0x00000000` -The following rust program demonstrates how to calculate the `PrefixedStorageKey` of the +The following Rust program demonstrates how to calculate the `PrefixedStorageKey` of the contracts child trie, as well as the hashed key for the storage struct, which can then be used with the `chilstate` RPC endpoint function `getStorage` in polkadot-js to receive the root storage struct of the contract: From e02fab33fe6bea331e3e6224278aaad6aa4bb555 Mon Sep 17 00:00:00 2001 From: xermicus Date: Wed, 1 Feb 2023 12:58:59 +0100 Subject: [PATCH 68/76] impl nandos comments Signed-off-by: xermicus --- .../datastructures/custom.md | 9 ++++++--- .../datastructures/storage-in-metadata.md | 15 +++++++++------ .../datastructures/storage-layout.md | 11 +++++------ 3 files changed, 20 insertions(+), 15 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md index 5c22f46596..e7ee6b792c 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md @@ -36,7 +36,8 @@ pub struct MyContractStorage { } ``` -Even better: there is a macro `#[ink::storage_item]`, which derives all necessary traits for you. If there is no need to implement any special behaviour, the above code example +Even better: there is a macro `#[ink::storage_item]`, which derives all necessary traits for +you. If there is no need to implement any special behaviour, the above code example can be simplified further as follows: ```rust @@ -56,7 +57,8 @@ pub struct Outer { It is possible to use generic data types in your storage, as long as any generic type satisfies the required storage trait bounds. In fact, we already witnessed this in the -previous sections about the `Mapping`. +previous sections about the +[`Mapping`](https://docs.rs/ink_storage/4.0.0-beta.1/ink_storage/struct.Mapping.html). Let's say you want a mapping where accessing a non-existent key should just return it's default value, akin to how mappings work in Solidity. Additionally, you want to know @@ -113,7 +115,8 @@ pub struct MyContract { :::caution -Generic data types may substantially increase your contracts overall code size. +Generic data types may substantially increase your contracts overall code size, making it +more costly to store on-chain. The reason for this is [Rust's monomorphization](https://rustwasm.github.io/twiggy/concepts/generic-functions-and-monomorphization.html). ::: diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md index 338936b527..10c09d421f 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md @@ -65,13 +65,17 @@ The storage will be reflected inside the metadata as like follows: } ``` -We observe that the storage layout is represented as a tree, where tangible storage values end up -inside a `leaf`. Because of `Packed` encoding, leafs can share the same storage key, and +We observe that the storage layout is represented as a tree, where tangible storage values +end up inside a `leaf`. Because of +[`Packed`](https://docs.rs/ink_storage_traits/4.0.0-beta.1/ink_storage_traits/trait.Packed.html) +encoding, leafs can share the same storage key, and in order to reach them you'd need fetch and decode the whole storage cell under this key. A `root_key` is meant to either be used to directly access a `Packed` storage field or to serve as the base key for calculating the actual keys needed to access values in non-`Packed` fields (such as `Mapping`s). +Layouts under a root key can contain `leaf`s directly, but also `struct`, `enum` or `array` +layout, which are used ## Storage key calculation for ink! `Mapping` values @@ -117,7 +121,7 @@ You'll need to account for that as well. With that in mind, to directly access storage items of any on-chain contract using a childState [`RPC call`](https://polkadot.js.org/apps/#/rpc), you'll need the following: -- The child trie ID of the contract, represented as a `PrefixedStorageKey` +- The child trie ID of the contract, represented as a [`PrefixedStorageKey`](https://docs.rs/sp-storage/10.0.0/sp_storage/struct.PrefixedStorageKey.html) - The hashed storage key of the storage field ### Finding the contracts child trie ID @@ -145,9 +149,8 @@ let trie_id = (&account, nonce).using_encoded(Blake2_256::hash); ``` ### Calculate the `PrefixedStorageKey` from the child trie ID -A [`PrefixedStorageKey`](https://docs.rs/sp-storage/10.0.0/sp_storage/struct.PrefixedStorageKey.html) -can be construct using the -[`ChildInfo`](https://docs.rs/sp-storage/10.0.0/sp_storage/enum.ChildInfo.html) primitive as follows: +A `PrefixedStorageKey` based on the child trie ID can be constructed using the `ChildInfo` +primitive as follows: ```rust use sp_core::storage::ChildInfo; diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index d70ea2d19e..e92a886289 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -38,15 +38,14 @@ interact with most of the storage fields. On the other hand, this can get problematic if we're storing a large `ink::prelude::vec::Vec` in the contract storage but provide messages that do not need to read and write from this -`ink::prelude::vec::Vec`. In that scenario, each and every contract message bears runtime -overhead by dealing with that `ink::prelude::vec::Vec`, regardless whether they access it or -not. This results in extra gas costs. To solve this problem we need to turn our storage -into a non-packed layout somehow. +`Vec`. In that scenario, each and every contract message bears runtime overhead by dealing +with that `Vec`, regardless whether they access it or not. This results in extra gas costs. +To solve this problem we need to turn our storage into a non-packed layout somehow. :::caution -If any type exhibiting `Packed` layout gets large enough (an ever growing -`ink::prelude::vec::Vec` migth be a prime candidate for this), it will break your contract. +If any type exhibiting `Packed` layout gets large enough (an ever growing `Vec` migth be a +prime candidate for this), it will break your contract. This is because for encoding and decoding storage items, there is a buffer with only limited capacity (around 16KB in the default configuration) available. This means any contract trying to decode more than that will trap! If you are unsure about the potential size a From ac6d467dbc653a85c5ef55e947bc3d756068cdd5 Mon Sep 17 00:00:00 2001 From: xermicus Date: Wed, 1 Feb 2023 13:09:42 +0100 Subject: [PATCH 69/76] impl greens comments Signed-off-by: xermicus --- .../version-4.0.0-alpha.1/datastructures/custom.md | 3 +++ .../version-4.0.0-alpha.1/datastructures/mapping.md | 5 +---- .../version-4.0.0-alpha.1/datastructures/overview.md | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md index e7ee6b792c..c1299cbc6e 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md @@ -53,6 +53,9 @@ pub struct Outer { } ``` +Naturally, you can as well implement any required trait manually. Please directly refer to +the relevant trait documentations for more information. + ## Generic storage fields It is possible to use generic data types in your storage, as long as any generic type diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md index 4a2e16b296..1f52fcb41a 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md @@ -103,10 +103,7 @@ for n in 0..5 { ``` Furthermore, it follows that mapping values do not have a contiguos storage layout and it is -therefore not possible to iterate over the contents of a map. -Circumventing this restriction by storing populated keys inside an `ink_prelude::Vec` might -not always be advisable: As accessing a storage cell is relatively expensive, this might -result in very high gas costs for large mappings. +not possible to iterate over the contents of a map. ### Updating values diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md index 81d931a82a..82c220ed32 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md @@ -13,7 +13,7 @@ similar to traditional hash tables and comparable to the `mapping` type Solidity As a core ingredient to the ink! language, its main advantage is being simple and lightweight: It favors being efficient in terms of gas costs and code size over providing a lot of high-level functionality found in other implementations -like the `HashMap` type from the Rust `std` library. +like the `ink::prelude::collections::HashMap` type. Overall, the ink! `Mapping` will be solid choice for most contracts. Moreover, smart contracts developers can implement advanced features themselves. From 3418267238d9048408ec9d5c7a65684817c9f636 Mon Sep 17 00:00:00 2001 From: xermicus Date: Wed, 1 Feb 2023 15:09:53 +0100 Subject: [PATCH 70/76] fix oopise Signed-off-by: xermicus --- .../version-4.0.0-alpha.1/datastructures/storage-in-metadata.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md index 10c09d421f..e02f857ca4 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md @@ -74,8 +74,6 @@ in order to reach them you'd need fetch and decode the whole storage cell under A `root_key` is meant to either be used to directly access a `Packed` storage field or to serve as the base key for calculating the actual keys needed to access values in non-`Packed` fields (such as `Mapping`s). -Layouts under a root key can contain `leaf`s directly, but also `struct`, `enum` or `array` -layout, which are used ## Storage key calculation for ink! `Mapping` values From 8e0891d2e1cdde7a40a93e177f8a418ce7f5a189 Mon Sep 17 00:00:00 2001 From: xermicus Date: Wed, 1 Feb 2023 16:29:08 +0100 Subject: [PATCH 71/76] link ink::storage_item macro docs Signed-off-by: xermicus --- .../version-4.0.0-alpha.1/datastructures/custom.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md index c1299cbc6e..6e26b8c9e4 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md @@ -36,9 +36,10 @@ pub struct MyContractStorage { } ``` -Even better: there is a macro `#[ink::storage_item]`, which derives all necessary traits for -you. If there is no need to implement any special behaviour, the above code example -can be simplified further as follows: +Even better: there is a macro +[`#[ink::storage_item`](https://docs.rs/ink_macro/4.0.0-beta.1/ink_macro/attr.storage_item.html), +which derives all necessary traits for you. If there is no need to implement any special +behaviour, the above code example can be simplified further as follows: ```rust /// A custom type that we can use in our contract storage From 69421841f92fe2229c1bfbdc23661e9ca75b1198 Mon Sep 17 00:00:00 2001 From: Hernando Castano Date: Wed, 1 Feb 2023 18:05:30 -0800 Subject: [PATCH 72/76] Fix some typos --- .../datastructures/storage-in-metadata.md | 5 +++-- .../datastructures/storage-layout.md | 12 ++++++------ 2 files changed, 9 insertions(+), 8 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md index e02f857ca4..227b20ef1b 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md @@ -125,8 +125,9 @@ childState [`RPC call`](https://polkadot.js.org/apps/#/rpc), you'll need the fol ### Finding the contracts child trie ID The child trie ID is the `Blake2_256` hash of the contracts instantiation nonce -concatenated to it's `AccountId`. You can find it in [`polkadot-js chainstate query interface`](https://polkadot.js.org/apps/#/chainstate), where you need to execute the -`contracts_contractInfoOf` state query. +concatenated to it's `AccountId`. You can find it in +[`polkadot-js chainstate query interface`](https://polkadot.js.org/apps/#/chainstate), +where you need to execute the `contracts_contractInfoOf` state query. It can also be calculate manually according to the following code snippet. The instantiation note of the contract must be still be known. You can get it using the diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index e92a886289..16a16163ac 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -33,7 +33,7 @@ storage will always need to operate on the entire contract storage struct. For example, if we have a somewhat small contract storage struct consisting of only a few tiny fields, pulling everything from the storage inside every message is not -problematic. It may even be advantegous - especially if we expect most messages to +problematic. It may even be advantageous - especially if we expect most messages to interact with most of the storage fields. On the other hand, this can get problematic if we're storing a large `ink::prelude::vec::Vec` @@ -44,8 +44,8 @@ To solve this problem we need to turn our storage into a non-packed layout someh :::caution -If any type exhibiting `Packed` layout gets large enough (an ever growing `Vec` migth be a -prime candidate for this), it will break your contract. +If any type exhibiting `Packed` layout gets large enough (an ever growing `Vec` might be +a prime candidate for this), it will break your contract. This is because for encoding and decoding storage items, there is a buffer with only limited capacity (around 16KB in the default configuration) available. This means any contract trying to decode more than that will trap! If you are unsure about the potential size a @@ -55,7 +55,7 @@ number of elements, instead. ::: ## Eager Loading vs. Lazy Loading -ink! provides means of breaking the storage up into ismaller pieces, which can be loaded +ink! provides means of breaking the storage up into smaller pieces, which can be loaded on demand, with the [`Lazy`](https://paritytech.github.io/ink/ink/storage/struct.Lazy.html) primitive. Wrapping any storage field inside a `Lazy` struct makes the storage @@ -133,13 +133,13 @@ pub struct MyContract { } ``` -This may be advantegous: Your storage key will always stay the same, regardless of +This may be advantageous: Your storage key will always stay the same, regardless of the version of your contract or ink! itself (note that the key calculation algorithm may change with future ink! versions). :::tip -Using `ManualKey` instead of `AutoKey` might be especially desireable for upgradable +Using `ManualKey` instead of `AutoKey` might be especially desirable for upgradable contracts, as using `AutoKey` might result in a different storage key for the same field in a newer version of the contract. This may break your contract after an upgrade 😱! From 04fd49762c6e894518020ea742113c672324a526 Mon Sep 17 00:00:00 2001 From: Hernando Castano Date: Wed, 1 Feb 2023 18:33:26 -0800 Subject: [PATCH 73/76] Couple of small nits --- .../version-4.0.0-alpha.1/datastructures/custom.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md index 6e26b8c9e4..f127b05060 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md @@ -31,7 +31,7 @@ pub struct Inner { } #[ink(storage)] -pub struct MyContractStorage { +pub struct ContractStorage { inner: Inner, } ``` @@ -49,7 +49,7 @@ pub struct Inner { } #[ink(storage)] -pub struct Outer { +pub struct ContractStorage { inner: Inner, } ``` @@ -69,7 +69,6 @@ it's default value, akin to how mappings work in Solidity. Additionally, you wan how many values there are in the mapping (its length). This could be implemented as a thin wrapper around the ink! `Mapping` as follows: - ```rust /// Values for this map need to implement the `Default` trait. /// Naturally, they also must be compatible with contract storage. @@ -119,8 +118,9 @@ pub struct MyContract { :::caution -Generic data types may substantially increase your contracts overall code size, making it +Generic data types may substantially increase your contracts overall code size, making it more costly to store on-chain. + The reason for this is [Rust's monomorphization](https://rustwasm.github.io/twiggy/concepts/generic-functions-and-monomorphization.html). ::: From e22dab2266344bd1237719b596fd1fa631b11c14 Mon Sep 17 00:00:00 2001 From: Hernando Castano Date: Wed, 1 Feb 2023 18:36:29 -0800 Subject: [PATCH 74/76] Fix some typos --- .../version-4.0.0-alpha.1/datastructures/mapping.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md index 1f52fcb41a..e59a89c0d8 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md @@ -44,7 +44,7 @@ mod mycontract { Self { balances } } - /// Retreive the balance of the caller. + /// Retrieve the balance of the caller. #[ink(message)] pub fn get_balance(&self) -> Option { let caller = self.env().caller(); @@ -92,7 +92,7 @@ mapping, it will not load the whole mapping but only the value being accessed. ```rust // This causes only a single storage access and the decoding of a single "MyValue" struct, -// no matter ho many elements there are inside the mapping. +// no matter how many elements there are inside the mapping. let foo: MyValue = my_mapping.get(0)?; for n in 0..5 { @@ -102,7 +102,7 @@ for n in 0..5 { } ``` -Furthermore, it follows that mapping values do not have a contiguos storage layout and it is +Furthermore, it follows that mapping values do not have a contiguous storage layout and it is not possible to iterate over the contents of a map. From ce6bb9f21962fb6e97a21f207f72625c85beb223 Mon Sep 17 00:00:00 2001 From: Hernando Castano Date: Wed, 1 Feb 2023 18:40:17 -0800 Subject: [PATCH 75/76] Remove trailing whitespaces for entire PR --- .../datastructures/custom.md | 30 ++++---- .../datastructures/mapping.md | 28 ++++---- .../datastructures/overview.md | 24 +++---- .../datastructures/storage-in-metadata.md | 60 ++++++++-------- .../datastructures/storage-layout.md | 72 +++++++++---------- 5 files changed, 107 insertions(+), 107 deletions(-) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md index f127b05060..a355aa273c 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md @@ -8,13 +8,13 @@ manipulate the contract's storage. However, contract authors should know that th also create their own custom data structures. ## Using custom types on storage -Any custom type wanting to be compatible with ink! storage must implement the -[`Storable`](https://docs.rs/ink_storage_traits/4.0.0-beta/ink_storage_traits/trait.Storable.html) +Any custom type wanting to be compatible with ink! storage must implement the +[`Storable`](https://docs.rs/ink_storage_traits/4.0.0-beta/ink_storage_traits/trait.Storable.html) trait, so it can be SCALE [`encoded`](https://docs.rs/parity-scale-codec/3.2.2/parity_scale_codec/trait.Encode.html) -and +and [`decoded`](https://docs.rs/parity-scale-codec/3.2.2/parity_scale_codec/trait.Decode.html). -Additionally, the traits +Additionally, the traits [`StorageLayout`](https://docs.rs/ink_storage/latest/ink_storage/traits/trait.StorageLayout.html) and [`TypeInfo`](https://docs.rs/scale-info/2.3.1/scale_info/trait.TypeInfo.html) are required as well. But don't worry, usually these traits can just be derived: @@ -36,9 +36,9 @@ pub struct ContractStorage { } ``` -Even better: there is a macro -[`#[ink::storage_item`](https://docs.rs/ink_macro/4.0.0-beta.1/ink_macro/attr.storage_item.html), -which derives all necessary traits for you. If there is no need to implement any special +Even better: there is a macro +[`#[ink::storage_item`](https://docs.rs/ink_macro/4.0.0-beta.1/ink_macro/attr.storage_item.html), +which derives all necessary traits for you. If there is no need to implement any special behaviour, the above code example can be simplified further as follows: ```rust @@ -54,20 +54,20 @@ pub struct ContractStorage { } ``` -Naturally, you can as well implement any required trait manually. Please directly refer to +Naturally, you can as well implement any required trait manually. Please directly refer to the relevant trait documentations for more information. ## Generic storage fields -It is possible to use generic data types in your storage, as long as any generic type -satisfies the required storage trait bounds. In fact, we already witnessed this in the -previous sections about the +It is possible to use generic data types in your storage, as long as any generic type +satisfies the required storage trait bounds. In fact, we already witnessed this in the +previous sections about the [`Mapping`](https://docs.rs/ink_storage/4.0.0-beta.1/ink_storage/struct.Mapping.html). -Let's say you want a mapping where accessing a non-existent key should just return -it's default value, akin to how mappings work in Solidity. Additionally, you want to know -how many values there are in the mapping (its length). This could be implemented as a -thin wrapper around the ink! `Mapping` as follows: +Let's say you want a mapping where accessing a non-existent key should just return +it's default value, akin to how mappings work in Solidity. Additionally, you want to know +how many values there are in the mapping (its length). This could be implemented as a +thin wrapper around the ink! `Mapping` as follows: ```rust /// Values for this map need to implement the `Default` trait. diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md index e59a89c0d8..ceaf4043bf 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/mapping.md @@ -1,5 +1,5 @@ --- -title: Working with Mapping +title: Working with Mapping slug: /datastructures/mapping --- @@ -16,11 +16,11 @@ pub struct MyContract { ``` This means that for a given key, you can store a unique instance of a value type. In this -case, each "user" gets credited their own balance. +case, each "user" gets credited their own balance. ## Example: Using a `Mapping` -The following example contract utilizes a `Mapping` so that anyone can deposit and withdraw +The following example contract utilizes a `Mapping` so that anyone can deposit and withdraw balance for their own account: ```rust @@ -74,20 +74,20 @@ mod mycontract { ## Considerations when using the `Mapping` type -One of the main purposes of the ink! `Mapping` is to allow storing a lot of values. +One of the main purposes of the ink! `Mapping` is to allow storing a lot of values. :::note -There are many additional datastructures accessible under `ink::prelude::collections`, such -such as `HashMap` or `BTreeMap` (to name a few). Note that these datastructures all exhibit +There are many additional datastructures accessible under `ink::prelude::collections`, such +such as `HashMap` or `BTreeMap` (to name a few). Note that these datastructures all exhibit `Packed` storage loading behavior, as opposed to the ink! `Mapping`! ::: ### Storage loading behaviour -Each `Mapping` value lives under it's own storage key. Briefly, this means that `Mapping`s -are lazily loaded in ink!. In other words, if your message only accesses a single key of a +Each `Mapping` value lives under it's own storage key. Briefly, this means that `Mapping`s +are lazily loaded in ink!. In other words, if your message only accesses a single key of a mapping, it will not load the whole mapping but only the value being accessed. ```rust @@ -103,15 +103,15 @@ for n in 0..5 { ``` Furthermore, it follows that mapping values do not have a contiguous storage layout and it is -not possible to iterate over the contents of a map. +not possible to iterate over the contents of a map. ### Updating values -The attentive reader may have noticed that accessing mapping values via the `Mapping::get()` -method will result in an owned value (a local copy), as opposed to a direct reference -into the storage. Changes to this value won't be reflected in the contracts storage -"automatically". To avoid this common pitfall, the value must be inserted again at the same +The attentive reader may have noticed that accessing mapping values via the `Mapping::get()` +method will result in an owned value (a local copy), as opposed to a direct reference +into the storage. Changes to this value won't be reflected in the contracts storage +"automatically". To avoid this common pitfall, the value must be inserted again at the same key after it was modified. The `transfer` function from above example illustrates this: ```rust @@ -120,7 +120,7 @@ pub fn transfer(&mut self) { // `balance` is a local value and not a reference to the value on storage! let balance = self.balances.get(caller).unwrap_or(0); let endowment = self.env().transferred_value(); - // The following line of code would have no effect to the balance of the + // The following line of code would have no effect to the balance of the // caller stored in contract storage: // // balance += endowment; diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md index 82c220ed32..f92c4154a1 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md @@ -4,24 +4,24 @@ slug: /datastructures/overview --- The `ink_storage` crate acts as the standard storage library for ink! smart contracts. -At the moment it provides two primitives for interacting with storage, +At the moment it provides two primitives for interacting with storage, [`Mapping`](https://docs.rs/ink_storage/4.0.0-beta.1/ink_storage/struct.Mapping.html) and [`Lazy`](https://docs.rs/ink_storage/4.0.0-beta.1/ink_storage/struct.Lazy.html). -`Mapping` is a mapping of key-value pairs directly to the contract storage. It is very +`Mapping` is a mapping of key-value pairs directly to the contract storage. It is very similar to traditional hash tables and comparable to the `mapping` type Solidity offers. -As a core ingredient to the ink! language, its main advantage is being simple and -lightweight: It favors being efficient in terms of gas costs and code size -over providing a lot of high-level functionality found in other implementations +As a core ingredient to the ink! language, its main advantage is being simple and +lightweight: It favors being efficient in terms of gas costs and code size +over providing a lot of high-level functionality found in other implementations like the `ink::prelude::collections::HashMap` type. -Overall, the ink! `Mapping` will be solid choice for most contracts. Moreover, smart +Overall, the ink! `Mapping` will be solid choice for most contracts. Moreover, smart contracts developers can implement advanced features themselves. -`Lazy` is a wrapper type that can be used over any other storage compatible type. -This allows smart contract developers fine grained manual control over the layout of -the contract storage by assigning a separate storage cell for the field. For example, -it can be used to prevent the contract from eagerly loading large storage fields +`Lazy` is a wrapper type that can be used over any other storage compatible type. +This allows smart contract developers fine grained manual control over the layout of +the contract storage by assigning a separate storage cell for the field. For example, +it can be used to prevent the contract from eagerly loading large storage fields during each contract call. -Conceivably, it may be desirable to change certain aspects on how your contract deals with -it's storage variables. You can find out more about this in the section about the ink! +Conceivably, it may be desirable to change certain aspects on how your contract deals with +its storage variables. You can find out more about this in the section about the ink! [Storage Layout](https://use.ink/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout). diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md index 227b20ef1b..c15582d4c1 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md @@ -3,8 +3,8 @@ title: Metadata Format slug: /datastructures/storage-in-metadata --- -The storage layout of a contract is reflected inside the metadata. It allows third-party -tooling to work with contract storage and can also help to better understand the storage +The storage layout of a contract is reflected inside the metadata. It allows third-party +tooling to work with contract storage and can also help to better understand the storage layout of any given contract. Given a contract with the following storage: @@ -65,45 +65,45 @@ The storage will be reflected inside the metadata as like follows: } ``` -We observe that the storage layout is represented as a tree, where tangible storage values -end up inside a `leaf`. Because of -[`Packed`](https://docs.rs/ink_storage_traits/4.0.0-beta.1/ink_storage_traits/trait.Packed.html) -encoding, leafs can share the same storage key, and +We observe that the storage layout is represented as a tree, where tangible storage values +end up inside a `leaf`. Because of +[`Packed`](https://docs.rs/ink_storage_traits/4.0.0-beta.1/ink_storage_traits/trait.Packed.html) +encoding, leafs can share the same storage key, and in order to reach them you'd need fetch and decode the whole storage cell under this key. -A `root_key` is meant to either be used to directly access a `Packed` storage field -or to serve as the base key for calculating the actual keys needed to access values in +A `root_key` is meant to either be used to directly access a `Packed` storage field +or to serve as the base key for calculating the actual keys needed to access values in non-`Packed` fields (such as `Mapping`s). ## Storage key calculation for ink! `Mapping` values -Base storage keys are always 4 bytes in size. However, the storage API of the contracts -pallet supports keys of arbitrary length. In order to reach a mapping value, the storage +Base storage keys are always 4 bytes in size. However, the storage API of the contracts +pallet supports keys of arbitrary length. In order to reach a mapping value, the storage key of said value is calculated at runtime. -The formula to calculate the base storage key `S` used to access a mapping value under the +The formula to calculate the base storage key `S` used to access a mapping value under the key `K` for a mapping with base key `B` can be expressed as follows: ``` S = B + scale::encode(K) ``` - -In words, find the base (root) key of the mapping and concatenate it with the -SCALE encoded key of the mapped value to obtain the actual storage key used to + +In words, find the base (root) key of the mapping and concatenate it with the +SCALE encoded key of the mapped value to obtain the actual storage key used to access the mapped value. ## Accessing storage items with the `contractsApi` runtime call API -There are two ways to query for storage fields of smart contracts from outside a contract. +There are two ways to query for storage fields of smart contracts from outside a contract. Both methods are accessible via the [`polkadot-js`](https://polkadot.js.org/apps/) web UI. -The straight forward way to query a contracts storage is via a -[`runtime API`](https://polkadot.js.org/apps/#/runtime) call, using the `contractsApi` -endpoint provided by the contracts pallet. The endpoint provides a `getStorage` method, +The straight forward way to query a contracts storage is via a +[`runtime API`](https://polkadot.js.org/apps/#/runtime) call, using the `contractsApi` +endpoint provided by the contracts pallet. The endpoint provides a `getStorage` method, which just expects a contract address and a storage key as arguments. -For example, to access the root storage struct under the key `0x00000000` of a contract, -just specify the contract's address and the storage key `0x00000000` as-is. The API call +For example, to access the root storage struct under the key `0x00000000` of a contract, +just specify the contract's address and the storage key `0x00000000` as-is. The API call will return the scale-encoded root storage struct of the contract. ## Accessing storage items with the `childState` RPC call API @@ -112,12 +112,12 @@ Under the hood, each contract gets its own [child trie](https://paritytech.github.io/substrate/master/frame_support/storage/child/index.html), where its storage items are actually stored. Additionally, the contracts pallet uses the -[`Blake2 128 Concat`](https://paritytech.github.io/substrate/master/frame_support/struct.Blake2_128Concat.html) -[`Transparent hashing algorithm`](https://docs.substrate.io/build/runtime-storage/#transparent-hashing-algorithms) +[`Blake2 128 Concat`](https://paritytech.github.io/substrate/master/frame_support/struct.Blake2_128Concat.html) +[`Transparent hashing algorithm`](https://docs.substrate.io/build/runtime-storage/#transparent-hashing-algorithms) to calculate storage keys for any stored item inside the child trie. You'll need to account for that as well. -With that in mind, to directly access storage items of any on-chain contract using a +With that in mind, to directly access storage items of any on-chain contract using a childState [`RPC call`](https://polkadot.js.org/apps/#/rpc), you'll need the following: - The child trie ID of the contract, represented as a [`PrefixedStorageKey`](https://docs.rs/sp-storage/10.0.0/sp_storage/struct.PrefixedStorageKey.html) - The hashed storage key of the storage field @@ -129,8 +129,8 @@ concatenated to it's `AccountId`. You can find it in [`polkadot-js chainstate query interface`](https://polkadot.js.org/apps/#/chainstate), where you need to execute the `contracts_contractInfoOf` state query. -It can also be calculate manually according to the following code snippet. The -instantiation note of the contract must be still be known. You can get it using the +It can also be calculate manually according to the following code snippet. The +instantiation note of the contract must be still be known. You can get it using the `contracts_nonce` chain state query in polkadot-js UI. ```rust @@ -148,7 +148,7 @@ let trie_id = (&account, nonce).using_encoded(Blake2_256::hash); ``` ### Calculate the `PrefixedStorageKey` from the child trie ID -A `PrefixedStorageKey` based on the child trie ID can be constructed using the `ChildInfo` +A `PrefixedStorageKey` based on the child trie ID can be constructed using the `ChildInfo` primitive as follows: ```rust @@ -159,7 +159,7 @@ let prefixed_storage_key = ChildInfo::new_default(&trie_id).into_prefixed_storag ### Calculate the storage key using transparent hashing Finally, we calculate the hashed storage key of the storage item we are wanting to access. -The algorithm is simple: `Blake2_128` hash the storage key and then concatenate the unhashed +The algorithm is simple: `Blake2_128` hash the storage key and then concatenate the unhashed key to the hash. Given you want to access the storage item under the `0x00000000`, it will look like this in code: @@ -178,9 +178,9 @@ Let's recap the last few paragraphs into a full example. Given: * With instantiation nonce of `1` * The root storage struct is to be found at base key `0x00000000` -The following Rust program demonstrates how to calculate the `PrefixedStorageKey` of the -contracts child trie, as well as the hashed key for the storage struct, which can then be -used with the `chilstate` RPC endpoint function `getStorage` in polkadot-js to receive +The following Rust program demonstrates how to calculate the `PrefixedStorageKey` of the +contracts child trie, as well as the hashed key for the storage struct, which can then be +used with the `chilstate` RPC endpoint function `getStorage` in polkadot-js to receive the root storage struct of the contract: ```rust diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md index 16a16163ac..79733371bc 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md @@ -3,9 +3,9 @@ title: Storage Layout slug: /datastructures/storage-layout --- -Smart contract authors are given some flexibility in regards on how they want to organize +Smart contract authors are given some flexibility in regards on how they want to organize the storage layout of their contracts. -Let's dive deeper into the concepts behind ink! storage to get a better understanding +Let's dive deeper into the concepts behind ink! storage to get a better understanding of some of its implications and limitations. ## Storage Organization @@ -17,10 +17,10 @@ to ink! by the contracts pallet: Storage Organization: Layout -Storage data is always encoded with the -[`SCALE`](https://docs.substrate.io/reference/scale-codec/) codec. -The storage API operates by storing and loading entries into and from a single storages -cells, where each storage cell is accessed under its own dedicated storage key. To some +Storage data is always encoded with the +[`SCALE`](https://docs.substrate.io/reference/scale-codec/) codec. +The storage API operates by storing and loading entries into and from a single storages +cells, where each storage cell is accessed under its own dedicated storage key. To some extent, the storage API works similar to a traditional key-value database. ## Packed vs Non-Packed layout @@ -28,42 +28,42 @@ extent, the storage API works similar to a traditional key-value database. Types that can be stored entirely under a single storage cell are considered [`Packed`](https://docs.rs/ink_storage_traits/4.0.0-beta.1/ink_storage_traits/trait.Packed.html). By default, ink! tries to store all storage struct fields under a single storage cell. -Consequentially, with a `Packed` storage layout, any message interacting with the contract +Consequentially, with a `Packed` storage layout, any message interacting with the contract storage will always need to operate on the entire contract storage struct. -For example, if we have a somewhat small contract storage struct consisting of only a few -tiny fields, pulling everything from the storage inside every message is not -problematic. It may even be advantageous - especially if we expect most messages to +For example, if we have a somewhat small contract storage struct consisting of only a few +tiny fields, pulling everything from the storage inside every message is not +problematic. It may even be advantageous - especially if we expect most messages to interact with most of the storage fields. On the other hand, this can get problematic if we're storing a large `ink::prelude::vec::Vec` -in the contract storage but provide messages that do not need to read and write from this -`Vec`. In that scenario, each and every contract message bears runtime overhead by dealing -with that `Vec`, regardless whether they access it or not. This results in extra gas costs. +in the contract storage but provide messages that do not need to read and write from this +`Vec`. In that scenario, each and every contract message bears runtime overhead by dealing +with that `Vec`, regardless whether they access it or not. This results in extra gas costs. To solve this problem we need to turn our storage into a non-packed layout somehow. :::caution If any type exhibiting `Packed` layout gets large enough (an ever growing `Vec` might be -a prime candidate for this), it will break your contract. -This is because for encoding and decoding storage items, there is a buffer with only limited -capacity (around 16KB in the default configuration) available. This means any contract -trying to decode more than that will trap! If you are unsure about the potential size a -datastructure might get, consider using an ink! `Mapping`, which can store an arbitrary +a prime candidate for this), it will break your contract. +This is because for encoding and decoding storage items, there is a buffer with only limited +capacity (around 16KB in the default configuration) available. This means any contract +trying to decode more than that will trap! If you are unsure about the potential size a +datastructure might get, consider using an ink! `Mapping`, which can store an arbitrary number of elements, instead. -::: +::: ## Eager Loading vs. Lazy Loading -ink! provides means of breaking the storage up into smaller pieces, which can be loaded +ink! provides means of breaking the storage up into smaller pieces, which can be loaded on demand, with the [`Lazy`](https://paritytech.github.io/ink/ink/storage/struct.Lazy.html) primitive. Wrapping any storage field inside a `Lazy` struct makes the storage -struct in which that field appears also +struct in which that field appears also non-`Packed`, preventing it from being eagerly loaded during arbitrary storage operations. -The following example demonstrates how we can solve the problem introduced in the above -section. You'll notice that for the lazily loaded storage field, we now work with getters +The following example demonstrates how we can solve the problem introduced in the above +section. You'll notice that for the lazily loaded storage field, we now work with getters and setters to access and modify the underlying storage value: ```rust @@ -108,20 +108,20 @@ mod mycontract { :::caution -`ink::prelude::vec::Vec`'s are always loaded in their entirety. This is because all elements -of the `ink::prelude::vec::Vec` live under a single storage key. Wrapping the -`ink::prelude::vec::Vec` inside `Lazy`, like the -provided example above does, has no influence on its inner layout. If you are dealing with +`ink::prelude::vec::Vec`'s are always loaded in their entirety. This is because all elements +of the `ink::prelude::vec::Vec` live under a single storage key. Wrapping the +`ink::prelude::vec::Vec` inside `Lazy`, like the +provided example above does, has no influence on its inner layout. If you are dealing with large or sparse arrays on contract storage, consider using a `Mapping` instead. ::: ## Manual vs. Automatic Key Generation -By default, keys are calculated automatically for you, thanks to the +By default, keys are calculated automatically for you, thanks to the [`AutoKey`](https://docs.rs/ink_storage_traits/4.0.0-beta.1/ink_storage_traits/struct.AutoKey.html) -primitive. They'll be generated at compile time and ruled out for conflicts. -However, for non-`Packed` types like `Lazy` or the `Mapping`, the +primitive. They'll be generated at compile time and ruled out for conflicts. +However, for non-`Packed` types like `Lazy` or the `Mapping`, the [`ManualKey`](https://docs.rs/ink_storage_traits/4.0.0-beta.1/ink_storage_traits/struct.ManualKey.html) primitive allows manual control over the storage key of a field like so: @@ -133,13 +133,13 @@ pub struct MyContract { } ``` -This may be advantageous: Your storage key will always stay the same, regardless of -the version of your contract or ink! itself (note that the key calculation algorithm may +This may be advantageous: Your storage key will always stay the same, regardless of +the version of your contract or ink! itself (note that the key calculation algorithm may change with future ink! versions). :::tip -Using `ManualKey` instead of `AutoKey` might be especially desirable for upgradable +Using `ManualKey` instead of `AutoKey` might be especially desirable for upgradable contracts, as using `AutoKey` might result in a different storage key for the same field in a newer version of the contract. This may break your contract after an upgrade 😱! @@ -147,9 +147,9 @@ in a newer version of the contract. This may break your contract after an upgrad ## Considerations -It might be worthwhile to think about the desired storage layout of your contract. While -using a `Packed` layout will keep your contracts overall code size smaller, it can cause -unnecessarily high gas costs. Thus we consider it a good practice to break up large +It might be worthwhile to think about the desired storage layout of your contract. While +using a `Packed` layout will keep your contracts overall code size smaller, it can cause +unnecessarily high gas costs. Thus we consider it a good practice to break up large or complex storage layouts into reasonably sized distinct storage cells. :::note From 5481b4dc2f39f4c29dc72381431e8a01ed722d73 Mon Sep 17 00:00:00 2001 From: xermicus Date: Thu, 2 Feb 2023 12:02:08 +0100 Subject: [PATCH 76/76] better explain storage_item macro Signed-off-by: xermicus --- .../version-4.0.0-alpha.1/datastructures/custom.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md b/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md index a355aa273c..39943dcbf8 100644 --- a/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md +++ b/versioned_docs/version-4.0.0-alpha.1/datastructures/custom.md @@ -57,6 +57,20 @@ pub struct ContractStorage { Naturally, you can as well implement any required trait manually. Please directly refer to the relevant trait documentations for more information. +:::note + +The `#[ink::storage_item]` macro is responsible for storage key calculation of +non-[`Packed`](https://docs.rs/ink_storage_traits/4.0.0-beta.1/ink_storage_traits/trait.Packed.html) +types. Without it, the key for non-`Packed` fields will be zero. Using this macro is +necessary if you don't plan to use a +[`ManualKey`](https://docs.rs/ink_storage_traits/4.0.0-beta.1/ink_storage_traits/struct.ManualKey.html) +on a non-`Packed` type. + +Types with custom implementations of the ink! storage traits can still use this macro only +for key calculation by disabling the derives: `#[ink::storage_item(derive = false)]`. + +::: + ## Generic storage fields It is possible to use generic data types in your storage, as long as any generic type