Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pallet_contracts] Add support for transient storage in contracts host functions #4566

Merged
merged 70 commits into from
Jul 16, 2024
Merged
Show file tree
Hide file tree
Changes from 63 commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
1732628
Add transient storage
smiasojed May 23, 2024
6cda150
Add unit tests
smiasojed May 23, 2024
6670170
Merge remote-tracking branch 'origin/master' into sm/contracts-tstore
smiasojed May 23, 2024
826e073
Add storage limit
smiasojed May 23, 2024
daf9ad1
Limit storage per contract
smiasojed May 24, 2024
d4a29be
Refactored
smiasojed May 28, 2024
53ec270
Add terminate function to transient storage
smiasojed May 30, 2024
33f6c2d
Add unit tests
smiasojed May 31, 2024
199edaa
Refactored to single BtreeMap
smiasojed Jun 3, 2024
e683b72
Add comments
smiasojed Jun 3, 2024
1c3c844
Cleanup
smiasojed Jun 4, 2024
18cd9ad
Fix unit tests
smiasojed Jun 4, 2024
d5736d5
Merge remote-tracking branch 'origin/master' into sm/contracts-tstore
smiasojed Jun 4, 2024
58d87dc
Fix tests after master merge
smiasojed Jun 4, 2024
29f1305
Rename storage err
smiasojed Jun 5, 2024
63aaaaf
Benchmarking test
smiasojed Jun 18, 2024
78960e0
Fix unit tests
smiasojed Jun 18, 2024
f09b80b
Merge branch 'master' of https://github.com/paritytech/polkadot-sdk i…
Jun 18, 2024
6d63e58
".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime…
Jun 18, 2024
6f713ae
Max storage elements set to 2000
smiasojed Jun 18, 2024
d6e1c43
Merge branch 'master' of https://github.com/paritytech/polkadot-sdk i…
Jun 18, 2024
8c11871
".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime…
Jun 18, 2024
664221b
Storage len 50
smiasojed Jun 18, 2024
d579005
".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime…
Jun 18, 2024
6667429
Merge remote-tracking branch 'origin/master' into sm/contracts-tstore
smiasojed Jun 20, 2024
8d91109
Merge remote-tracking branch 'origin/master' into sm/contracts-tstore
smiasojed Jun 24, 2024
5794910
Change transient storage meter
smiasojed Jun 25, 2024
da8316e
Fix benchmark warnings
smiasojed Jun 25, 2024
ac65687
Merge branch 'master' of https://github.com/paritytech/polkadot-sdk i…
Jun 25, 2024
d8298f0
".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime…
Jun 25, 2024
339e935
Refactor benchmarks
smiasojed Jun 26, 2024
735faae
Refactor transient storage
smiasojed Jun 26, 2024
51fb39c
Add test benchmarks
smiasojed Jun 27, 2024
2f3abc4
Merge branch 'master' of https://github.com/paritytech/polkadot-sdk i…
Jun 27, 2024
b2c7005
".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime…
Jun 27, 2024
01365cf
Refactored benchmarks
smiasojed Jun 27, 2024
a2dfb23
Add wasm tests
smiasojed Jun 28, 2024
ca1a323
Add E2E tests
smiasojed Jun 28, 2024
399d541
Merge branch 'master' of https://github.com/paritytech/polkadot-sdk i…
Jun 28, 2024
6d8dc05
".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime…
Jun 28, 2024
1ed42f4
Add docs
smiasojed Jul 1, 2024
3ec4626
Fix tests
smiasojed Jul 1, 2024
2d3a823
Fix contracts-rococo configuration
smiasojed Jul 1, 2024
09ff908
Fmt
smiasojed Jul 1, 2024
5fa1b1b
Update PR doc
smiasojed Jul 1, 2024
f6aeb87
Cleanup
smiasojed Jul 1, 2024
4863431
Remove transient storage limit in benchmark
smiasojed Jul 2, 2024
baa943f
Cleanup
smiasojed Jul 2, 2024
7257857
Merge remote-tracking branch 'origin/master' into sm/contracts-tstore
smiasojed Jul 2, 2024
bcde2de
Fix description
smiasojed Jul 2, 2024
9a204bd
".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime…
Jul 2, 2024
037f0e6
Update substrate/frame/contracts/src/exec.rs
smiasojed Jul 3, 2024
ea62ece
Update substrate/frame/contracts/src/exec.rs
smiasojed Jul 3, 2024
0888164
Update substrate/frame/contracts/src/transient_storage.rs
smiasojed Jul 4, 2024
176bbc5
Update substrate/frame/contracts/src/transient_storage.rs
smiasojed Jul 4, 2024
fb8f0ad
Change fn visibility and code cleanup
smiasojed Jul 4, 2024
4b148b9
Update tests
smiasojed Jul 4, 2024
057fd0d
Update tests
smiasojed Jul 4, 2024
6acc50f
Remove double key creation
smiasojed Jul 5, 2024
b9aa452
Update panic messages
smiasojed Jul 5, 2024
2bb933e
Update substrate/frame/contracts/src/lib.rs
smiasojed Jul 5, 2024
0c189a6
Cleanup benchmarks
smiasojed Jul 5, 2024
865886d
Improve panic messages
smiasojed Jul 5, 2024
03f10f9
Marked functions as unstable
smiasojed Jul 8, 2024
5c59336
Sealed HostFn trait
smiasojed Jul 9, 2024
8d36584
Refactor benchmarks
smiasojed Jul 9, 2024
4db9c95
Merge branch 'master' of https://github.com/paritytech/polkadot-sdk i…
Jul 9, 2024
f2cfb76
".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime…
Jul 9, 2024
3a8c2d2
Merge remote-tracking branch 'origin/master' into sm/contracts-tstore
smiasojed Jul 10, 2024
45603b8
Comment added
smiasojed Jul 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ impl Config for Runtime {
type AddressGenerator = DefaultAddressGenerator;
type MaxCodeLen = ConstU32<{ 123 * 1024 }>;
type MaxStorageKeyLen = ConstU32<128>;
type MaxTransientStorageSize = ConstU32<{ 1 * 1024 * 1024 }>;
type UnsafeUnstableInterface = ConstBool<true>;
type UploadOrigin = EnsureSigned<Self::AccountId>;
type InstantiateOrigin = EnsureSigned<Self::AccountId>;
Expand Down
23 changes: 23 additions & 0 deletions prdoc/pr_4566.prdoc
athei marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Schema: Polkadot SDK PRDoc Schema (prdoc) v1.0.0
# See doc at https://raw.githubusercontent.com/paritytech/polkadot-sdk/master/prdoc/schema_user.json

title: "[pallet_contracts] Add support for transient storage in contracts host functions"

doc:
- audience: Runtime User
description: |
This PR implements transient storage, which behaves identically to regular storage
but is kept only in memory and discarded after every transaction.
This functionality is similar to the `TSTORE` and `TLOAD` operations used in Ethereum.
The following new host functions have been introduced: `get_transient_storage`,
`set_transient_storage`, `take_transient_storage`, `clear_transient_storage` and
`contains_transient_storage`.
These functions are declared as unstable and thus are not activated.

crates:
- name: pallet-contracts
bump: major
- name: pallet-contracts-uapi
bump: major
- name: contracts-rococo-runtime
bump: minor
1 change: 1 addition & 0 deletions substrate/bin/node/runtime/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1371,6 +1371,7 @@ impl pallet_contracts::Config for Runtime {
type UploadOrigin = EnsureSigned<Self::AccountId>;
type InstantiateOrigin = EnsureSigned<Self::AccountId>;
type MaxDebugBufferLen = ConstU32<{ 2 * 1024 * 1024 }>;
type MaxTransientStorageSize = ConstU32<{ 1 * 1024 * 1024 }>;
type RuntimeHoldReason = RuntimeHoldReason;
#[cfg(not(feature = "runtime-benchmarks"))]
type Migrations = ();
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
// This file is part of Substrate.

// Copyright (C) Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0

// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

//! This calls another contract as passed as its account id. It also creates some transient storage.
#![no_std]
#![no_main]

use common::input;
use uapi::{HostFn, HostFnImpl as api};

#[no_mangle]
#[polkavm_derive::polkavm_export]
pub extern "C" fn deploy() {}

#[no_mangle]
#[polkavm_derive::polkavm_export]
pub extern "C" fn call() {
input!(
buffer,
len: u32,
input: [u8; 4],
callee: [u8; 32],
);

let data = [0u8; 16 * 1024];
let value = &data[..len as usize];
api::set_transient_storage(buffer, value);

// Call the callee
api::call_v2(
uapi::CallFlags::empty(),
callee,
0u64, // How much ref_time weight to devote for the execution. 0 = all.
0u64, // How much proof_size weight to devote for the execution. 0 = all.
None,
&0u64.to_le_bytes(), // Value transferred to the contract.
input,
None,
)
.unwrap();
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
// This file is part of Substrate.

// Copyright (C) Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0

// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

#![no_std]
#![no_main]

use common::input;
use uapi::{HostFn, HostFnImpl as api};

#[no_mangle]
#[polkavm_derive::polkavm_export]
pub extern "C" fn deploy() {}

#[no_mangle]
#[polkavm_derive::polkavm_export]
pub extern "C" fn call() {
input!(len: u32, );

let buffer = [0u8; 16 * 1024];
let data = &buffer[..len as usize];

// Place a garbage value in the transient storage, with the size specified by the call input.
let mut key = [0u8; 32];
key[0] = 1;

api::set_transient_storage(&key, data);
}
55 changes: 55 additions & 0 deletions substrate/frame/contracts/fixtures/contracts/transient_storage.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
// This file is part of Substrate.

// Copyright (C) Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0

// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

//! This contract tests the transient storage APIs.
#![no_std]
#![no_main]

use common::unwrap_output;
use uapi::{HostFn, HostFnImpl as api};

#[no_mangle]
#[polkavm_derive::polkavm_export]
pub extern "C" fn deploy() {}

#[no_mangle]
#[polkavm_derive::polkavm_export]
pub extern "C" fn call() {
const KEY: [u8; 32] = [1u8; 32];
const VALUE_1: [u8; 4] = [1u8; 4];
const VALUE_2: [u8; 4] = [2u8; 4];
const VALUE_3: [u8; 4] = [3u8; 4];

let existing = api::set_transient_storage(&KEY, &VALUE_1);
assert_eq!(existing, None);
assert_eq!(api::contains_transient_storage(&KEY), Some(VALUE_1.len() as _));
unwrap_output!(val, [0u8; 4], api::get_transient_storage, &KEY);
assert_eq!(**val, VALUE_1);

let existing = api::set_transient_storage(&KEY, &VALUE_2);
assert_eq!(existing, Some(VALUE_1.len() as _));
unwrap_output!(val, [0u8; 4], api::get_transient_storage, &KEY);
assert_eq!(**val, VALUE_2);

api::clear_transient_storage(&KEY);
assert_eq!(api::contains_storage(&KEY), None);

let existing = api::set_transient_storage(&KEY, &VALUE_3);
assert_eq!(existing, None);
unwrap_output!(val, [0u8; 32], api::take_transient_storage, &KEY);
assert_eq!(**val, VALUE_3);
}
31 changes: 28 additions & 3 deletions substrate/frame/contracts/src/benchmarking/call_builder.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,11 @@

use crate::{
benchmarking::{Contract, WasmModule},
exec::Stack,
exec::{Ext, Key, Stack},
storage::meter::Meter,
wasm::Runtime,
BalanceOf, Config, DebugBufferVec, Determinism, ExecReturnValue, GasMeter, Origin, Schedule,
TypeInfo, WasmBlob, Weight,
BalanceOf, Config, DebugBufferVec, Determinism, Error, ExecReturnValue, GasMeter, Origin,
Schedule, TypeInfo, WasmBlob, Weight,
};
use codec::{Encode, HasCompact};
use core::fmt::Debug;
Expand Down Expand Up @@ -160,6 +160,31 @@ where
)
}

/// Add transient_storage
pub fn with_transient_storage(ext: &mut StackExt<T>, size: u32) -> Result<(), &'static str> {
let amount = ext.transient_storage().meter().current().amount;
let limit = ext.transient_storage().meter().current().limit;
Copy link
Contributor

@pgherveou pgherveou Jul 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe

let &MeterEntry { amount, limit } = ext.transient_storage().meter().current();

ext.transient_storage().meter().current_mut().limit = size;
for i in 1u32.. {
let mut key_data = i.to_le_bytes().to_vec();
while key_data.last() == Some(&0) {
key_data.pop();
}
let key = Key::<T>::try_from_var(key_data).unwrap();
if let Err(e) = ext.set_transient_storage(&key, Some(Vec::new()), false) {
// Restore previous settings.
ext.transient_storage().meter().current_mut().limit = limit;
ext.transient_storage().meter().current_mut().amount = amount;
if e == Error::<T>::OutOfTransientStorage.into() {
break;
} else {
return Err("Initialization of the transient storage failed");
}
}
}
Ok(())
}

/// Prepare a call to the module.
pub fn prepare_call<'a>(
ext: &'a mut StackExt<'a, T>,
Expand Down
Loading
Loading