Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent storage item reads/writes at compile time by types that are not whitelisted #149

Open
gpestana opened this issue Jul 24, 2023 · 6 comments
Labels
I5-enhancement An additional feature request. I10-unconfirmed Issue might be valid, but it's not yet known. T1-FRAME This PR/Issue is related to core FRAME, the framework.

Comments

@gpestana
Copy link
Contributor

gpestana commented Jul 24, 2023

It would be useful to limit at compile time the types that can access (read and/or write) a pallet storage item as a way to ensure safety and encapsulation. This can probably be achieved by augmenting the current outer macro to statically check if the storage item is only accessed by the whitelisted structs.

For example:

#[pallet::storage(allow_only(Ticker))]
pub type Count<T: Config> = StorageValue<_, u32, ValueQuery>;

impl<T: Config> Pallet<T> {
 fn add_count() {
  Count::<T>::mutate(|c| *c + 1) // compile time error because `Count` is accessed outside of the `Ticker` impl.
 }
}

// snip..

pub struct Ticker<T> {
 // snip..
}

impl <T: Config> Ticker<T> {
 fn add_count() {
    // set locks, do checks/ something else that is hard to enforce outside this impl..

    // and then..
    Count::<T>::mutate(|c| *c + 1) // OK, whitelisted
  }
}

This would be very useful to ensure that there is a strict control on how and when a storage item is mutated and/or read. An example of this is paritytech/substrate#14582, where we want to encapsulate all the logic to update the staking ledger (locks, etc) in a struct impl.

The annotation syntax is open for discussion. I think it would be useful to have some degree of granularity and whitelist only reads, writes or reads and write through the annotation.

@gpestana gpestana changed the title Prevent storage item reads/writes at compile time Prevent storage item reads/writes at compile time by types that are not whitelisted Jul 24, 2023
@sam0x17
Copy link
Contributor

sam0x17 commented Jul 24, 2023

This could probably be accomplished with a syn visitor pattern during the parse phase of #[pallet] 👍🏻

@xlc
Copy link
Contributor

xlc commented Jul 24, 2023

Can we stop adding more macro magics and try to figure out some mechanism using standard Rust concepts? Well, we cannot avoid macro magics but please try to restraint yourself from doing anything crazy with it.

In the example provided, an alternative implementation could be

// the macro already generated `struct Ticker` so we can just extend it
#[pallet::storage(getter = false, setter = false)] // accessors are not public
pub type Ticker<T: Config> = StorageValue<_, u32, ValueQuery>;

impl <T: Config> Ticker<T> {
 fn add_count() {
    // set locks, do checks/ something else that is hard to enforce outside this impl..

    // and then..
    Self::<T>::mutate(|c| *c + 1) // able to access private accessors
  }
}

@DamianStraszak
Copy link

How would it interact with migrations though?

@gpestana
Copy link
Contributor Author

How would it interact with migrations though?

With the visitor pattern approach it should be possible to whitelist migrations as well by default. If the setters and getters are not public, I think that's going to be less straightforward.

@sam0x17
Copy link
Contributor

sam0x17 commented Jul 31, 2023

If we're opposed to building this into the macro system, this could become a lint. Does clippy add the ability to define repo-specific custom syntax checkers?

@bkchr
Copy link
Member

bkchr commented Aug 17, 2023

The idea from @xlc looks reasonable!

@juangirini juangirini transferred this issue from paritytech/substrate Aug 24, 2023
@the-right-joyce the-right-joyce added I5-enhancement An additional feature request. I10-unconfirmed Issue might be valid, but it's not yet known. T1-FRAME This PR/Issue is related to core FRAME, the framework. and removed J0-enhancement labels Aug 25, 2023
gpestana added a commit that referenced this issue Oct 15, 2023
This PR refactors the staking ledger logic to encapsulate all reads and
mutations of `Ledger`, `Bonded`, `Payee` and stake locks within the
`StakingLedger` struct implementation.

With these changes, all the reads and mutations to the `Ledger`, `Payee`
and `Bonded` storage map should be done through the methods exposed by
StakingLedger to ensure the data and lock consistency of the operations.
The new introduced methods that mutate and read Ledger are:

- `ledger.update()`: inserts/updates a staking ledger in storage;
updates staking locks accordingly (and ledger.bond(), which is synthatic
sugar for ledger.update())
- `ledger.kill()`: removes all Bonded and StakingLedger related data for
a given ledger; updates staking locks accordingly;
`StakingLedger::get(account)`: queries both the `Bonded` and `Ledger`
storages and returns a `Option<StakingLedger>`. The pallet impl exposes
fn ledger(account) as synthatic sugar for `StakingLedger::get(account)`.

Retrieving a ledger with `StakingLedger::get()` can be done by providing
either a stash or controller account. The input must be wrapped in a
`StakingAccount` variant (Stash or Controller) which is treated
accordingly. This simplifies the caller API but will eventually be
deprecated once we completely get rid of the controller account in
staking. However, this refactor will help with the work necessary when
completely removing the controller.

Other goals:

- No logical changes have been introduced in this PR;
- No breaking changes or updates in wallets required;
- No new storage items or need to perform storage migrations;
- Centralise the changes to bonds and ledger updates to simplify the
OnStakingUpdate updates to the target list (related to
#443)

Note: it would be great to prevent or at least raise a warning if
`Ledger<T>`, `Payee<T>` and `Bonded<T>` storage types are accessed
outside the `StakingLedger` implementation. This PR should not get
blocked by that feature, but there's a tracking issue here
#149

Related and step towards
#443
bgallois pushed a commit to duniter/duniter-polkadot-sdk that referenced this issue Mar 25, 2024
This PR refactors the staking ledger logic to encapsulate all reads and
mutations of `Ledger`, `Bonded`, `Payee` and stake locks within the
`StakingLedger` struct implementation.

With these changes, all the reads and mutations to the `Ledger`, `Payee`
and `Bonded` storage map should be done through the methods exposed by
StakingLedger to ensure the data and lock consistency of the operations.
The new introduced methods that mutate and read Ledger are:

- `ledger.update()`: inserts/updates a staking ledger in storage;
updates staking locks accordingly (and ledger.bond(), which is synthatic
sugar for ledger.update())
- `ledger.kill()`: removes all Bonded and StakingLedger related data for
a given ledger; updates staking locks accordingly;
`StakingLedger::get(account)`: queries both the `Bonded` and `Ledger`
storages and returns a `Option<StakingLedger>`. The pallet impl exposes
fn ledger(account) as synthatic sugar for `StakingLedger::get(account)`.

Retrieving a ledger with `StakingLedger::get()` can be done by providing
either a stash or controller account. The input must be wrapped in a
`StakingAccount` variant (Stash or Controller) which is treated
accordingly. This simplifies the caller API but will eventually be
deprecated once we completely get rid of the controller account in
staking. However, this refactor will help with the work necessary when
completely removing the controller.

Other goals:

- No logical changes have been introduced in this PR;
- No breaking changes or updates in wallets required;
- No new storage items or need to perform storage migrations;
- Centralise the changes to bonds and ledger updates to simplify the
OnStakingUpdate updates to the target list (related to
paritytech#443)

Note: it would be great to prevent or at least raise a warning if
`Ledger<T>`, `Payee<T>` and `Bonded<T>` storage types are accessed
outside the `StakingLedger` implementation. This PR should not get
blocked by that feature, but there's a tracking issue here
paritytech#149

Related and step towards
paritytech#443
jonathanudd pushed a commit to jonathanudd/polkadot-sdk that referenced this issue Apr 10, 2024
* Add skeleton for worst case import_unsigned_header

* Fix a typo

* Add benchmark test for best case unsigned header import

* Add finality verification to worst case bench

* Move `insert_header()` from mock to test_utils

Allows the benchmarking code to use this without having to pull it in from the mock.

* Add a rough bench to test a finalizing a "long" chain

* Try to use complexity parameter for finality bench

* Improve long finality bench

* Remove stray dot file

* Remove old "worst" case bench

* Scribble some ideas down for pruning bench

* Prune headers during benchmarking

* Clean up some comments

* Make finality bench work for entire range of complexity parameter

* Place initialization code into a function

* Add bench for block finalization with caching

* First attempt at bench with receipts

* Try and trigger validator set change

* Perform a validator set change during benchmarking

* Move `validators_change_receipt()` to shared location

Allows unit tests and benchmarks to access the same helper function
and const

* Extract a test receipt root into a constant

* Clean up description of pruning bench

* Fix cache and pruning tests

* Remove unecessary `build_custom_header` usage

* Get rid of warnings

* Remove code duplication comment

I don't think its entirely worth it to split out so few lines of code.
The benches aren't particularly hard to read anyways.

* Increase the range of the complexity parameter

* Use dynamic number of receipts while benchmarking

As part of this change we have removed the hardcoded TEST_RECEIPT_ROOT
and instead chose to calculate the receipt root on the fly. This will
make tests and benches less fragile.

* Prune a dynamic number of headers
claravanstaden pushed a commit to claravanstaden/polkadot-sdk that referenced this issue Sep 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
I5-enhancement An additional feature request. I10-unconfirmed Issue might be valid, but it's not yet known. T1-FRAME This PR/Issue is related to core FRAME, the framework.
Projects
Status: Backlog
Development

No branches or pull requests

7 participants