feat: expose validator information as host functions #2504

bowenwang1996 · 2020-04-19T22:48:10Z

Expose validator information as host functions:

validator_stake(account_id_len: u64, account_id_ptr: u64, stake_ptr: u64) -> []
total_validator_stake(stake_ptr: u64) -> []

so that voting contract can calculate the weight of each vote. Resolves #2475. Implements the runtime change specified in near/NEPs#62

Test plan

Test with near-sdk-rs (TODO).

gitpod-io · 2020-04-19T22:48:14Z

evgenykuzyakov

I think it has to go through NEP first.

While I agree, that we might want to know the current stake of a given validator, we also want to be able to go through the list of all validators.

bowenwang1996 · 2020-05-12T18:36:20Z

NEP near/NEPs#62

frol

I want to raise a concern that VMContext is getting bigger, and it merges various sorts of data from all over the places unconditionally (it locks epoch_manager to fetch this hashmap of validators for every single call even if it is not used inside the contract).

I understand the time constraints, so I am ok unblocking this PR if you think it is fine for now.

P.S. I am marking it as "request changes" since otherwise I will get pinged by GitHub bot for not providing a review.

neard/src/runtime.rs

frol · 2020-05-12T19:08:24Z

runtime/near-vm-logic/src/logic.rs

+    /// For not nul-terminated account id:
+    /// `base + read_memory_base + read_memory_byte * num_bytes + utf8_decoding_base + utf8_decoding_byte * num_bytes + memory_write_base + memory_write_size * 16`
+    ///
+    /// For nul-terminated account id :
+    /// `base + (read_memory_base + read_memory_byte) * num_bytes + utf8_decoding_base + utf8_decoding_byte * num_bytes + memory_write_base + memory_write_size * 16`


Why? How do we end up in this weird state?

This is related to how get_utf8_string works.

Let's just reference get_utf8_string for costs here. I am as a reader was confused.

runtime/near-vm-logic/src/logic.rs

runtime/runtime-params-estimator/src/wasmer_estimator.rs

bowenwang1996 · 2020-05-12T19:39:52Z

I want to raise a concern that VMContext is getting bigger, and it merges various sorts of data from all over the places unconditionally

That is true, but unless we do some static analysis on the code invoked in a block (which I think is more expensive), I don's see a better solution.

frol

That is true, but unless we do some static analysis on the code invoked in a block (which I think is more expensive), I don's see a better solution.

I can think of a solution of using static capabilities set on the contract level. Like on Web & Mobile, you ask for permission to get some information. In our case, we can benefit from using metadata enabling certain capabilities statically on compilation or deployment. /cc @evgenykuzyakov @nearmax @vgrichina @ilblackdragon @fckt

P.S. I am approving this PR due to the time constraints. We can design the capabilities for contracts later, when we address the metadata topic.

frol · 2020-05-12T20:22:59Z

runtime/near-vm-logic/src/logic.rs

+    /// For not nul-terminated account id:
+    /// `base + read_memory_base + read_memory_byte * num_bytes + utf8_decoding_base + utf8_decoding_byte * num_bytes + memory_write_base + memory_write_size * 16`
+    ///
+    /// For nul-terminated account id :
+    /// `base + (read_memory_base + read_memory_byte) * num_bytes + utf8_decoding_base + utf8_decoding_byte * num_bytes + memory_write_base + memory_write_size * 16`


Let's just reference get_utf8_string for costs here. I am as a reader was confused.

neard/src/runtime.rs

frol · 2020-05-13T14:48:21Z

core/chain-configs/src/lib.rs

@@ -7,4 +7,4 @@ pub use genesis_config::{
 };

 /// Current latest version of the protocol
-pub const PROTOCOL_VERSION: u32 = 10;
+pub const PROTOCOL_VERSION: u32 = 11;


Update after #2574

evgenykuzyakov

I think we should expose this information through External trait instead of passing it to VMContext. Otherwise every simple function call now requires to reconstruct the map. Including view calls.
If RuntimeExt instead takes reference to EpochManager trait that exposes this information, then we don't need to construct the map. It will allow optimizations later.

It will matter, especially when there are sharding and there are 800+ seats.

MaksymZavershynskyi · 2020-05-13T21:42:54Z

For the context: #2610

MaksymZavershynskyi · 2020-05-13T21:43:56Z

@bowenwang1996 Should we close this PR in favor of the new design that we have discussed on Tue May 12?

MaksymZavershynskyi · 2020-05-13T21:45:09Z

CC @mikhailOK

bowenwang1996 · 2020-05-14T00:44:01Z

@nearmax it is different. This is for the voting contract.

bowenwang1996 · 2020-05-16T00:54:23Z

@nearmax please let me know how I should price ext usage.

MaksymZavershynskyi · 2020-05-16T23:50:04Z

As discussed during the middleware sync our infrastructure is not ready to estimate this specific host function correctly, not until we extract epoch manager and add it to the standalone runtime. Until then we will overshoot the cost of this function to avoid grinding. I suggest the price of this function to be as high as the price of deploying a 1MB contract. @bowenwang1996 , when adding this fee please sync with @olonho to also make a change in the price estimation tool to avoid breaking it.

neard/src/runtime.rs

core/primitives/src/types.rs

core/primitives/src/errors.rs

runtime/near-vm-logic/src/logic.rs

runtime/near-vm-runner/src/imports.rs

runtime/runtime/src/ext.rs

MaksymZavershynskyi

There is a bug with order of application of fees. Also, I couldn't find the test for the new functionality. Also, please don't forget to upate nomicon and near-sdk-rs. CC @willemneal for near-sdk-as.

runtime/near-vm-logic/src/logic.rs

core/primitives/src/errors.rs

bowenwang1996 · 2020-05-21T19:50:09Z

@nearmax tests are in near-vm-runner/tests/test_rs_contract.rs:

def_test_ext!(
    ext_validator_stake_alice,
    b"ext_validator_stake",
    &(100u128).to_le_bytes(),
    b"alice"
);
def_test_ext!(ext_validator_stake_bob, b"ext_validator_stake", &(1u128).to_le_bytes(), b"bob");
def_test_ext!(ext_validator_stake_carol, b"ext_validator_stake", &(0u128).to_le_bytes(), b"carol");

def_test_ext!(
    ext_validator_total_stake,
    b"ext_validator_total_stake",
    &(100u128 + 1).to_le_bytes()
)

MaksymZavershynskyi

LGTM assuming the internals of epoch manager are used correctly. Please see the comment about bumping versions.

MaksymZavershynskyi · 2020-05-21T20:37:32Z

runtime/near-vm-logic/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "near-vm-logic"
-version = "0.8.0"
+version = "0.9.0"


Please update version of all crates in nearcore/runtime. We currently keep them in sync, until we decide to split them in the future.

In #2504 I made a mistake of not making `validators` in `MockedExternal` public and this is problematic for testing on near-sdk-rs side (cannot change validator information). This PR fixes the issue. Test plan --------- Existing tests pass.

feat: expose validator information as host functions

cf8f564

bowenwang1996 requested a review from evgenykuzyakov April 19, 2020 22:48

evgenykuzyakov requested changes Apr 21, 2020

View reviewed changes

bowenwang1996 mentioned this pull request May 4, 2020

feat: voting contract near/NEPs#62

Closed

Merge branch 'master' into expose-epoch-info

b19654e

bowenwang1996 marked this pull request as ready for review May 12, 2020 18:36

bowenwang1996 requested review from lexfrl, frol, MaksymZavershynskyi, olonho and SkidanovAlex as code owners May 12, 2020 18:36

bowenwang1996 requested a review from evgenykuzyakov May 12, 2020 18:36

frol requested changes May 12, 2020

View reviewed changes

update protocol version

668aca7

bowenwang1996 force-pushed the expose-epoch-info branch from 58899e5 to 668aca7 Compare May 12, 2020 19:37

frol approved these changes May 12, 2020

View reviewed changes

frol mentioned this pull request May 12, 2020

fix(runtime costs): adjust runtime costs #2574

Merged

frol reviewed May 13, 2020

View reviewed changes

evgenykuzyakov requested changes May 13, 2020

View reviewed changes

use ext

7d54e92

bowenwang1996 requested a review from evgenykuzyakov May 16, 2020 00:54

evgenykuzyakov reviewed May 17, 2020

View reviewed changes

neard/src/runtime.rs Outdated Show resolved Hide resolved

neard/src/runtime.rs Outdated Show resolved Hide resolved

lexfrl reviewed May 18, 2020

View reviewed changes

core/primitives/src/types.rs Outdated Show resolved Hide resolved

lexfrl approved these changes May 18, 2020

View reviewed changes

bowenwang1996 added 4 commits May 20, 2020 10:15

address comments

cb2b9cb

Merge branch 'master' into expose-epoch-info

01bf462

address comment

634be5f

add pricing

26a800d

bowenwang1996 requested a review from evgenykuzyakov May 20, 2020 22:41

fix tests

f1891f2

evgenykuzyakov requested changes May 21, 2020

View reviewed changes

core/primitives/src/errors.rs Outdated Show resolved Hide resolved

runtime/near-vm-logic/src/logic.rs Outdated Show resolved Hide resolved

runtime/near-vm-runner/src/imports.rs Outdated Show resolved Hide resolved

runtime/runtime/src/ext.rs Outdated Show resolved Hide resolved

bowenwang1996 added 4 commits May 21, 2020 09:28

address comments

17aa971

Merge branch 'master' into expose-epoch-info

8c899a3

bump version

5372d4c

change cost

a2a3c73

evgenykuzyakov approved these changes May 21, 2020

View reviewed changes

SkidanovAlex approved these changes May 21, 2020

View reviewed changes

MaksymZavershynskyi suggested changes May 21, 2020

View reviewed changes

runtime/near-vm-logic/src/logic.rs Outdated Show resolved Hide resolved

runtime/near-vm-logic/src/logic.rs Show resolved Hide resolved

core/primitives/src/errors.rs Show resolved Hide resolved

address comments

555cd31

bowenwang1996 requested a review from MaksymZavershynskyi May 21, 2020 19:50

MaksymZavershynskyi approved these changes May 21, 2020

View reviewed changes

bowenwang1996 added 4 commits May 21, 2020 15:11

bump verions of runtime crates

26d542a

Merge branch 'master' into expose-epoch-info

d371471

bump protocol version

94974db

Merge branch 'master' into expose-epoch-info

9c8d0eb

bowenwang1996 merged commit 442998a into master May 21, 2020

bowenwang1996 deleted the expose-epoch-info branch May 21, 2020 23:28

bowenwang1996 mentioned this pull request May 22, 2020

fix: expose validators in MockedExternal #2709

Merged

mfornet mentioned this pull request Sep 27, 2021

Expose epoch manager information #4885

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: expose validator information as host functions #2504

feat: expose validator information as host functions #2504

bowenwang1996 commented Apr 19, 2020 •

edited

Loading

gitpod-io bot commented Apr 19, 2020 •

edited

Loading

evgenykuzyakov left a comment

bowenwang1996 commented May 12, 2020

frol left a comment

frol May 12, 2020

bowenwang1996 May 12, 2020

frol May 12, 2020

bowenwang1996 commented May 12, 2020

frol left a comment

frol May 12, 2020

frol May 13, 2020

evgenykuzyakov left a comment •

edited

Loading

MaksymZavershynskyi commented May 13, 2020

MaksymZavershynskyi commented May 13, 2020

MaksymZavershynskyi commented May 13, 2020

bowenwang1996 commented May 14, 2020 •

edited

Loading

bowenwang1996 commented May 16, 2020

MaksymZavershynskyi commented May 16, 2020

MaksymZavershynskyi left a comment

bowenwang1996 commented May 21, 2020

MaksymZavershynskyi left a comment

MaksymZavershynskyi May 21, 2020

bowenwang1996 May 21, 2020

feat: expose validator information as host functions #2504

feat: expose validator information as host functions #2504

Conversation

bowenwang1996 commented Apr 19, 2020 • edited Loading

Test plan

gitpod-io bot commented Apr 19, 2020 • edited Loading

evgenykuzyakov left a comment

Choose a reason for hiding this comment

bowenwang1996 commented May 12, 2020

frol left a comment

Choose a reason for hiding this comment

frol May 12, 2020

Choose a reason for hiding this comment

bowenwang1996 May 12, 2020

Choose a reason for hiding this comment

frol May 12, 2020

Choose a reason for hiding this comment

bowenwang1996 commented May 12, 2020

frol left a comment

Choose a reason for hiding this comment

frol May 12, 2020

Choose a reason for hiding this comment

frol May 13, 2020

Choose a reason for hiding this comment

evgenykuzyakov left a comment • edited Loading

Choose a reason for hiding this comment

MaksymZavershynskyi commented May 13, 2020

MaksymZavershynskyi commented May 13, 2020

MaksymZavershynskyi commented May 13, 2020

bowenwang1996 commented May 14, 2020 • edited Loading

bowenwang1996 commented May 16, 2020

MaksymZavershynskyi commented May 16, 2020

MaksymZavershynskyi left a comment

Choose a reason for hiding this comment

bowenwang1996 commented May 21, 2020

MaksymZavershynskyi left a comment

Choose a reason for hiding this comment

MaksymZavershynskyi May 21, 2020

Choose a reason for hiding this comment

bowenwang1996 May 21, 2020

Choose a reason for hiding this comment

bowenwang1996 commented Apr 19, 2020 •

edited

Loading

gitpod-io bot commented Apr 19, 2020 •

edited

Loading

evgenykuzyakov left a comment •

edited

Loading

bowenwang1996 commented May 14, 2020 •

edited

Loading