Math API ecrecover and ripemd160 extension (#4380)

* feat: Extend the Math API with EVM precompiles.

* feat: Add Math API stubs for ECRecover. (#3921)

* test: Add RIPEMD-160 and BLAKE2b to the params estimator.

* fix: Improve error output from the params estimator.

Co-authored-by: Aleksey Kladov <aleksey.kladov@gmail.com>

* test: Add ECRecover stubs to the params estimator.

* fix: Widen ecrecover() parameter type to pass CI checks.

* feat(chain): Bump the protocol version.

* feat(runtime): Implement ECRecover in the Math API. (#3921)

* docs: Document the ecrecover() function.

* fix(chain): Guard protocol upgrade behind EVM feature.

* add crypto feature gates

* fix nightly compile

* cargo fmt

* fix two ripemd160 missing feature gates

* rename crypto_extras feature to protocol_feature equivalents

* change repo to our git

* put everything under protocol_feature_evm

* point to alternative blake2 branch temporarily

* Add blake2b f function

* add blake2b_f to config

* post rebase fixes

* wrap all ext_costs in evm feature

* add blake_2b_f to runtime estimator

* cargo fmt

* add new hash algos to feature

* add vm-logic test for blake2b_f

* fix derive formatting

* use same blake2 library

* change blake F tests to test 1 round

* fix blake f test

* cleanup calls_helper

* add missing feature gated fields from ExtCostsConfig

* add blake2b_f compression docs

* remove unused `blake2b_f_byte` as sizes are fixed

* add missing blake2b_f to imports

* runtime estimator mod docs

* Add near-blake2 lib to near-crypto

* fix test imports

* add more pub methods to blake2

* add blake2 state success tests

* cargo fmt

* Add initial blake2 logic to vm

* Fix ecrecover

* Change blake2 `update` to `update_inner`

* Update params estimator

* Remove duplicate evm feature

* Remove pattern

* Fix some compilation errors

* Fix runtime-params-estimator compile

* Fix tests

* Reduce blake2s param estimator rounds from 12 to 10

* Add protocol_feature_evm feature to standalone runner

* Add estimated gas values

* Update estimated gas value note

* cargo fmt

* Update c2-chacha

* Remove nightly features from lib blake2

* Add blake2 pass tests

* Fix up ecrecover test

* Fix blake2s args

* Remove blake2 lib

* Update documentation for blake2

* Pin comment for blake2 lib

* Fix with_state

* Return a bool for ecrecover, not abort

* Fix VarBlake2b -> VarBlake2s

* Add errors to RPC errors schema

* Add bool return to wrapped imports

* Take t0 and t1 for blake2b

* Charge ripemd160 by blocks, not bytes

* Correctly send both t0 and t1 to blake2

* Remove too many rounds error

* Remove any near-blake2 errors

* Remove hash data overflow error

* Rename Blake2InvalidStateLength -> Blake2StateLengthExceeded

* Small ecrecover clarifications

* Return public key in ecrecover

* Correct blake2 message block math

* Correct ripemd160 message block math

* Read blake2s as u32 not u64

* Remove `state_len` and blake2 len error

* Add `saturating_mul` to blake2 math

* Don't allocate an extra vec on blake2

* Remove unused import

* Fix args in wrapped imports

* Fix blake2s test

* Remove blake2 and added protocol_feature_evm conditional comps

* Remove optional from crypto imports in logic

* Revert from near-blake2 to upstream blake2

* Drop libsecp256k1 crate in favour of near-crypto

* Improve code quality by using U256

* Pass hash_bytes as ref

* Revert "Pass hash_bytes as ref"

This reverts commit 434145b.

* matklad nit

Co-authored-by: Aleksey Kladov <aleksey.kladov@gmail.com>

* Fix ecrecover exports arg sig

* Move when malleability check happens

* Remove last check in signature values

Co-authored-by: Aleksey Kladov <aleksey.kladov@gmail.com>

* Don't use `v` in a check, now unused

* Update description of ripemd160 with correct cost

* Use u32 for malleability_flag

* Introduce protocol_feature_math_extension

* Use our own div_ceil implementation

* Include protocol_feature_math_extension in runtime-params-estimator/nightly_protocol_features

* Nightly protocol version bump

* Simplify message_blocks computation

* Remove unused div_ceil function

* Ignore first byte of result

* Remove uneeded return at end of function

Co-authored-by: EgorKulikov <egor@egork.net>

* Remove check_v and add in option to skip reject upper range

* Use `get_vec_from_memory_or_register` over `memory_get_into`

* Remove unused

* Separate `v`, change args to `u64`

* Remove redundant check

* 64th byte, not 65

Co-authored-by: Aleksey Kladov <aleksey.kladov@gmail.com>

* Fix test

* Update doc cost

* Fix estimator

* Small fixes

* Add missing feature `protocol_feature_math_extension` from neard

* Add near-primitives to math extension params estimator

* Fix byte length

* Fix Cargo.toml

* Nit. Typos

* Nit. param estimator is missing feature dependency

* Add feature support for build.sh of test-contract

* Add consts for SECP256K1 malleability values

* Remove map err, change to unwrap

* Change v check to something more simple

* Always do signature check

* Check one s bound

* Allow for specifying a register

* Remove math extension feature from primitives core

* Nit. Convering byte to bool

* Add measuring support.

* Revert "Remove math extension feature from primitives core"

This reverts commit ac53dcb.

* Const SECP256K1 half + 1

* Revert "Add measuring support."

This reverts commit 210ee55.

* Update gas costs with correct values

* Nit. Method signature typo causing runtime failure

* Fix fee. It seems like a copy-paste typo

* Undo the wrong fix

* Add malleability_flag check as requested in #4380 (comment)

* syntax fix

Co-authored-by: Arto Bendiken <arto@near.org>
Co-authored-by: Aleksey Kladov <aleksey.kladov@gmail.com>
Co-authored-by: Michael Birch <michael@near.org>
Co-authored-by: Michael Birch <michael.birch@aurora.dev>
Co-authored-by: EgorKulikov <egor@egork.net>
Co-authored-by: Maksym Zavershynskyi <max@near.org>
Co-authored-by: Nikolay Igotti <igotti@gmail.com>
Co-authored-by: Bowen Wang <bowenwang1996@users.noreply.github.com>
Co-authored-by: Bowen Wang <bowen@near.org>

core/crypto/Cargo.toml

@@ -13,6 +13,7 @@ c2-chacha = "0.3"
 curve25519-dalek = "3"
 derive_more = "0.99.9"
 ed25519-dalek = "1"
+ethereum-types = "0.11.0"
 lazy_static = "1.4"
 libc = "0.2"
 parity-secp256k1 = "0.7"

core/crypto/src/lib.rs

@@ -1,7 +1,8 @@
 pub use errors::{ParseKeyError, ParseKeyTypeError, ParseSignatureError};
 pub use key_file::KeyFile;
 pub use signature::{
-    ED25519PublicKey, KeyType, PublicKey, Secp256K1PublicKey, SecretKey, Signature,
+    ED25519PublicKey, KeyType, PublicKey, Secp256K1PublicKey, Secp256K1Signature, SecretKey,
+    Signature,
 };
 pub use signer::{EmptySigner, InMemorySigner, Signer};
 

core/crypto/src/signature.rs

@@ -7,8 +7,10 @@ use std::str::FromStr;
 
 use borsh::{BorshDeserialize, BorshSerialize};
 use ed25519_dalek::ed25519::signature::{Signature as _, Signer as _, Verifier as _};
+use ethereum_types::U256;
 use lazy_static::lazy_static;
 use rand_core::OsRng;
+use secp256k1::Message;
 use serde::{Deserialize, Serialize};
 
 lazy_static! {
@@ -553,9 +555,64 @@ impl<'de> serde::Deserialize<'de> for SecretKey {
     }
 }
 
+const SECP256K1_N: U256 =
+    U256([0xbfd25e8cd0364141, 0xbaaedce6af48a03b, 0xfffffffffffffffe, 0xffffffffffffffff]);
+
+// Half of SECP256K1_N + 1.
+const SECP256K1_N_HALF_ONE: U256 =
+    U256([0xdfe92f46681b20a1, 0x5d576e7357a4501d, 0xffffffffffffffff, 0x7fffffffffffffff]);
+
 #[derive(Clone)]
 pub struct Secp256K1Signature([u8; 65]);
 
+impl Secp256K1Signature {
+    pub fn check_signature_values(&self, reject_upper: bool) -> bool {
+        let mut r_bytes = [0u8; 32];
+        r_bytes.copy_from_slice(&self.0[0..32]);
+        let r = U256::from(r_bytes);
+
+        let mut s_bytes = [0u8; 32];
+        s_bytes.copy_from_slice(&self.0[32..64]);
+        let s = U256::from(s_bytes);
+
+        let s_check = if reject_upper {
+            // Reject upper range of s values (ECDSA malleability)
+            SECP256K1_N_HALF_ONE
+        } else {
+            SECP256K1_N
+        };
+
+        r < SECP256K1_N && s < s_check
+    }
+
+    pub fn recover(
+        &self,
+        msg: [u8; 32],
+    ) -> Result<Secp256K1PublicKey, crate::errors::ParseSignatureError> {
+        let recoverable_sig = secp256k1::RecoverableSignature::from_compact(
+            &SECP256K1,
+            &self.0[0..64],
+            secp256k1::RecoveryId::from_i32(i32::from(self.0[64])).unwrap(),
+        )
+        .map_err(|err| crate::errors::ParseSignatureError::InvalidData {
+            error_message: err.to_string(),
+        })?;
+        let msg = Message::from(msg);
+
+        let res = SECP256K1
+            .recover(&msg, &recoverable_sig)
+            .map_err(|err| crate::errors::ParseSignatureError::InvalidData {
+                error_message: err.to_string(),
+            })?
+            .serialize_vec(&SECP256K1, false);
+
+        // Can not fail
+        let pk = Secp256K1PublicKey::try_from(&res[1..65]).unwrap();
+
+        Ok(pk)
+    }
+}
+
 impl From<[u8; 65]> for Secp256K1Signature {
     fn from(data: [u8; 65]) -> Self {
         Self(data)

core/primitives-core/Cargo.toml

@@ -24,7 +24,8 @@ lazy_static = "1.4"
 
 [features]
 default = []
-protocol_feature_add_account_versions = []
 protocol_feature_evm = []
+protocol_feature_add_account_versions = []
 protocol_feature_alt_bn128 = []
 protocol_feature_tx_size_limit = []
+protocol_feature_math_extension = []

core/primitives-core/src/config.rs

@@ -223,6 +223,17 @@ pub struct ExtCostsConfig {
     /// Cost of getting sha256 per byte
     pub keccak512_byte: Gas,
 
+    #[cfg(feature = "protocol_feature_math_extension")]
+    /// Cost of getting ripemd160 base
+    pub ripemd160_base: Gas,
+    #[cfg(feature = "protocol_feature_math_extension")]
+    /// Cost of getting ripemd160 per message block
+    pub ripemd160_block: Gas,
+
+    #[cfg(feature = "protocol_feature_math_extension")]
+    /// Cost of calling ecrecover
+    pub ecrecover_base: Gas,
+
     /// Cost for calling logging.
     pub log_base: Gas,
     /// Cost for logging per byte
@@ -353,6 +364,13 @@ impl Default for ExtCostsConfig {
             keccak256_byte: SAFETY_MULTIPLIER * 7157035,
             keccak512_base: SAFETY_MULTIPLIER * 1937129412,
             keccak512_byte: SAFETY_MULTIPLIER * 12216567,
+            #[cfg(feature = "protocol_feature_math_extension")]
+            ripemd160_base: SAFETY_MULTIPLIER * 284558362,
+            #[cfg(feature = "protocol_feature_math_extension")]
+            // Cost per byte is 3542227. There are 64 bytes in a block.
+            ripemd160_block: SAFETY_MULTIPLIER * 226702528,
+            #[cfg(feature = "protocol_feature_math_extension")]
+            ecrecover_base: SAFETY_MULTIPLIER * 1121789875000,
             log_base: SAFETY_MULTIPLIER * 1181104350,
             log_byte: SAFETY_MULTIPLIER * 4399597,
             storage_write_base: SAFETY_MULTIPLIER * 21398912000,
@@ -423,6 +441,12 @@ impl ExtCostsConfig {
             keccak256_byte: 0,
             keccak512_base: 0,
             keccak512_byte: 0,
+            #[cfg(feature = "protocol_feature_math_extension")]
+            ripemd160_base: 0,
+            #[cfg(feature = "protocol_feature_math_extension")]
+            ripemd160_block: 0,
+            #[cfg(feature = "protocol_feature_math_extension")]
+            ecrecover_base: 0,
             log_base: 0,
             log_byte: 0,
             storage_write_base: 0,
@@ -494,6 +518,12 @@ pub enum ExtCosts {
     keccak256_byte,
     keccak512_base,
     keccak512_byte,
+    #[cfg(feature = "protocol_feature_math_extension")]
+    ripemd160_base,
+    #[cfg(feature = "protocol_feature_math_extension")]
+    ripemd160_block,
+    #[cfg(feature = "protocol_feature_math_extension")]
+    ecrecover_base,
     log_base,
     log_byte,
     storage_write_base,
@@ -618,6 +648,12 @@ impl ExtCosts {
             keccak256_byte => config.keccak256_byte,
             keccak512_base => config.keccak512_base,
             keccak512_byte => config.keccak512_byte,
+            #[cfg(feature = "protocol_feature_math_extension")]
+            ripemd160_base => config.ripemd160_base,
+            #[cfg(feature = "protocol_feature_math_extension")]
+            ripemd160_block => config.ripemd160_block,
+            #[cfg(feature = "protocol_feature_math_extension")]
+            ecrecover_base => config.ecrecover_base,
             log_base => config.log_base,
             log_byte => config.log_byte,
             storage_write_base => config.storage_write_base,
@@ -692,6 +728,9 @@ impl ExtCosts {
             "keccak256_byte",
             "keccak512_base",
             "keccak512_byte",
+            "ripemd160_base",
+            "ripemd160_block",
+            "ecrecover_base",
             "log_base",
             "log_byte",
             "storage_write_base",

core/primitives/Cargo.toml

@@ -51,7 +51,8 @@ protocol_feature_fix_storage_usage = []
 protocol_feature_restore_receipts_after_fix = []
 protocol_feature_cap_max_gas_price = []
 protocol_feature_count_refund_receipts_in_gas_limit = []
-nightly_protocol_features = ["nightly_protocol", "protocol_feature_evm", "protocol_feature_block_header_v3", "protocol_feature_alt_bn128", "protocol_feature_add_account_versions", "protocol_feature_tx_size_limit", "protocol_feature_allow_create_account_on_delete", "protocol_feature_fix_storage_usage", "protocol_feature_restore_receipts_after_fix", "protocol_feature_cap_max_gas_price", "protocol_feature_count_refund_receipts_in_gas_limit"]
+protocol_feature_math_extension = ["near-primitives-core/protocol_feature_math_extension"]
+nightly_protocol_features = ["nightly_protocol", "protocol_feature_evm", "protocol_feature_block_header_v3", "protocol_feature_alt_bn128", "protocol_feature_add_account_versions", "protocol_feature_tx_size_limit", "protocol_feature_allow_create_account_on_delete", "protocol_feature_fix_storage_usage", "protocol_feature_restore_receipts_after_fix", "protocol_feature_cap_max_gas_price", "protocol_feature_count_refund_receipts_in_gas_limit", "protocol_feature_math_extension"]
 nightly_protocol = []
 
 [dev-dependencies]

core/primitives/src/version.rs

@@ -109,6 +109,8 @@ pub enum ProtocolFeature {
     CapMaxGasPrice,
     #[cfg(feature = "protocol_feature_count_refund_receipts_in_gas_limit")]
     CountRefundReceiptsInGasLimit,
+    #[cfg(feature = "protocol_feature_math_extension")]
+    MathExtension,
 }
 
 /// Current latest stable version of the protocol.
@@ -153,6 +155,8 @@ impl ProtocolFeature {
             ProtocolFeature::CapMaxGasPrice => 113,
             #[cfg(feature = "protocol_feature_count_refund_receipts_in_gas_limit")]
             ProtocolFeature::CountRefundReceiptsInGasLimit => 114,
+            #[cfg(feature = "protocol_feature_math_extension")]
+            ProtocolFeature::MathExtension => 114,
         }
     }
 }

nearcore/Cargo.toml

@@ -71,10 +71,14 @@ protocol_feature_add_account_versions = ["near-primitives/protocol_feature_add_a
 protocol_feature_tx_size_limit = ["near-primitives/protocol_feature_tx_size_limit", "node-runtime/protocol_feature_tx_size_limit"]
 protocol_feature_allow_create_account_on_delete = ["node-runtime/protocol_feature_allow_create_account_on_delete"]
 protocol_feature_fix_storage_usage = ["near-primitives/protocol_feature_fix_storage_usage", "node-runtime/protocol_feature_fix_storage_usage"]
-nightly_protocol_features = ["nightly_protocol", "near-primitives/nightly_protocol_features", "near-client/nightly_protocol_features", "near-epoch-manager/nightly_protocol_features", "near-store/nightly_protocol_features", "protocol_feature_evm", "protocol_feature_block_header_v3", "protocol_feature_alt_bn128", "protocol_feature_add_account_versions", "protocol_feature_tx_size_limit", "protocol_feature_allow_create_account_on_delete", "protocol_feature_fix_storage_usage", "protocol_feature_restore_receipts_after_fix", "protocol_feature_cap_max_gas_price"]
+nightly_protocol_features = ["nightly_protocol", "near-primitives/nightly_protocol_features", "near-client/nightly_protocol_features", "near-epoch-manager/nightly_protocol_features", "near-store/nightly_protocol_features", "protocol_feature_evm", "protocol_feature_block_header_v3", "protocol_feature_alt_bn128", "protocol_feature_add_account_versions", "protocol_feature_tx_size_limit", "protocol_feature_allow_create_account_on_delete", "protocol_feature_fix_storage_usage", "protocol_feature_restore_receipts_after_fix", "protocol_feature_cap_max_gas_price", "protocol_feature_math_extension"]
 nightly_protocol = ["near-primitives/nightly_protocol", "near-jsonrpc/nightly_protocol"]
 protocol_feature_restore_receipts_after_fix = ["near-primitives/protocol_feature_restore_receipts_after_fix", "near-chain/protocol_feature_restore_receipts_after_fix", "node-runtime/protocol_feature_restore_receipts_after_fix"]
 protocol_feature_cap_max_gas_price = ["near-primitives/protocol_feature_cap_max_gas_price", "near-chain/protocol_feature_cap_max_gas_price"]
+protocol_feature_math_extension = [
+    "near-primitives/protocol_feature_math_extension",
+    "node-runtime/protocol_feature_math_extension"
+]
 
 # enable this to build neard with wasmer 1.0 runner
 # now if none of wasmer0_default, wasmer1_default or wasmtime_default is enabled, wasmer0 would be default

neard/Cargo.toml

@@ -51,6 +51,7 @@ nightly_protocol_features = ["nearcore/nightly_protocol_features"]
 nightly_protocol = ["nearcore/nightly_protocol"]
 protocol_feature_restore_receipts_after_fix = ["nearcore/protocol_feature_restore_receipts_after_fix"]
 protocol_feature_cap_max_gas_price = ["nearcore/protocol_feature_cap_max_gas_price"]
+protocol_feature_math_extension = ["nearcore/protocol_feature_math_extension"]
 
 sandbox = ["nearcore/sandbox"]
 

runtime/near-vm-errors/Cargo.toml

@@ -22,6 +22,7 @@ near-rpc-error-macro = { path = "../../tools/rpctypegen/macro", version = "0.1.0
 
 [features]
 dump_errors_schema = ["near-rpc-error-macro/dump_errors_schema"]
+protocol_feature_math_extension = []
 protocol_feature_alt_bn128 = []
 
 [package.metadata.workspaces]

runtime/near-vm-errors/src/lib.rs

@@ -212,6 +212,9 @@ pub enum HostError {
     /// Serialization error for alt_bn128 functions
     #[cfg(feature = "protocol_feature_alt_bn128")]
     AltBn128SerializationError { msg: String },
+    /// General errors for ECDSA recover.
+    #[cfg(feature = "protocol_feature_math_extension")]
+    ECRecoverError { msg: String },
 }
 
 /// Errors specifically from native EVM.
@@ -498,6 +501,8 @@ impl std::fmt::Display for HostError {
             AltBn128DeserializationError { msg } => write!(f, "AltBn128 deserialization error: {}", msg),
             #[cfg(feature = "protocol_feature_alt_bn128")]
             AltBn128SerializationError { msg } => write!(f, "AltBn128 serialization error: {}", msg),
+            #[cfg(feature = "protocol_feature_math_extension")]
+            ECRecoverError { msg } => write!(f, "ECDSA recover error: {}", msg),
         }
     }
 }

runtime/near-vm-logic/Cargo.toml

@@ -17,10 +17,12 @@ base64 = "0.13"
 borsh = "0.8.1"
 bs58 = "0.4"
 byteorder = "1.2"
+ripemd160 = "0.9.0"
 serde = { version = "1", features = ["derive"] }
 sha2 = ">=0.8,<0.10"
 sha3 = ">=0.8,<0.10"
 
+near-crypto = { path = "../../core/crypto" }
 near-primitives = { path = "../../core/primitives" }
 near-primitives-core = { path = "../../core/primitives-core", version = "0.1.0" }
 near-vm-errors = { path = "../near-vm-errors", version = "3.0.0" }
@@ -36,6 +38,7 @@ default = []
 protocol_feature_evm = ["near-primitives-core/protocol_feature_evm"]
 protocol_feature_alt_bn128 = ["bn", "near-primitives-core/protocol_feature_alt_bn128", "near-vm-errors/protocol_feature_alt_bn128"]
 protocol_feature_allow_create_account_on_delete = []
+protocol_feature_math_extension = ["near-primitives/protocol_feature_math_extension", "near-vm-errors/protocol_feature_math_extension"]
 
 # Use this feature to enable counting of fees and costs applied.
 costs_counting = []

runtime/near-vm-logic/src/gas_counter.rs

@@ -144,18 +144,18 @@ impl GasCounter {
         self.deduct_gas(value, value)
     }
 
-    /// A helper function to pay per byte gas
-    pub fn pay_per_byte(&mut self, cost: ExtCosts, num_bytes: u64) -> Result<()> {
-        let use_gas = num_bytes
+    /// A helper function to pay a multiple of a cost.
+    pub fn pay_per(&mut self, cost: ExtCosts, num: u64) -> Result<()> {
+        let use_gas = num
             .checked_mul(cost.value(&self.ext_costs_config))
             .ok_or(HostError::IntegerOverflow)?;
 
-        self.inc_ext_costs_counter(cost, num_bytes);
+        self.inc_ext_costs_counter(cost, num);
         self.update_profile_host(cost, use_gas);
         self.deduct_gas(use_gas, use_gas)
     }
 
-    /// A helper function to pay base cost gas
+    /// A helper function to pay base cost gas.
     pub fn pay_base(&mut self, cost: ExtCosts) -> Result<()> {
         let base_fee = cost.value(&self.ext_costs_config);
         self.inc_ext_costs_counter(cost, 1);