-
Notifications
You must be signed in to change notification settings - Fork 140
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
NEP-364: Efficient signature verification host functions #364
Changes from 14 commits
34a9391
1d4dc44
56d08f0
0129422
8962147
67ac13b
055bd09
70265a5
d6c0771
a7789b3
1d2777f
479edff
5ed5b18
1e8d3df
80ac231
f3b5c39
e3af2cb
ba8a3d3
bdf64d0
59257ca
d25d8f8
3dac4ea
73e0290
003493e
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,180 @@ | ||
--- | ||
NEP: 364 | ||
Title: Efficient signature verification and hashing precompile functions | ||
Author: Blas Rodriguez Irizar <rodrigblas@gmail.com> | ||
DiscussionsTo: https://github.com/nearprotocol/neps/pull/364 | ||
Status: Draft | ||
Type: Runtime Spec | ||
Category: Contract | ||
Created: 15-Jun-2022 | ||
--- | ||
|
||
## Summary | ||
|
||
This NEP introduces the request of adding into the NEAR runtime a pre-compiled | ||
function used to verify signatures that can help IBC compatible light clients run on-chain. | ||
|
||
## Motivation | ||
|
||
Signature verification and hashing are ubiquitous operations in light clients, | ||
especially in PoS consensus mechanisms. Based on Polkadot's consensus mechanism | ||
there will be a need for verification of ~200 signatures every minute | ||
(Polkadot’s authority set is ~300 signers and it may be increased in the future: https://polkadot.polkassembly.io/referendum/16). | ||
|
||
Therefore, a mechanism to perform these operations cost-effectively in terms | ||
of gas and speed would be highly beneficial to have. Currently, NEAR does not have any native signature verification toolbox. | ||
This implies that a light client operating inside NEAR will have to import a library | ||
compiled to WASM as mentioned in [Zulip](https://near.zulipchat.com/#narrow/stream/295302-general/topic/light_client). | ||
|
||
Polkadot uses [three different cryptographic schemes](https://wiki.polkadot.network/docs/learn-keys) | ||
for its keys/accounts, which also translates into different signature types. However, for this NEP the focus is on: | ||
|
||
- The vanilla ed25519 implementation uses Schnorr signatures. | ||
|
||
## Rationale and alternatives | ||
|
||
Add a signature verification signatures function into the runtime as host functions. | ||
|
||
- ED25519 signature verification function using `ed25519_dalek` crates into NEAR runtime as pre-compiled functions. | ||
|
||
Benchmarks were run using a signature verifier smart contract on-chain importing the aforementioned functions from | ||
widespread used crypto Rust crates. The biggest pitfall of these functions running wasm code instead of native | ||
is performance and gas cost. Our [benchmarks](https://github.com/blasrodri/near-test) show the following results: | ||
|
||
```log | ||
near call sitoula-test.testnet verify_ed25519 '{"signature_p1": [145,193,203,18,114,227,14,117,33,213,121,66,130,14,25,4,36,120,46,142,226,215,7,66,122,112,97,30,249,135,61,165], "signature_p2": [221,249,252,23,105,40,56,70,31,152,236,141,154,122,207,20,75,118,79,90,168,6,221,122,213,29,126,196,216,104,191,6], "msg": [107,97,106,100,108,102,107,106,97,108,107,102,106,97,107,108,102,106,100,107,108,97,100,106,102,107,108,106,97,100,115,107], "iterations": 10}' --accountId sitoula-test.testnet --gas 300000000000000 | ||
# transaction id DZMuFHisupKW42w3giWxTRw5nhBviPu4YZLgKZ6cK4Uq | ||
``` | ||
|
||
With `iterations = 130` **all these calls return ExecutionError**: `'Exceeded the maximum amount of gas allowed to burn per contract.'` | ||
With iterations = 50 these are the results: | ||
Comment on lines
+49
to
+50
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is one iteration one signature verification? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I've added a reference in the motivation section. Would you like it to be more specific, being included in the comment of each host function? |
||
|
||
``` | ||
ed25519: tx id 6DcJYfkp9fGxDGtQLZ2m6PEDBwKHXpk7Lf5VgDYLi9vB (299 Tgas) | ||
``` | ||
|
||
- Performance in wall clock time when you compile the signature validation library directly from rust to native. | ||
Here are the results on an AMD Ryzen 9 5900X 12-Core Processor machine: | ||
|
||
``` | ||
# 10k signature verifications | ||
ed25519: took 387ms | ||
``` | ||
|
||
- Performance in wall clock time when you compile the library into wasm first and then use the single-pass compiler in Wasmer 1 to then compile to native. | ||
|
||
``` | ||
ed25519: took 9926ms | ||
``` | ||
|
||
As an extra data point, when passing `--enable-simd` instead of `--singlepass` | ||
|
||
``` | ||
ed25519: took 3085ms | ||
``` | ||
|
||
Steps to reproduce: | ||
commit: `31cf97fb2e155d238308f062c4b92bae716ac19f` in `https://github.com/blasrodri/near-test` | ||
|
||
```sh | ||
# wasi singlepass | ||
cargo wasi build --bin benches --release | ||
wasmer compile --singlepass ./target/wasm32-wasi/release/benches.wasi.wasm -o benches_singlepass | ||
wasmer run ./benches_singlepass | ||
``` | ||
|
||
```sh | ||
# rust native | ||
cargo run --bin benches --release | ||
``` | ||
|
||
Overall: the difference between the two versions (native vs wasi + singlepass is) | ||
|
||
``` | ||
ed25519: 25.64x slower | ||
``` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. can you also mention is the expected improvement after this NEP ? How many verifications are you able to fit in a 300 TGas transaction ? |
||
|
||
### What is the impact of not doing this? | ||
|
||
Costs of running IBC-compatible trustless bridges would be very high. Plus, the fact that signature verification | ||
is such an expensive operation will force the contract to split the process of batch verification of signatures | ||
into multiple transactions. | ||
|
||
### Why is this design the best in the space of possible designs? | ||
|
||
Adding existing proved and vetted crypto crates into the runtime is a safe workaround. It will boost performance | ||
between 20-25x according to our benchmarks. This will both reduce operating costs significantly and will also | ||
enable the contract to verify all the signatures in one transaction, which will simplify the contract design. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If Polkadot uses 3 different types of keys - are you still able to verify all the signatures in one transaction ? (for example in a scenario, where all the polkadot validators are using sr25519 ? ) |
||
|
||
### What other designs have been considered and what is the rationale for not choosing them? | ||
|
||
One possible alternative would be to improve the runtime implementation so that it can compile WASM code to a sufficiently | ||
fast machine code. Even when it may not be as fast as LLVM native produced code it could still be acceptable for | ||
these types of use cases (CPU intensive functions) and will certainly avoid the need of adding host functions. | ||
The effort of adding such a feature will be significantly higher than adding these host functions one by one. | ||
But on the other side, it will decrease the need of including more host functions in the future. | ||
|
||
Another alternative is to deal with the high cost of computing/verifying these signatures in some other manner. | ||
Decreasing the overall cost of gas and increasing the limits of gas available to attach to the contract could be a possibility. | ||
Introducing such modification for some contracts, and not for some others can be rather arbitrary | ||
and not straightforward in the implementation, but an alternative nevertheless. | ||
|
||
## Specification | ||
nagisa marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
This NEP aims to introduce the following host function: | ||
|
||
```rust | ||
/// Ed25519 is a public-key signature system with several attractive features | ||
/// | ||
/// Proof of Stake Validator sets can contain different signature schemes. | ||
/// Ed25519 is one of the most used ones across blockchains, and hence it's importance to be added. | ||
/// For further reference, visit: https://ed25519.cr.yp.to | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I wonder if we should add more inline information about how The motivation is to allow potential new client implementations to be able to build the client by only reading Nomicon and NEPs. |
||
/// # Cost | ||
/// | ||
/// `base + write_register_base + write_register_byte * num_bytes + ed25519_verify_base + ed25519_verify_byte * num_bytes` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Where did these gas costs come from? Please see my comment on the implementation PR and update accordingly. near/nearcore#7165 (comment) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks! Will update this as soon as the PR with the implementation reflects the right costs. |
||
pub fn ed25519_verify( | ||
&mut self, | ||
sig_len: u64, | ||
sig_ptr: u64, | ||
msg_len: u64, | ||
msg_ptr: u64, | ||
pub_key_len: u64, | ||
pub_key_ptr: u64, | ||
) -> Result<u64>; | ||
``` | ||
|
||
And a `rust-sdk` possible implementation could look like this: | ||
|
||
```rs | ||
|
||
pub fn ed25519_verify(sig: &ed25519::Signature, msg: &[u8], pub_key: &ed25519::Public) -> bool; | ||
|
||
``` | ||
|
||
Once this NEP is approved and integrated, these functions will be available in the `near_sdk` crate in the | ||
`env` module. | ||
|
||
## Security Implications (Optional) | ||
|
||
We have chosen these crates because they are the ones already integrated into the | ||
[sp_io](https://docs.rs/sp-io/latest/sp_io/)](https://docs.rs/sp-io/latest/sp_io/) | ||
crate heavily used in the [substrate](https://github.com/paritytech/substrate) repository. | ||
|
||
## Unresolved Issues (Optional) | ||
|
||
- What parts of the design do you expect to resolve through the NEP process before this gets merged? | ||
Both the function signatures and crates are up for discussion. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I suggest that we use the library that nearcore already uses for ed25519 signatures. This would avoid introducing new dependencies and ensure that NEAR signatures themselves can be 100% verified in smart contracts There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As far as I can see the proposal is the same as the one being used today |
||
|
||
## Future possibilities | ||
|
||
I currently do not envision any extension in this regard. | ||
|
||
## Copyright | ||
|
||
[copyright]: #copyright | ||
|
||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/). | ||
|
||
``` | ||
|
||
``` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please set the correct number
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
unrelated to this specific PR, but I see a process issue (so, cc @frol): our process for NEPs should not require modification of shared files (like readme.md), as that is very prone to conflicts.