Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merkle Proof Verification #10

Merged
merged 63 commits into from
Aug 31, 2022
Merged
Show file tree
Hide file tree
Changes from 54 commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
ae1eb94
Add libraries and merkle proof directories
Jul 18, 2022
13da239
Add merkle proof verification function
Jul 18, 2022
1ad21c9
Add forc.toml
Jul 18, 2022
dea732a
Library setup
Jul 20, 2022
ab1f021
Add test setup
Jul 20, 2022
4ab2c31
Add test outline
Jul 20, 2022
ada728f
Merge branch 'master' into bitzoic-6
Jul 21, 2022
656fa9f
Update SDK v0.18
Jul 21, 2022
f09797d
Add Mutli Merkle Proof
Jul 28, 2022
f963b00
Add documentation
Jul 28, 2022
129b06e
Minor updates
Aug 2, 2022
409ac06
Merge Master
Aug 9, 2022
fec7618
Update function definitions to take array
Aug 9, 2022
acb1acd
Add merkle_tree to tests
Aug 9, 2022
994c4ed
Add merkle proof test
Aug 10, 2022
4211321
Add fail merkle proof test
Aug 10, 2022
b656edc
Add Multi Merkle Proof Test
Aug 11, 2022
2fd6e72
Add Fails Multi Merkle Proof Test
Aug 11, 2022
db03afe
Add Multi Merkle proof guard
Aug 11, 2022
0fb67b7
Add comments and TODOs
Aug 11, 2022
5a63f43
Add MultiProof Error Enum
Aug 11, 2022
dfc7da1
Update arithmetic
Aug 11, 2022
cd2b809
Add Invalid Multi Merkle Proof Test
Aug 11, 2022
451b1e9
Add Merkle Proof unwrapping None Test
Aug 11, 2022
970320d
Run formatter
Aug 11, 2022
f4db1da
Add forc to Cargo.toml
Aug 11, 2022
3a08c7c
Move build.sh
Aug 11, 2022
aaac086
Update build.sh
Aug 11, 2022
4345d17
Remove lock from build.sh
Aug 11, 2022
3630e11
Update build.sh AGAIN
Aug 11, 2022
15bff3b
Add fuel-core
Aug 11, 2022
2418fff
Fix fuel-core services
Aug 11, 2022
8db2efd
Reverting fuel-core because failing CI
Aug 11, 2022
5751db1
Add comment and remove Forc.lock
Aug 11, 2022
d5ee2fa
Refactor to split process and verify functions
Aug 12, 2022
8b33d85
Add tests for process single and multi merkle proof
Aug 12, 2022
7e30c6f
Bump SDK to v0.20
Aug 12, 2022
2cd2a62
Updated based on pr review suggestions
Aug 12, 2022
1ec4936
Merge branch 'master' into bitzoic-6
bitzoic Aug 12, 2022
0e649c1
Update proof to reflect RFC-6962
Aug 19, 2022
04a1671
Update Cargo.toml to circumnavigate fuel-core issues
Aug 19, 2022
56e3904
Update tests to use fuel-merkle repo
Aug 19, 2022
08fdcc9
Remove multi-merkle proof
Aug 19, 2022
eb09fdd
Remove multi-merkle proof tests
Aug 19, 2022
25f569c
Rename to binary merkle proof
Aug 19, 2022
25bde44
Add manual proof tests
Aug 19, 2022
d4fff0b
Add leaf_digest and node_digest tests
Aug 19, 2022
c78f6b9
Add path_length_from_key function
Aug 19, 2022
68577eb
Fix formatting issues
Aug 19, 2022
9df9601
Add README.md
Aug 23, 2022
3602eed
Add SPECIFICATION.md
Aug 23, 2022
609b189
Add more comments
Aug 23, 2022
fc23195
Commit suggestions
Aug 24, 2022
72a6a51
Break tests into smaller modules
Aug 24, 2022
a6861ef
Commit suggestions
Aug 25, 2022
4e0b688
Commit suggestions
Aug 25, 2022
ed9c5a3
Update comment
Aug 25, 2022
13f100f
Update README and SPECIFICATION on suggestions
Aug 30, 2022
3b78848
More README and SPECIFICATION updates
Aug 30, 2022
d58d656
Update test based on suggestions
Aug 30, 2022
dd7ba44
Add LEAF and NODE comment
Aug 30, 2022
a5e44fe
Run fmt
Aug 30, 2022
a3e7c4f
Fix typos
Aug 31, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Syntax highlighting of sway files as rust
*.sw linguist-language=Rust
13 changes: 10 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,19 @@ These libraries contain helper functions, generalized standards, and other tools

### Libraries


- [Binary Merkle Proof](./sway_libs/src/merkle_proof/) is used to verify Binary Merkle Trees computed off-chain.

## Using a library

If you wish to use any of the libraries then clone this repository and go through the general [installation](https://fuellabs.github.io/sway/latest/introduction/installation.html) steps required to use our tools, and import the library into your contract.
To import the Merkle Proof library the following should be added to the project's `Forc.toml` file under `[dependencies]`:

<!-- TODO: This should not point to the master branch but instead to a release -->
`sway_libs = { git = "https://github.com/FuelLabs/sway-libs", branch = "master" }`
bitzoic marked this conversation as resolved.
Show resolved Hide resolved

You may then import your dersired library in your Sway Smart Contract as so:
bitzoic marked this conversation as resolved.
Show resolved Hide resolved

`use sway_libs::binary_merkle_proof::verify_proof;`

## Contributing

Check [CONTRIBUTING.md](./CONTRIBUTING.md) for more info!
Check [CONTRIBUTING.md](./CONTRIBUTING.md) for more info!
bitzoic marked this conversation as resolved.
Show resolved Hide resolved
2 changes: 2 additions & 0 deletions sway_libs/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
out
target
2 changes: 2 additions & 0 deletions sway_libs/src/lib.sw
Original file line number Diff line number Diff line change
@@ -1 +1,3 @@
library sway_libs;

dep merkle_proof/binary_merkle_proof;
92 changes: 92 additions & 0 deletions sway_libs/src/merkle_proof/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# Overview

Merkle trees allow for on-chain verification of off-chain data. With the merkle root posted on-chain, the generation of proofs off-chain can provide verifibly true data.

More information can be found in the [specification](./SPECIFICATION.md).

## Known Issues

Due to the Fuel VM padding of copy types smaller than one word, the current hashing of the leaf and node values do not match RFC-6962. The present implementation is the equavalent of hashing a `u64` as opposed to the standard `u8`. As a result, hashing of smaller sized copy types may result in unexpected behavior. For example, the `sha-256(0u8)` will result in the same hash as `sha-256(0u64)`. This will cause an incompatability with the [Fuel-Merkle](https://github.com/FuelLabs/fuel-merkle) repository. The issue regarding this can be tracked [here](https://github.com/FuelLabs/sway/issues/2594).

# Using the Library

## Using the Merkle Proof Library In Sway

### Importing Into Your Project
bitzoic marked this conversation as resolved.
Show resolved Hide resolved

To import the Merkle Proof library the following should be added to the project's `Forc.toml` file under `[dependencies]`:

<!-- TODO: This should not point to the master branch but instead to a release -->
```
sway_libs = { git = "https://github.com/FuelLabs/sway-libs", branch = "master" }
```

### Importing Into Your Contract

The following can be added to your Sway Contract's imports to include the Merkle Proof library:

```
use sway_libs::binary_merkle_proof::verify_proof;
```

### Using the Merkle Proof Library In Sway

Once imported, using the Merkle Proof library is as simple as calling the desired function. Here is a list of function definitions that you may use:

- `leaf_digest(data: b256) -> b256`
- `node_digest(left: b256, right: b256) -> b256`
- `process_proof(key: u64, merkle_leaf: b256, num_leaves: u64, proof: [b256; 2]) -> b256`
- `verify_proof(key: u64, merkle_leaf: b256, merkle_root: b256, num_leaves: u64, proof: [b256; 2]) -> bool`

> **NOTE** Fuel-RS currently does not support the Sway standard library's `vec` as a function argument. As a result, the current implementation uses an array.
bitzoic marked this conversation as resolved.
Show resolved Hide resolved

## Using the Merkle Proof Libaray in Fuels-RS
bitzoic marked this conversation as resolved.
Show resolved Hide resolved

To generate a Merkle Tree and corresponding proof for your Sway Smart Contract to use can be done with the [Fuel-Merkle](https://github.com/FuelLabs/fuel-merkle) crate.

### Importing Into Your Project

The import the Fuel-Merkle crate, the following should be added to the project's `Cargo.toml` file under `[dependencies]`:

```
fuel-merkle = { version = "0.3" }
```

### Importing Into Your Rust File

The following should be added to your Rust file to use the Fuel-Merkle crate.

```
use fuel_merkle::binary::in_memory::MerkleTree;
```

### Using Fuel-Merkle

#### Generating A Tree

To create a merkle tree using Fuel-Merkle is as simple as pushing your leaves in increasing order.

```
let mut tree = MerkleTree::new();
let leaves = ["A".as_bytes(), "B".as_bytes(), "C".as_bytes()].to_vec();
for datum in leaves.iter() {
tree.push(datum);
}
```

#### Generating And Verifying A Proof

To generate a proof for a specific leaf, you must have the index or key of the leaf. Simply call the prove function:

```
let mut proof = tree.prove(key).unwrap();
```

Once the proof has been generated, you may call the Sway Smart Contract's `verify_proof` function:

```
let merkle_root = proof.0;
let merkle_leaf = proof.1[0];
proof.1.remove(0);
contract_instance.verify_proof(key, merkle_leaf, merkle_root, num_leaves, proof.1).call().await;
```
27 changes: 27 additions & 0 deletions sway_libs/src/merkle_proof/SPECIFICATION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Overview

This document provides an overview of the Merkle Proof library.

It outlines the use cases, i.e. specification, and describes how to implement the library.

# Use Cases

A common use case for Merkle Tree verification is airdrops. An airdrop is a method of distribution a set amount of tokens to a specified number of users. These often include a list of addresses and amounts. By posting the root hash, users can provide a proof and claim their airdrop.

# Specification

All cryptographic primitives follow the [Fuel Specs](https://github.com/FuelLabs/fuel-specs/blob/master/specs/protocol/cryptographic_primitives.md).

## Hashing

All hashing is done with SHA-2-256 (also known as SHA-256), defined in [FIPS 180-4](https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.180-4.pdf).

## Merkle Trees

Currently only one tree structure is support, the Binary Merkle Tree. Implementation for a Binary Merkle Sum Tree and Sparse Merkle Tree found in the [fuel-merkle](https://github.com/FuelLabs/fuel-merkle) repository will be added soon.
bitzoic marked this conversation as resolved.
Show resolved Hide resolved
bitzoic marked this conversation as resolved.
Show resolved Hide resolved

### Binary Merkle Tree

Binary Merkle trees are constructed in the same fashion as described in [Certificate Transparency (RFC-6962)](https://tools.ietf.org/html/rfc6962). Leaves are hashed once to get leaf node values and internal node values are the hash of the concatenation of their children (either leaf nodes or other internal nodes).

For more information please check out the offical [Fuel Specs](https://github.com/FuelLabs/fuel-specs/blob/master/specs/protocol/cryptographic_primitives.md#binary-merkle-tree).
175 changes: 175 additions & 0 deletions sway_libs/src/merkle_proof/binary_merkle_proof.sw
Original file line number Diff line number Diff line change
@@ -0,0 +1,175 @@
// TODO: Function definitions that use arrays should be updated to Vecs once
// https://github.com/FuelLabs/fuels-rs/issues/353 is resolved

library binary_merkle_proof;

use std::{hash::sha256, revert::require};

pub enum ProofError {
InvalidKey: (),
InvalidProofLength: (),
}

pub const LEAF = 0u8;
bitzoic marked this conversation as resolved.
Show resolved Hide resolved
pub const NODE = 1u8;

/// Returns the computed leaf hash of "MTH({d(0)}) = SHA-256(0x00 || d(0))".
///
/// # Arguments
///
/// * 'data' - The hash of the leaf data.
pub fn leaf_digest(data: b256) -> b256 {
sha256((LEAF, data))
}

/// Returns the computed node hash of "MTH(D[n]) = SHA-256(0x01 || MTH(D[0:k]) || MTH(D[k:n]))".
///
/// # Arguments
///
/// * 'left' - The hash of the left node.
/// * 'right' - The hash of the right node.
pub fn node_digest(left: b256, right: b256) -> b256 {
sha256((NODE, left, right))
}

/// Calculates the length of the path to a leaf
///
/// # Arguments
///
/// * `key` - The key or index of the leaf.
/// * `num_leaves` - The total number of leaves in the merkle tree.
fn path_length_from_key(key: u64, num_leaves: u64) -> u64 {
let mut total_length = 0;
let mut num_leaves = num_leaves;
let mut key = key;

while true {
// The height of the left subtree is equal to the offset of the starting bit of the path
let path_length = starting_bit(num_leaves);
// Determine the number of leaves in the left subtree
let num_leaves_left_sub_tree = (1 <<(path_length - 1));

if key <= (num_leaves_left_sub_tree - 1) {
bitzoic marked this conversation as resolved.
Show resolved Hide resolved
// If the leaf is in the left subtreee, path length is full height of the left subtree
total_length = total_length + path_length;
break;
} else if num_leaves_left_sub_tree == 1 {
// If the left sub tree has only one leaf, path has one additional step
total_length = total_length + 1;
break;
} else if (num_leaves - num_leaves_left_sub_tree) <= 1 {
bitzoic marked this conversation as resolved.
Show resolved Hide resolved
// If the right sub tree only has one leaf, path has one additonal step
total_length = total_length + 1;
break;
} else {
// Otherwise add 1 to height and loop
total_length = total_length + 1;
key = key - num_leaves_left_sub_tree;
num_leaves = num_leaves - num_leaves_left_sub_tree;
}
}

total_length
}

/// This function will compute and return a merkle root given a leaf and corresponding proof.
///
/// # Arguments
///
/// * 'key' - The key or index of the leaf to prove.
/// * `merkle_leaf` - The hash of a leaf on the merkle tree.
bitzoic marked this conversation as resolved.
Show resolved Hide resolved
/// * 'num_leaves' - The number of leaves in the merkle tree.
/// * `proof` - The merkle proof that will be used to traverse the merkle tree and compute a root.
///
/// # Reverts
///
/// * When an incorrect proof length is provided.
/// * When there is one or no leaves and a proof is provided.
/// * When the key is greater than or equal to the number of leaves.
/// * When the computed height gets larger than the proof.
pub fn process_proof(key: u64, merkle_leaf: b256, num_leaves: u64, proof: [b256;
2]) -> b256 {
// let proof_length = proof.len();
let proof_length = 2;
require((num_leaves > 1 && proof_length == path_length_from_key(key, num_leaves)) || (num_leaves <= 1 && proof_length == 0), ProofError::InvalidProofLength);
require(key < num_leaves, ProofError::InvalidKey);

let mut digest = merkle_leaf;
// If there is no proof then the leaf is the root
bitzoic marked this conversation as resolved.
Show resolved Hide resolved
if proof_length == 0 {
return digest
}

let mut height = 1;
let mut stable_end = key;

// While the current subtree is complete, determine the position of the next
// sibling using the complete subtree algorithm.
while true {
// Determine if the subtree is complete.
let sub_tree_start_index = (key / (1 << height)) * (1 << height);
let sub_tree_end_index = sub_tree_start_index + (1 << height) - 1;

// If the Merkle tree does not have a leaf at the `sub_tree_end_index`, we deem that the
// subtree is not complete.
if sub_tree_end_index >= num_leaves {
break;
}
stable_end = sub_tree_end_index;
require(proof_length > height - 1, ProofError::InvalidProofLength);

// Determine if the key is in the first or the second half of the subtree.
if (key - sub_tree_start_index) <(1 <<(height - 1)) {
digest = node_digest(digest, proof[height - 1]);
} else {
digest = node_digest(proof[height - 1], digest);
}

height = height + 1;
}

// Determine if the next hash belongs to an orphan that was elevated.
if stable_end != (num_leaves - 1) {
require(proof_length > height - 1, ProofError::InvalidProofLength);
digest = node_digest(digest, proof[height - 1]);
height = height + 1;
}

// All remaining elements in the proof set will belong to the left sibling.
while(height - 1) < proof_length {
digest = node_digest(proof[height - 1], digest);
height = height + 1;
}

digest
}

/// Calculates the starting bit of the path to a leaf
///
/// # Arguments
///
/// * `num_leaves` - The number of leaves in the merkle tree.
fn starting_bit(num_leaves: u64) -> u64 {
let mut starting_bit = 0;

while(1 << starting_bit) < num_leaves {
starting_bit = starting_bit + 1;
}

starting_bit
}

/// This function will take a merkle leaf and proof and return whether the corresponding root
/// matches the root given.
///
/// # Arguments
///
/// * 'key' - The key or index of the leaf to verify.
/// * `merkle_leaf` - The hash of a leaf on the merkle tree.
/// * `merkle_root` - The pre-computed merkle root that will be used to verify the leaf and proof.
/// * 'num_leaves' - The number of leaves in the merkle tree.
/// * `proof` - The merkle proof that will be used to traverse the merkle tree and compute a root.
pub fn verify_proof(key: u64, merkle_leaf: b256, merkle_root: b256, num_leaves: u64, proof: [b256;
2]) -> bool {
process_proof(key, merkle_leaf, num_leaves, proof) == merkle_root
}
7 changes: 5 additions & 2 deletions tests/src/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,11 @@ edition = "2021"
license = "Apache-2.0"

[dependencies]
forc = { version = "0.19" }
fuels = { version = "0.20" }
forc = { version = "0.19.1" }
fuel-core = { version = "0.9.6 "}
fuel-merkle = { version = "0.3" }
fuels = { version = "0.19" }
sha2 = { version = "0.10" }
tokio = { version = "1.12", features = ["rt", "macros"] }

[[test]]
Expand Down
4 changes: 3 additions & 1 deletion tests/src/test_projects/harness.rs
Original file line number Diff line number Diff line change
@@ -1 +1,3 @@
// Add test modules here:
// Add test modules here:

mod merkle_proof;
2 changes: 2 additions & 0 deletions tests/src/test_projects/merkle_proof/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
out
target
8 changes: 8 additions & 0 deletions tests/src/test_projects/merkle_proof/Forc.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
[project]
authors = ["Fuel Labs <contact@fuel.sh>"]
entry = "main.sw"
license = "Apache-2.0"
name = "merkle_proof"

[dependencies]
sway_libs = { path = "../../../../sway_libs" }
1 change: 1 addition & 0 deletions tests/src/test_projects/merkle_proof/mod.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
mod tests;
Loading