Feat/whitelist merkletree #2

[kromsten]

Whitelist Merkle-Tree contract

A whitelist contract that relies on Merkle-Tree data structure for verifying inclusion of an address in a whitelist.

Only merkle root (and optionaly URI of a tree) are stored within the state. Inclusion can be verified by submitting a user address and hex-encoded list of merkle proofs. This approach allows significant reduction of gas usage during storage phase with a downside of having actual data off-chain and reliance on 3rd parties for providing inclusions proofs.

Inclusion operation is a slighly more complex and costly when compared to the standard map-based whitelist. The contract uses Sha256 for hashing concatenated proofs. Hashes are sorted on byte level prior to concatenation, which significantly simplifies the verification process by not requiring submission of leaf positions.

Important: Make sure that your algorithm for merkle-tree construction also sort the hashes. See example of extending rs-merkle library in tests/hasher.rs

Upgraded Vending Minter

The Vending Minter contract was upgraded to support merkle proofs as an argument for the mint function. The minter can be used to call both regular and the merkle-tree based whitelist contracts. The assumption is based on passage of merkle proofs as an argument, but also on the queried configuration of a whitelist contract. Having member_limit and num_members equal to zero, which is forbidden in regular whitelists, returned in the response object tells the minter that the whitelist uses merkle-trees.

Gas Usage

The contracts for the merkle-tree based whitelist and the updated minter that supports it were both deployed to the testnet to measure actual gas usage in production. The contracts were instantiated and tested with two different whitelist sizes: 703 and 91,750,400 entries

Instantiating

Naturally due to only needing to store a merkle-tree root in the state of the contract there is no difference between instantiating a whitelist with the smaller and the bigger list sizes and they both consume 190,350 units of gas.

Minting

Number of hashing operations required to check for inclusion of an address in a merkle-tree is at most Math.ceil[ log₂N ] and in some cases even smaller depending on the depth of a leaf within a tree.

In case of the smaller tree with 704 records it was needed to pass 8 hash proofs and an example mint transaction took 635,345 units of gas

The bigger tree with ~90 million records used 647,448 units of gas and required 24 proofs only (up to 27 with deeper leaves).

The jump from computing 8 to computing 24 proofs (+16) only took additional 8 thousands units of gas. Keep in mind that another increase in 16 proofs allow us to check for inclusion in a tree with 1 trillion addresses.

[kromsten]

Unlike the previous PR #429 attempting to propose the changes, this implementation doesn't alter the factory contract removing the need for another instantiation / migration to a new factory.

Mentioned in:
#373

[jhernandezb]

Is there a way to store the per address limit as part of the proof ? eg X address has limit of 2, Y address has limit of 5? We use this type of whitelist for our "flex" minter

Also I think we should allow modifying the root as long as end_time> now()

[kromsten]

Is there a way to store the per address limit as part of the proof ? eg X address has limit of 2, Y address has limit of 5? We use this type of whitelist for our "flex" minter

Having it as part of a proof would be problematic. Having additional data for each member and hashing a "member structure" instead of a simply hashing an address is something I wanted to do initially.

There is no challenge with generating a tree like that but proof verification won't be trivial.

It ether requires the whitelist contract to "brute force" the inclusion algorithm with all possible permutation for the member struct or it needs an extra payload from the "minter" essentially requiring the latter to be knowledgeable about the extra fields.

I believe it's already the case with "flex-minter" and all the logic for "per address limit":s resides in "minter contract" VS in "whitelist contract"

Also I think we should allow modifying the root as long as end_time> now()

Unlike the above there is no technical challenges with this one. There is a small theoretical issue that can be probably dismissed, but wanted to get feedback on it first.

Having a new merkle root will essentially invalidate all the old proofs. In case of the launchpad that is most likely not a problem and the proofs will be generated for each user dynamically using the latest tree with the new root.

The issue will only arise if someone decides to share the proofs to users manually through DM:s or any other way. Users who had already minted wouldn't be affected, but those who hadn't would have their proofs invalidated and would need to ask the team for the new ones.

Like I said this can be dismissed and I can easily add the method if that is not a common use case we should accommodate for.

[kromsten]

Like I said this can be dismissed and I can easily add the method if that is not a common use case we should accommodate for.

Given no objection addressing it as a problem added the method @jhernandezb asked about