[crypto/ed25519] ed25519 Concurrent Batch Verification + ZIP-215 Enforcement

[patrick-ogrady]

Resolves: #337

TL;DR: single verification cost drops from 45000 ns to 27000 ns when using the new library (-40%) and 45000 ns to 8904 ns (-80%) when using batch verify.

Results:

goos: darwin
goarch: arm64
pkg: github.com/ava-labs/hypersdk/crypto
BenchmarkStdLibVerifySingle-12       	   24030	     49676 ns/op
BenchmarkOasisVerifySingle-12        	   35736	     33637 ns/op
BenchmarkOasisVerifyCache-12         	   40842	     29192 ns/op
BenchmarkOasisBatchVerify/1-12       	   24301	     49368 ns/op
BenchmarkOasisBatchVerify/4-12       	   10000	    102825 ns/op
BenchmarkOasisBatchVerify/16-12      	    3878	    313535 ns/op
BenchmarkOasisBatchVerify/64-12      	    1038	   1229892 ns/op
BenchmarkOasisBatchVerify/128-12     	     538	   2228857 ns/op
BenchmarkOasisBatchVerify/512-12     	     166	   7086516 ns/op
BenchmarkOasisBatchVerify/1024-12    	      94	  12970639 ns/op
BenchmarkOasisBatchVerify/4096-12    	      24	  48236476 ns/op
BenchmarkOasisBatchVerify/16384-12   	       6	 187775667 ns/op
BenchmarkOasisBatchVerifyCache/1-12  	   26325	     46554 ns/op
BenchmarkOasisBatchVerifyCache/4-12  	   13917	     85865 ns/op
BenchmarkOasisBatchVerifyCache/16-12 	    4866	    243184 ns/op
BenchmarkOasisBatchVerifyCache/64-12 	    1369	    869321 ns/op
BenchmarkOasisBatchVerifyCache/128-12         	     687	   1744689 ns/op
BenchmarkOasisBatchVerifyCache/512-12         	     208	   5807465 ns/op
BenchmarkOasisBatchVerifyCache/1024-12        	     100	  10393727 ns/op
BenchmarkOasisBatchVerifyCache/4096-12        	      31	  38042855 ns/op
BenchmarkOasisBatchVerifyCache/16384-12       	       7	 148778535 ns/op
PASS
ok  	github.com/ava-labs/hypersdk/crypto	97.032s

Results on single core (~same -> can still parallelize):

goos: darwin
goarch: arm64
pkg: github.com/ava-labs/hypersdk/crypto
BenchmarkStdLibVerifySingle    	   27198	     43851 ns/op
BenchmarkOasisVerifySingle     	   36319	     33057 ns/op
BenchmarkOasisVerifyCache      	   43023	     27136 ns/op
BenchmarkOasisBatchVerify/1    	   24878	     48444 ns/op
BenchmarkOasisBatchVerify/4    	   10000	    101226 ns/op
BenchmarkOasisBatchVerify/16   	    3859	    303776 ns/op
BenchmarkOasisBatchVerify/64   	    1074	   1122474 ns/op
BenchmarkOasisBatchVerify/128  	     560	   2135260 ns/op
BenchmarkOasisBatchVerify/512  	     174	   6923636 ns/op
BenchmarkOasisBatchVerify/1024 	      94	  12679955 ns/op
BenchmarkOasisBatchVerify/4096 	      25	  47537617 ns/op
BenchmarkOasisBatchVerify/16384         	       6	 187736208 ns/op
BenchmarkOasisBatchVerifyCache/1        	   28065	     42653 ns/op
BenchmarkOasisBatchVerifyCache/4        	   14504	     83176 ns/op
BenchmarkOasisBatchVerifyCache/16       	    5005	    240122 ns/op
BenchmarkOasisBatchVerifyCache/64       	    1395	    863280 ns/op
BenchmarkOasisBatchVerifyCache/128      	     712	   1699842 ns/op
BenchmarkOasisBatchVerifyCache/512      	     214	   5878363 ns/op
BenchmarkOasisBatchVerifyCache/1024     	     100	  10013806 ns/op
BenchmarkOasisBatchVerifyCache/4096     	      32	  37275258 ns/op
BenchmarkOasisBatchVerifyCache/16384    	       7	 145889137 ns/op
PASS
ok  	github.com/ava-labs/hypersdk/crypto	95.193s

Before(~100ms/10k txs):

After (~45ms/10k txs):

Notably, there is no wait on signatures anymore in Verify.

TODO

• Enforce clearer verification semantics (likely ZIP215)
• Use multiple addresses for original cache test (just pre-store in cache first)
• Add test for cache with batch verify
• equally split work across workers (of number of cores) and spawn work to get best performance [with min split amount]: [crypto/ed25519] Add batch support to chain #343
• Use cache only for on-chain verification (otherwise easy to bust with RPC)
• Periodically perform batch verification of txs submitted over RPC/gossip: [crypto] Run Batch Verification on Incoming Gossip #345
• Link useful articles like https://hdevalence.ca/blog/2020-10-04-its-25519am and https://eprint.iacr.org/2020/1244.pdf
• collect WaitSignatures time before/after images
• Add new cache where only store expanded if add is called
• Record number of each auth object in parse for pre-allocation in batch
• Introduce generic cryptography structs: [morpheusvm/tokenvm/indexvm] Change the auth on one of these to better showcase how to provide your own crypto #267