Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Avalanche Warp Messaging Support #57

Merged
merged 58 commits into from
Mar 20, 2023
Merged

Add Avalanche Warp Messaging Support #57

merged 58 commits into from
Mar 20, 2023

Conversation

patrick-ogrady
Copy link
Contributor

@patrick-ogrady patrick-ogrady commented Feb 26, 2023

TODO

  • block verify with context (ShouldVerifyWithContext && VerifyWithContext)
  • async generate message signature and make queryable over P2P by anyone
  • allow actions to define a "warpable" payload: https://github.com/ava-labs/xsvm/blob/cbeff25ecbe3330cb3dc6effe065b1541c3bb77b/execute/tx.go#L88-L95
  • verify warp signature: https://github.com/ava-labs/xsvm/blob/cbeff25ecbe3330cb3dc6effe065b1541c3bb77b/execute/tx.go#L159-L181
  • every node is a relayer => query validators for their signatures and store on-disk
    • make this mode optional if you know you won't be using the node to generate new warp messages via the CLI
  • fetch collected signatures from API and aggregate in CLI: https://github.com/ava-labs/xsvm/blob/cbeff25ecbe3330cb3dc6effe065b1541c3bb77b/cmd/issue/importtx/cmd.go
  • verify multi-signature async and check at end of block (like with signatures)
  • add multi-plexed network router - Add network multi-plexing #59
    • append handler id at end to avoid possible array copy
  • Run verify with context
  • Add logging to networking failures
  • Add quick verification check before BLS sig check that each is allowed
  • Don't make app request directly from failed
  • Implement Simple Warp Messages on TokenVM #62
    • allow transfer to optionally reward tx submitter (relayer)
    • charge more for an action if it has more signers (blocked on warp.Signature.NumSigners method
    • protect against warp message replay by storing accepted warp IDs in state (cannot be based on signature, as this is mutable)
    • track balance exported to a given region and don't let more come back (https://github.com/ava-labs/xsvm/blob/cbeff25ecbe3330cb3dc6effe065b1541c3bb77b/state/storage.go#L133-L200)
    • set assetID on other chains as ID(sourceChainID, assetID) -> put raw info in metadata with empty owner
    • assets take on a new identity when they flow through a subnet (README) -> recommended point-to-point
    • handle fund return (when importing, track with existing assetID rather than a new one)
    • must make imported asset special to avoid people copying pattern and exporting unimported balances
    • should scope import/export to a specific chain so can contain a bug
    • store warp messages in state so they can be signed by new validators
    • (not needed) pad bitset if last X elements are not signers (length of signers in signature will be < validators)?: https://github.com/ava-labs/avalanchego/blob/d755f872a4bf0de12297b3994b729ea684f78519/vms/platformvm/warp/validator.go#L97-L105
    • charge fee in VM based on num signers: Update to avalanchego-1.9.10 #71
    • check wrap TxID is not duplicate before performing any validation
    • we must mark the tx as fee-paying if the warp message is not valid, rather than marking the block as invalid (DoS)
    • implement build block with context...if given a block without this, make sure not to include any warp messages
    • improve name for "StateMapping"
    • add provide warp message to action unmarshal (so can parse inner message and provide state keys)
    • codify warp support in registry so don't have to have a check in every unmarshal
    • add warp E2E transfer test (native and non-native round trip): Add tokenvm warp e2e tests (Part 1) #77
    • ensure we only persist warpID if tx successful
    • make sure warpID just includes payload (otherwise could do duplicate with anycast and chainID)
    • add import/export to registry and metrics
    • handle no signer case/empty payload/incorrect signature
      • test odd warp messages in integration test
    • block non-return export of imported assets (force point-to-point)
  • scope requestID to NodeID to avoid overflow with course tracking (avalanchego is ok as long as you don't reuse nodeID+requestID)
  • re-trigger wrap signature fetch when calling GetSignatures for any validators that have not yet sent sigs (and that aren't pending) -> protects against shutdowns and validator rotation
  • if signature requested from us and we haven't generated yet, generate and store (may have state synced past tx)
  • add a "trade-in" option where a relayed message can specify a max rate it will pay for a trade to get the native currency (relayer can then link an order that will execute upon tx acceptance) -> should just force orderIDs willing to accept + input (if none valid, just process with no swap) - Add support for "swap in" #80
  • add warp message cleanup refund (if user clears item from state, get some % of original fee back -> user-driven ack) -> may be simpler if we just say each user can have at most 1 pending warp message at a time? -> just limit outstanding per account -> lockup + refund (unrelated to current network fee, so no gas token)
  • add network/account switch to token-cli (so can easily transfer between known networks/use stored APIs) - Update token-cli #86
    • persist default on shutdown and show each run
    • must manually add networks instead of defaulting to local
    • add clear defaults in case of restart
    • if no default set yet, request on startup (if not genesis function)
    • add warp support to explorer
    • add multiple endpoints per network
  • make warpID just be the hash of the payload (store by sourceChainID) - Update warpID #85
  • add GitHub action timeouts in case of infinite loop bug
  • only enable BLS_PORTABLE on macos/windows
  • Add BLS support to ANR: (Update ANR #82 + CreateSpecificBlockchains avalanche-network-runner#467)
    • should be able to get down to a single restart (add nodes, create subnets, add blockchain configs, restart, add subnet validators, create chains)
  • warp result should be boolean - Enforce Boolean Warp Verification Results #92
    • attach AWM verification results to block for bootstrapping/lite clients
  • update TokenVM demo with a warp swap (move to Subnet B to swap a minted token) - Finalize AWM Doc Updates + Demo #93
  • Update docs - Finalize AWM Doc Updates + Demo #93
  • drop warp messages from builder if expired - Finalize AWM Doc Updates + Demo #93

@patrick-ogrady patrick-ogrady marked this pull request as draft February 26, 2023 21:58
vm/warp_manager.go Fixed Show fixed Hide fixed
vm/warp_manager.go Fixed Show fixed Hide fixed
* start work on network manager

* more progress

* making progress with the wrapped sender

* handle first method in request manager

* finish handler

* add multi-plexing over network
vm/warp_manager.go Fixed Show fixed Hide fixed
chain/block.go Fixed Show fixed Hide fixed
@patrick-ogrady patrick-ogrady added this to the v0.0.2 milestone Feb 27, 2023
@patrick-ogrady patrick-ogrady added the enhancement New feature or request label Feb 27, 2023
@patrick-ogrady patrick-ogrady linked an issue Feb 27, 2023 that may be closed by this pull request
* progress on import

* add warp transfer object

* add quick new assetID

* add reward allocation

* update incorrect configuration

* add clear callout for handling fund return

* read signers from signature

* DoS vuln to mark block as invalid with incorrect warp

* add more TODOs

* add another todo

* add to README

* add more TODOs

* add todo for dos

* add another note

* add stub for build with context

* update action interface to accept boolean of warp readiness

* wait for warp verification in block

* cleanup build/verification logic

* move warp message around

* making more progress

* cleanup tx verification

* update block handling for warp

* handle basic re-signing

* fetch missing keys if we don't have them

* chain processor passing

* add note about StateMapping

* fix empty key issue

* limit fetch frequency for warp manager

* fix vm interface

* allow type parser to take 2 args

* pass warp message to unmarshal

* use state manager

* fix unchecked err

* add base fee for warp

* naming

* update actions

* continuing to cleanup old approach

* update get asset func

* overhaul network manager (use unique requestIDs per nodeID)

* update mocks

* update type parser to have extra bool

* add warp check to registry

* remove interface changes

* add small blurb to readme

* update registry with new warp check

* more progress

* update WarpID

* only store warp info if successful

* add loan storage

* import compiles

* add some pre-verification

* override sourceID

* store native supply

* add native supply tracking

* add import/export to registry and metrics

* update required interfaces

* protect against fee overflow

* fix lint

* update tokenvm linting

* pass lint

* integration passing

* fix lint

* Add tokenvm warp e2e tests (Part 1) (#77)

* startup 2 subnets

* make startup more robust

* fix node shutdown issue

* e2e progress

* ensure warp message is initialized after emission

* progress

* progress

* add workflow timeouts

* add missing lock release

* cleanup tx init

* update digest

* fix warp message test

* handle empty warp message

* update verification override

* add export test

* add loan check

* remove ability to export warped asset that isn't a return

* add to readme

* fix lint

* add logging in skip case

* use custom ANR

* overhaul test

* up size of unit test runner

* add more logs

* use fixed bytes for decode

* more debugging

* fix signature fetch

* blocked on waiting for context

* get to warp verify

* more logging

* more lint

* add more correctness checks on imported asset

* fix integration test

* fix e2e balance check

* more incorrect vars

* passing new tests

* add double-warp test

* remove incorrect fail case

* make logging less verbose

* update bootstrap readiness logic

* improve logging

* increase timeout

* Add tokenvm e2e tests (part 2) (#79)

* add first part of tests

* round-trip test

* lint
* add parse to import

* add scaffold for swap in

* RFQ for swap

* cleanup import

* use the optional codec

* add fields to export

* add swap in test

* unify swap validity check params

* remove more println

* tweak fees

* fix lint

* more lint
* update ANR

* fix logging

* ensure server data is updated

* update ANR version
* modify storage

* warpID -> warpPayloadID

* remove *WarpMessage

* update actions
* work on new methods

* improve naming

* remove old options

* keep workdir

* progress

* add basic storage

* implement import key

* update key to load all

* cleanup assets

* cleanup key

* cleanup chain

* start action migration

* cleanup transfer

* move create asset

* remove mint asset

* complete close order

* move create_order

* migrate all transactions

* add support for import asset parse

* add parsing for export

* prep import asset cmd

* add export support

* populate export action

* add auto-return support

* update gitignore

* fix scripts run

* fix lint

* emit outputs after cluster spun up

* add powered by hypersdk

* make smaller

* more testing

* keep under

* more tweaks

* add line break

* more down

* add at end

* little bigger

* break apart

* update storage and added TODO

* use first uri for now

* add another comment

* add multi-chain support

* add support for custom db path

* add duplicate protection

* close database early if not needed

* silence included error

* nit

* add string util

* all working
* migrate actions to use boolean for warp verification

* update actions interface

* populate error check

* update chain builder with bool

* nits

* more nits

* add note about warp verification

* add note about max

* fix integration tests

* make warp result checking more effective
* starting main README

* make progress on main README

* updating README

* continue cleaning docs

* progress on explanation

* add warp image

* specify all things that must be handled

* finish explanation

* cleanup wording on describing what hypersdk does

* add result description

* finish main README

* more progress

* finish docs

* fix link

* prompt for chain during watch

* use prompt chain on export

* automatically perform import

* submit dummy transaction if too old

* send dummy transaction automatically

* return current chainID in defaultActor

* fix key usage

* fix builder (drop expired warp)

* cleanup chainID logs

* update demo for mint-trade

* add demo for export

* fix lint
@patrick-ogrady patrick-ogrady marked this pull request as ready for review March 20, 2023 00:41
@patrick-ogrady patrick-ogrady merged commit 32d4092 into main Mar 20, 2023
@patrick-ogrady patrick-ogrady deleted the warp-messaging branch March 20, 2023 00:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Avalanche Warp Messaging Support
1 participant