-
Notifications
You must be signed in to change notification settings - Fork 94
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Hardware wallets support #964
Comments
@sergeyboyko0791 Please post the info about libs and documentation you discover to this issue. |
I started researching how to integrate Ledger Nano S/X first into MarketMaker in WASM. Here's what I learned about it:
|
A few comments regarding integration Ledger/Trezor into MarketMaker in WASM
In summary, we can try to integrate Ledger and/or Trezor using WebUSB for Chrome only and continue looking for a solution to support every browser. |
I think running one app at a time is a security feature rather than "inconvenience" because a user is required to select a specific app. This adds an extra isolation layer to prevent malicious attempts to retrieve data from other supported coins. If I remember correctly on Trezor all you have to do is specify derivation path and/or coin name and get desired data from a device. U2F over browser in my experience is highly unstable, constant connection issues, timeouts etc. |
@sergeyboyko0791 Please consider this idea #928 (comment) while implementing abstractions for transaction signing. We should have the possibility to delegate this to a remote node that has access to an actual device. |
adding my edited comment from the "global todos list" (spreadsheet) from March 2021 for transparency and vision clarification: I see two good options for the UX of HW wallet support:
personally I prefer (1) but I understand it turn user experience poor so we likely stick to option (2) anyway One other factor and very high priority item is #740 With both HD and HW wallet support two of the most popular and wanted wallet features would get coverage. It would make sense to keep #740 in mind for the path derivation handlers, et cetera. |
At the first iteration, I'd prefer not to change the consensus protocol, but to implement the (1) option. |
* The starting point for working with Trezor * Add WebUSB driver that is used/planning to be used by Trezor and Ledger * Add TrezorClient implementation that allows to init a session and get PublicKey and UTXO address * Add the 'test_trezor' test suit for the web example * Update web-sys to 0.3.55 * Commit missing files * The starting point for working with Ledger (doesn't work at all) * Add a sketch of hid_driver to hw_common/transport * Add 'libusb' to 'hw_common/transport' * Add 'UsbTransport' for Trezor * Prepare TrezorClient for integration into mm2 * Add CryptoCtx that consists of either KeyPair or HardwareWallet * Add MmError::split and MmError::new_with_trace * Replace KeyDerivationPath with bip32::DerivationPath * Refactor TrezorResponse to avoid declaring 'Req' type and lifetime param * Add a RPC task manager that will return a task status and handle user action * Commit missing files * * Add 'MmInitTask' that can wait for user actions like entering Trezor pin * Add `MmInitError`, 'OrdermatchInitError', extend `P2PInitError` * Move 'lp_protocol' into 'common::mm_rpc_protocol' * Rename dispatcher_v2.rs to dispatcher.rs * Add the 'mmrpc: 2.0h' protocol * Refactor 'lp_init' to allow initializing `CryptoCtx` with either passphrase or Hardware Wallet device * Refactor 'init_and_migrate_db' to avoid deadlocking * Add 'TrezorClient::sign_tx' * Sign first RICK, QTUM transactions using Trezor * Add 'utxo_signer', move all functions related to signing UTXO transactions to this crate * Add and implement 'TrezorSigner' that interacts with 'TrezorClient' * Add PoC 'WithdrawTask', 'init_withdraw', 'withdraw_status' * Remove 'mmrpc: 2.0h' * Add missing files * Refactor 'TrezorClient', 'CryptoCtx' * Remove an instance of 'TrezorClient' from 'CryptoCtx' * Replace direct using 'secp256k1_key_pair' with 'CryptoCtx::secp256k1_pubkey' in some places * Finish implementing 'init_withdraw' * Add the 'trezor_coin' field to coins config * Move 'utxo_common::withdraw' to 'utxo_withdraw.rs', refactor it by converting the function to 'UtxoWithdraw' * Remove 'WithdrawTaskError' * Add the 'InitWithdrawCoin' trait, implement it for 'UtxoStandardCoin' and 'QtumCoin' * Add the 'withdraw_user_action' RPC call * Prepare UtxoCoinFields for HD wallets support * Replace 'UtxoCoinFields::my_address' with 'UtxoCoinFields::address_mode' * Replace 'UtxoCoinFields::key_pair' with 'UtxoCoinFields::priv_key_policy' * Add 'init_utxo', 'init_utxo_status' RPC calls * Make 'RpcTaskManager' generic * Move 'common::rpc_task' into the 'rpc_task' crate * Add 'CoinsActivationContext', 'MmInitContext' * Push missing files * Remove protoc-rust code generation from the 'trezor' crate * Add 'trezor/src/proto/message*' fiels generated by 'protoc-rust' * Fix Zcoin compile errors * Minor changes * Remove excess the 'hidapi' dependency * Remove unused 'test_trezor' WASM static function * Fix compilation error * Comment out 'hid_driver' transport within 'hw_common' crate * Remove 'test_trezor' from WASM test suit * Bump wasm-bindgen-test-runner v0.2.78 * PR fixes - iteration#1 * Rename and refactor 'AddressMode' to 'DerivationMethod' * Move 'DerivationMethod' and 'PrivKeyPolicy' to 'lp_coins', make them generic * Add 'TxProviderError', return it instead of 'UtxoSignTxError' from 'TxProvider::get_rpc_transaction' * Fix rustfmt warnings * Replace 'protoc-rust' with 'prost-build' * Display 'from' address of the 'init_withdraw' RPC call * Change withdraw display message * Add button acknowledging confirmation * Unite 'in_progress', 'awaiting', 'finished' containers to 'tasks' by adding 'TaskStatusExt' * Make `TrezorClient` thread-safe * Add `TrezorSession` that is a wrapper over mutex guard * Keep the connection to a Trezor device in `HwWalletContext` while it's alive * Fix WASM compilation error * Fix ZCoin compilation error * Reinitialize `HwWalletContext::hw_wallet` if we reconnect to a device
In the PR mentioned above I implemented requesting balances of HD accounts and addresses like it does hw-kmd-wallet:
These steps are performed on coin activation and every time user calls @artemii235 @tonymorony what do you think? |
* Refactor `init_utxo` according to a unified `init_standalone_coin` protocol * Move `PrivKeyBuildPolicy` to `lp_coins.rs` * Refactor `grpc_web_multi_url_request` * Add `MmRpcResult::map_err`, `RpcTaskStatus::map_err` * Push missing files * Initialize UTXO coin with a pubkey requested from a Trezor device * Move `UtxoCoinBuilder`, `UtxoArcBuilder` from `utxo.rs` to a separate module * Add `extended_pubkey`, `derivation_path` to `HDWalletInfo` * Implement requesting balances of HD wallet addresses * Move `extended_pubkey` from `UtxoHDWallet` to `UtxoHDAccount` * Register `UtxoStandardCoin` at `InitStandaloneCoinActivationOps::init_standalone_coin` * Add `RpcDerivationPath` * Push missed `coin_balance.rs` * Increase `WAIT_DOCKER_READY_TIMEOUT_MS` to 60s * Bump the `ring` crate version to 0.16.15 * Add `wallet_balance` RPC call * Fix zhtlc feature compilation errors * Add ZCoinBuilder * Minor changes * Optimize UTXO coin builders not to clone `UtxoActivationParams` * Refactor `NativeClient::list_transactions_by_address` to `NativeClient::is_address_list_transactions_empty` * Fix WASM, ZCash compile errors * Fix rustfmt warnings * Fix compilation of docker tests * Add QTUM coin initialization with Trezor * Add `init_qtum`, `init_qtum_status`, `init_qtum_user_action` RPC calls * Add `derivation_path` to `HDAccountBalance` * Fix `recreate_swap_data::convert_maker_to_taker_events` * Return the previous implementation of `grpc_web_multi_url_request` * Split `wallet_balance` into `hd_account_balance` and `check_hd_account_balance` RPC calls * Add check_utxo_maturity to electrum/enable RPC call (#1201) * Add `check_utxo_maturity` to electrum/enable RPC call * Add `QtumCoinBuilder` and `QtumCoinWithIguanaPrivKeyBuilder` that override `check_utxo_maturity` * Add `UtxoCommonOps::list_all_unspent_ordered` that behaves as `UtxoCommonOps::list_unspent_ordered` before * `UtxoCommonOps::list_unspent_ordered` checks the `check_utxo_maturity` flag to use either `list_all_unspent_ordered` or `list_mature_unspent_ordered` * Rename `UtxoCommonOps::ordered_mature_unspents` to `UtxoCommonOps::list_mature_unspent_ordered` * Add `get_new_hd_address`, `init_create_new_hd_account` RPC calls * Use `tokio::RwLock` instead of `parking_lot::Mutex` to store HD accounts * Move HD wallet related traits and structures from lp_coins.rs to coin_hd_wallet.rs * TODO use `HwRpcTaskAwaitingStatus`, `HwRpcTaskUserAction`, `HwRpcTaskUserActionRequest`, `InitRpcTaskResponse`, `RpcTaskActionError`, `RpcTaskStatusError`, `RpcTaskStatusRequest` everywhere it's possible * Use RPC common structures and errors wherever possible * Use `futures::lock::Mutex` instead of `tokio::lock::RwLock` * Fix utxo_tests compile errors * Perform naming refactoring * Add the `HDWalletCoinAndBalanceOps` trait alias * Add `HDWalletOps` and `HDAccountOps` * Move the `HDWallet` related methods from `HDWalletCoinOps` into `HDWalletOps` * Move the `HDAccount` related methods from `HDWalletCoinOps` into `HDAccountOps` * Rename `get_new_hd_address` to `get_new_address`, `hd_account_balance` to `account_balance`, `init_create_hd_account` to `init_create_account` RPC calls * Refactor `XPubExtractorUnchecked` * Refactor generic type bounds (#1211) * Possible refactoring. * Removed more bounds. * Added HDAccount for HDWalletCoinOps to reduce "as HDWalletOps" constructions. * Continue refactoring * Add a default implementation of `HDWalletCoinOps::generate_new_address` * Remove `HDWalletCoinOps::address_balance` * Make `HDWalletBalanceRpcOps` independent of `HDWalletBalanceOps` Co-authored-by: Artem Vitae <artem@vitae.com> * Check if the specified `WithdrawRequest::from` DerivationPath/AccountId refers to an activated account/address * Add the `WithdrawFrom::AccountId` variant to allow the user to specify account_id, chain and address_id * Add `Bip44DerivationPath`, `Bip44PathToCoin`, `Bip44PathToAccount` * Avoid extracting a pubkey from a Trezor device on `init_withdraw` * Fix PR issues * Avoid using `clippy::type_complexity` by refactoring the `RpcTask` module * Typo: address_unspendable_balanAce * Fix PR issues, make `RpcTaskId` unique * Rename `ExpectedHDWalletDerivationMethod` to `CoinIsActivatedNotWithHDWallet` * Reorder RPCs alphabetically * impl `From<KeyPair>` * Fix rustfmt * Remove `KeyPairArc::from_key_pair` Co-authored-by: Artem Vitae <artem@vitae.com>
I'd like to share my thoughts on how to activate coins with Trezor and provide HD wallet functionality. Cons:
|
We've just had a quite productive call with @sergeyboyko0791 @Milerius @yurii-khi and concluded that |
I'd like to share the integration progress and the what we need to do (in my opinion) to complete the implementation of the wallet functionality:
|
* Add `HDWalletCoinWithStorageOps`, `HDWalletCoinStorage` * Add `HDAccountStorageItem` IndexedDb table * Add and implement `HDWalletIndexedDbStorage`, `HDWalletDb` in WASM * Integrate HD wallet storage to mm2 * Load HD accounts on coin initialization * Updating number of known addresses on `get_new_address`, `scan_for_new_addresses` RPC calls * Clearing DB if an error occurs on loading HD accounts * Don't extract xpub on coin initialization, extract it on `get_activation_result` if necessary * Remove `UtxoArcWithIguanaPrivKeyBuilder` * Implement HD wallet sqlite storage * Replace `wallet_id` column with `coin`, `mm2_rmd160` and `hd_wallet_rmd160` within the storage * Add `HDWalletMockStorage` for testing purposes * Minor refactoring * Rename `AddressChecker` to `AddressScanner` * Refactor `DerivationMethodNotSupported` * Rename `DerivationMethodNotSupported` error to `UnexpectedDerivationMethod` * Minor changes * Temporary disable the `polygon_check_if_my_payment_sent` test * Don't allow using `HDWalletMockStorage` in WASM * Make `HDWalletSqliteStorage::update_addresses_number` safe * Make `CreateNewAccountParams::scan` boolean * Minor fixes * PR fixes
) * Simultaneous activation of coins in two modes (Iguana and Trezor) * Refactor `CryptoCtx` to be a structure that contains `IguanaArc` always and a constructible `HwArc` * Add `init_trezor` RPC call that initializes `HwArc` * Rename `KeyPairCtx` to `IguanaCtx` * Comment out `mm_init_task`, `rpc_command` modules, `mm_init_status`, `mm_init_user_action` RPC calls * PR fixes * Avoid using `AsyncRwLock` for `CryptoCtx::hw_ctx` initialization by adding `HardwareWalletState` * Remove commented code, `mm_init_task.rs`, `rpc_command.rs` files * Simplify trait bounds * Inherit `UtxoCommonOps` from `AsRef<UtxoCoinFields> + Clone + Send + Sync + 'static` * Remove `async-std` from the `crypto` dependencies #964
#964 #740 * Restructure TX history storage modules * Add `coins::tx_history_storage` module * Move `load_history`, `save_history`, `clear` from `TxHistoryDb` implementation to static functions * Add `BeBigUint` into `common::indexed_db` * Cover `BeBigUint` with unit tests * Add `IndexedDbTxHistoryStorage` * Add `indexed_db::DbTable::count`, `indexed_db::DbTable::count_all` * Add methods to `indexed_db::DbTable` that take multiple index keys as arguments * Remove `DbTransaction::wait_for_complete` as it doesn't work properly * Integrate HD addresses into TxHistory V2 Storage scheme * Add tx history `WalletId` * Add `SqlQuery` that acts like `SqlBuilder` but manages parameters * Add `<WalletId>_tx_from_address`, `<WalletId>_tx_to_address` SQL tables * Minor changes * Add `try_serialize_index_value` macro * Avoid declaring public methods, traits, types * Fix PR issues * Move big JSON payloads to separate files * Add the `indexed_db::test_transaction_abort_on_error` test * Allow the `SqlQuery` to validate SQL identifiers and values * Combine `tx_from_address` and `tx_to_address` tables into single `tx_address` * Fix clippy warning * Fix PR issues * Avoid adding `mm2src/coins/for_tests/RICK_<ID>.json` files * Minor changes
#964 #740 * Restructure TX history storage modules * Add `coins::tx_history_storage` module * Move `load_history`, `save_history`, `clear` from `TxHistoryDb` implementation to static functions * Add `BeBigUint` into `common::indexed_db` * Cover `BeBigUint` with unit tests * Add `IndexedDbTxHistoryStorage` * Add `indexed_db::DbTable::count`, `indexed_db::DbTable::count_all` * Add methods to `indexed_db::DbTable` that take multiple index keys as arguments * Remove `DbTransaction::wait_for_complete` as it doesn't work properly * Integrate HD addresses into TxHistory V2 Storage scheme * Add tx history `WalletId` * Add `SqlQuery` that acts like `SqlBuilder` but manages parameters * Add `<WalletId>_tx_from_address`, `<WalletId>_tx_to_address` SQL tables * Minor changes * Add `try_serialize_index_value` macro * Avoid declaring public methods, traits, types * Fix PR issues * Move big JSON payloads to separate files * Add the `indexed_db::test_transaction_abort_on_error` test * Allow the `SqlQuery` to validate SQL identifiers and values * Combine `tx_from_address` and `tx_to_address` tables into single `tx_address` * Fix clippy warning * Fix PR issues * Avoid adding `mm2src/coins/for_tests/RICK_<ID>.json` files * Minor changes
* Refactor `init_trezor` RPC * Add an optional expected `device_pubkey` to the `init_trezor` request * Add connected `device_pubkey` to the `init_trezor_status` result * `init_trezor` has to be able to reinitialize a Hardware wallet * Refactor `init_utxo`, `init_trezor` RPCs * Rename `WaitForTrezorPin` user action to `EnterTrezorPin` * Rename `ReadPublicKeyFromTrezor` in progress status to `WaitingForUserToConfirmPubkey` * Add `ticker` field to the `init_utxo`, `init_qtum` results * Refactor `init_utxo`, `init_qtum` RPCs * Remove unused error types * Add `crypto::HwRpcError` * Replace HW related errors with `crypto::HwRpcError` for `init_utxo` RPC * Refactor `init_withdraw` RPC * Replace HW related errors with `crypto::HwRpcError` for `init_withdraw` RPC * Refactor RPCs * Replace HW related errors with `crypto::HwRpcError` within `HDWalletRpcError` and `Mm2InitError` * Add `EnumFromTrait` and `EnumFromInner` derive macros * Add `init_account_balance`, `init_account_balance_status` RPCs * Add `common::SerdeInfallible` * Add `can_get_new_address` and refactor `get_new_address` RPCs * Refactor `get_new_address` to return an error if the last address is not used yet * Split `HDWalletRpcError` into `GetNewAddressRpcError` and `CreateNewAccountRpcError` * Push missing `can_get_new_address` to the dispatcher * Push missing `can_get_new_address` to the dispatcher * Allow the user to cancel RPC tasks * Add `task` RPC namespace, move all `init_*` methods there * Minor changes * Minor fix * Add and change comments * Fix `enum_from` doc tests. * Handle `Trezor::PassphraseRequest` * Add `EnterPassphrase` awaiting status * Add `TrezorPassphrase` user action * Rename `AwaitingStatus::EnterPassphrase` to `AwaitingStatus::EnterTrezorPassphrase` * Return Hardware wallet device info on `init_trezor::status` * Rename `InProgressStatus::WaitingForUserToConfirmPubkey` to `InProgressStatus::FollowHwDeviceInstructions` * Add `get_enabled_coins` RPCv2 * Remove `TrezorModel` enum * Remove `TrezorUtxoCoin` enum, replace it with a regular `String` * Refactor task API * Replace `Ready` status with `Ok` and `Error` * Replace magic pubkey prefixes with `xpub` prefix on `TrezorSession::get_public_key` * Keep one Trezor session to avoid unnecessary Passphrase request * Fix tests * Remove unused deps from `rpc_task` RPC * `get_new_address`, `task::enable_utxo`, `task::enable_qtum` refactoring * Create at least `min_addresses_number` addresses on coin activation * Allow to generate up to `gap_limit` empty addresses on `get_new_address` RPC * Improve `test_scan_for_new_addresses` test * Add `test_can_get_new_address` test * Implement `Debug` trait for `MmError` manually * Fix fmt * Fix tests, add comments * Specify destination address derivation path for change output * Fix PR issues * Refactor `gen_new_addresses_if_required` * Fix deserializing `min_addresses_number` from legacy UTXO request * Remove `KEEP_XPUB_MAGIC` constant * Minor PR fix * Fix compile error * Fix `get_new_address` RPC * Fix `test_get_new_address`
#964 (#1467) * Refactor `BchAndSlpTxHistory` making it generic over `Coin` type * Add `UtxoTxHistoryOps`, implement it for `BchCoin` * Remove `UtxoStandardOps` implementation for `BchCoin` * TODO implement `WaitForHistoryUpdateTrigger` state * Implement `UtxoTxHistoryOps` and `CoinWithTxHistoryV2` for `UtxoStandardCoin` * Add `taget: MyTxHistoryTarget` field to `my_tx_history` request * Add `utxo_common/utxo_tx_history_common.rs` for TX history related common impl * Add `HDWalletCoinOps::derive_known_addresses` * Refactor `GetTxHistoryFilters` by requiring to set `from_addresses` * Fix `tx_history_v2_tests` * Start history background fetching on UTXO/QTUM coin initialization * Implement `UtxoTxHistoryOps` and `CoinWithTxHistoryV2` for `QtumCoin` * Implement * Implement `WaitForHistoryUpdateTrigger` state * Add `ElectrumClient::scripthash_get_history_batch` * Implement `UtxoTxHistoryOps::request_tx_history` according to `DerivationMethod` * Add `CoinBalanceReportOps` trait * Don't spawn legacy tx_history loop on `init_utxo`, `init_qtum` RPCs * Rename `EnableCoinBalance` to `CoinBalanceReport` * Add `UtxoTxHistoryV2::my_addresses` to optimize `tx_details_by_hash` * Rename `utxo_tx_history_common.rs` to `utxo_tx_history_v2_common.rs` * Implement `UtxoTxHistoryOps::tx_details_by_hash` optimized for TX history V2 * Add `UtxoTxHistoryOps::tx_from_storage_or_rpc` * Fix `CoinWithTxHistoryV2::get_tx_history_filters`, `UtxoTxHistoryOps::request_tx_history` * Add `UtxoTxHistoryError` * Test `UtxoTxHistoryOps::tx_details_by_hash` along with `UtxoStandardOps::tx_details_by_hash` * Final refactoring * Add `UtxoMyAddressesHistoryError` for `UtxoTxHistoryOps::my_addresses` * Optimize `UtxoTxHistoryOps::request_tx_history` by passing `my_addresses` argument * Fix fmt * Fix `test_bch_and_slp_testnet_history` test * Fix compile error * Refactor BCH to fix `test_bch_and_slp_testnet_history` * Use `utxo_common::request_tx_history` for `BchCoin` instead of `utxo_tx_history_v2_common`'s * Fix PR issues * Optimize TX history states by specifying `FetchingTxHashes::fetch_for_addresses` * Rename `UtxoTxHistoryOps::get_addresses_balances` to `my_addresses_balances` * Add `UtxoTxHistoryOps::address_from_str` to parse addresses within `UtxoTxHistoryOps::my_addresses_balances` result * Add `AddrFromStrError` * Add `for_addresses` argument to the `unique_tx_hashes_num_in_history` function * Add `SqlQuery::count_distinct` * Add `for_addresses` argument to the `history_contains_unconfirmed_txes` and `get_unconfirmed_txes_from_history` functions * Minor changes * Add `test_hd_utxo_tx_history` native and WASM test * Fix `SqlTxHistoryStorage` to repeat the same ordering as in `compare_transaction_details` * Fix tests * Use `utxo_common::utxo_tx_history_v2_common::request_tx_history` within for BCH coin * Move `utxo_coin_fields_for_test` to `utxo_common_tests.rs` * Add `test_bch_and_slp_testnet_history` in WASM * Add `T_BCH_ELECTRUMS` listening on WSS port * Ignore `solana_prerequisites` test * Skip serializing of `ElectrumRpcRequest::protocol` if it's None * Add timeout to `wait_till_history_has_records` helper * Fix ETH tests * Fix `test_convert_segwit_address` and `test_validateaddress_segwit`
Hardware Wallet integration is not definitely finished, but we have reached the point that it can be integrated into GUIs and used in cli. |
Can we please add a method for returning a user's xpub key? |
Looks like blockbook could help with this https://github.com/trezor/blockbook/blob/master/docs/api.md#get-xpub |
yes definitely. Its a trivial addition and actually a requirement for efficient HD key/account management (enables mm2, GUIs and 3rd party implementations to derivate entire key-tree for any-pre-checks/pre-syncs/et cetera) - we will add this during the next iterations. |
It would be a huge booster to add hardware wallets (Trezor/Ledger) support into
The text was updated successfully, but these errors were encountered: