Address slow fee estimation on Byron wallets

[KtorZ]

Issue Number

#2126

Overview

• fc1da51
  📍 extend post-restoration benchmark wallet overview with number of addresses and transactions
  We had only the UTxO, which gave little clarity about how many addresses and transactions did that correspond to. In practice, they're of the same order of magnitude, but it's good to have the actual number.

• a234e49
  📍 do not construct dummy bootstrap witnesses but apply a correction on shelley witnesses afterwards

  This is slightly simpler, but it also prevent some nasty things happening with the bootstrap witnesses: the 'Ord' instances used to compare and sort bootstrap witnesses require to serialize them fully, which involve two hashes of bytestrings; when done for 50 inputs, and 100 times in a row in multiple selection this little thing start to matter quite a lot.

• fc9d4ff
  📍 adjust byron migration test assertion due to recent change
  The fee estimation is now slightly faster, but the approximation is slightly worse, so we end up paying a little more for byron witnesses.

Comments

[KtorZ]

The fee estimation for Byron witnesses is slightly less accurate than before with this change, so fees are evaluated slightly in excess. We could spend more time tweaking numbers but I think it's good enough already even though it means that transactions costs a little bit more.

[paweljakubas]

we want to revert this to our it, right?

[piotr-iohk]

Using regular Hspec.it makes sense here I think, because those tests use specific mnemonic wallets, so in case when those tests fail the subsequent run will result in 409 - wallet already exists anyway...

[KtorZ]

Using regular Hspec.it makes sense here I think, because those tests use specific mnemonic wallets

Indeed. We don't want to retry these tests because they use a hard-coded mnemonic. So I changed it back to the standard hspec "it" to make debugging easier.

[rvl]

What about prefixing the retrying version it with sh to clarify its function? (sorry, couldn't resist 😁)

[paweljakubas]

the same here

[paweljakubas]

the same here

[paweljakubas]

👍

[piotr-iohk]

lgtm.

Manually tested on testnet and fee estimation is now instantaneous on random and icarus Byron wallets.
(Would be nice to see full benchmarks after changes 😅)

[paweljakubas]

Resigning from ledger bits (using their serialization infrastructure) for performance reason and "hardcoding" input correction numbers basing on what we know in each case is included seems to be very good move. Especially if during fee estimation we invoke this part 100 times

[KtorZ]

@piotr-iohk, you're welcome:

All results:                
BenchSeqResults:            
  benchName: 0-percent-seq  
  restoreTime: 138.6 s      
  readWalletTime: 4.515 ms  
  listAddressesTime: 3.228 ms
  listTransactionsTime: 361.9 μs
  estimateFeesTime: 2.271 s 
  walletOverview:           
     number of addresses: 20
     number of transactions: -0
     = Total value of -0 lovelace across 0 UTxOs
      ... 10                0
      ... 100               0
      ... 1000              0
      ... 10000             0
      ... 100000            0
      ... 1000000           0
      ... 10000000          0
      ... 100000000         0
      ... 1000000000        0
      ... 10000000000       0
      ... 100000000000      0
      ... 1000000000000     0
      ... 10000000000000    0
      ... 100000000000000   0
      ... 1000000000000000  0
      ... 10000000000000000 0
      ... 45000000000000000 0
                            
BenchRndResults:            
  benchName: 0-percent-rnd  
  restoreTime: 126.7 s      
  readWalletTime: 1.405 ms  
  listAddressesTime: 501.5 μs
  listTransactionsTime: 361.3 μs
  importOneAddressTime: 742.3 μs
  importManyAddressesTime: 77.84 ms
  estimateFeesTime: 1.090 s 
  walletOverview:           
     number of addresses: -0
     number of transactions: -0
     = Total value of -0 lovelace across 0 UTxOs
      ... 10                0
      ... 100               0
      ... 1000              0
      ... 10000             0
      ... 100000            0
      ... 1000000           0
      ... 10000000          0
      ... 100000000         0
      ... 1000000000        0
      ... 10000000000       0
      ... 100000000000      0
      ... 1000000000000     0
      ... 10000000000000    0
      ... 100000000000000   0
      ... 1000000000000000  0
      ... 10000000000000000 0
      ... 45000000000000000 0
                            
BenchRndResults:            
  benchName: 0.1-percent-rnd
  restoreTime: 126.8 s      
  readWalletTime: 5.034 ms  
  listAddressesTime: 4.184 ms
  listTransactionsTime: 101.6 ms
  importOneAddressTime: 3.865 ms
  importManyAddressesTime: 84.47 ms
  estimateFeesTime: 8.039 ms
  walletOverview:           
     number of addresses: 83
     number of transactions: 110
     = Total value of 5922204907552 lovelace across 51 UTxOs
      ... 10                0
      ... 100               0
      ... 1000              0
      ... 10000             1
      ... 100000            0
      ... 1000000           2
      ... 10000000          1
      ... 100000000         2
      ... 1000000000        12
      ... 10000000000       11
      ... 100000000000      18
      ... 1000000000000     4
      ... 10000000000000    0
      ... 100000000000000   0
      ... 1000000000000000  0
      ... 10000000000000000 0
      ... 45000000000000000 0
                            
BenchRndResults:            
  benchName: 0.2-percent-rnd
  restoreTime: 131.1 s      
  readWalletTime: 10.35 ms  
  listAddressesTime: 9.818 ms
  listTransactionsTime: 153.9 ms
  importOneAddressTime: 8.465 ms
  importManyAddressesTime: 101.3 ms
  estimateFeesTime: 9.720 ms
  walletOverview:           
     number of addresses: 198
     number of transactions: 250
     = Total value of 14480655251559 lovelace across 116 UTxOs
      ... 10                1
      ... 100               1
      ... 1000              1
      ... 10000             1
      ... 100000            2
      ... 1000000           3
      ... 10000000          1
      ... 100000000         6
      ... 1000000000        22
      ... 10000000000       26
      ... 100000000000      42
      ... 1000000000000     10
      ... 10000000000000    0
      ... 100000000000000   0
      ... 1000000000000000  0
      ... 10000000000000000 0
      ... 45000000000000000 0
                            
BenchRndResults:            
  benchName: 0.4-percent-rnd
  restoreTime: 139.5 s      
  readWalletTime: 20.22 ms  
  listAddressesTime: 24.88 ms
  listTransactionsTime: 268.2 ms
  importOneAddressTime: 18.06 ms
  importManyAddressesTime: 123.9 ms
  estimateFeesTime: 14.08 ms
  walletOverview:           
     number of addresses: 417
     number of transactions: 479
     = Total value of 32229614849981 lovelace across 270 UTxOs
      ... 10                1
      ... 100               3
      ... 1000              1
      ... 10000             3
      ... 100000            8
      ... 1000000           4
      ... 10000000          6
      ... 100000000         21
      ... 1000000000        45
      ... 10000000000       59
      ... 100000000000      97
      ... 1000000000000     22
      ... 10000000000000    0
      ... 100000000000000   0
      ... 1000000000000000  0
      ... 10000000000000000 0
      ... 45000000000000000 0
                            
BenchSeqResults:            
  benchName: 0.1-percent-seq
  restoreTime: 137.7 s      
  readWalletTime: 5.483 ms  
  listAddressesTime: 14.59 ms
  listTransactionsTime: 88.57 ms
  estimateFeesTime: 15.01 ms
  walletOverview:           
     number of addresses: 103
     number of transactions: 110
     = Total value of 5922204907552 lovelace across 51 UTxOs
      ... 10                0
      ... 100               0
      ... 1000              0
      ... 10000             1
      ... 100000            0
      ... 1000000           2
      ... 10000000          1
      ... 100000000         2
      ... 1000000000        12
      ... 10000000000       11
      ... 100000000000      18
      ... 1000000000000     4
      ... 10000000000000    0
      ... 100000000000000   0
      ... 1000000000000000  0
      ... 10000000000000000 0
      ... 45000000000000000 0
                            
BenchSeqResults:            
  benchName: 0.2-percent-seq
  restoreTime: 142.5 s      
  readWalletTime: 8.658 ms  
  listAddressesTime: 8.180 ms
  listTransactionsTime: 125.9 ms
  estimateFeesTime: 17.37 ms
  walletOverview:           
     number of addresses: 218
     number of transactions: 250
     = Total value of 14480655251559 lovelace across 116 UTxOs
      ... 10                1
      ... 100               1
      ... 1000              1
      ... 10000             1
      ... 100000            2
      ... 1000000           3
      ... 10000000          1
      ... 100000000         6
      ... 1000000000        22
      ... 10000000000       26
      ... 100000000000      42
      ... 1000000000000     10
      ... 10000000000000    0
      ... 100000000000000   0
      ... 1000000000000000  0
      ... 10000000000000000 0
      ... 45000000000000000 0
                            
BenchSeqResults:            
  benchName: 0.4-percent-seq
  restoreTime: 149.0 s      
  readWalletTime: 35.49 ms  
  listAddressesTime: 24.57 ms
  listTransactionsTime: 268.5 ms
  estimateFeesTime: 27.87 ms
  walletOverview:           
     number of addresses: 437
     number of transactions: 479
     = Total value of 32229614849981 lovelace across 270 UTxOs
      ... 10                1
      ... 100               3
      ... 1000              1
      ... 10000             3
      ... 100000            8
      ... 1000000           4
      ... 10000000          6
      ... 100000000         21
      ... 1000000000        45
      ... 10000000000       59
      ... 100000000000      97
      ... 1000000000000     22
      ... 10000000000000    0
      ... 100000000000000   0
      ... 1000000000000000  0
      ... 10000000000000000 0
      ... 45000000000000000 0

[KtorZ]

bors r+

[iohk-bors]

Canceled.

[KtorZ]

bors r+

[iohk-bors]

Build failed (retrying...):

• buildkite/cardano-wallet

[iohk-bors]

Build succeeded:

• buildkite/cardano-wallet
• ci/hydra-build:required