-
Notifications
You must be signed in to change notification settings - Fork 4
My Notary Operator notes
Here is my notes and hints how to diagnose and fix issues with running iguana and providing necessary environment for it, collected while being a NN operator.
NN operator's wallet should have several unspent utxos of 10,000 sat (or 100,000 for some coins) to create notarisations. To maintain those utxos user usually periodically calls iguana splitfunds rpc which sometimes may fail with 'couldnt create duplicates tx', so you eventually may run out of utxos needed for dPoW
I found out several possible reasons why such an error may begin to appear (although splitfunds may have worked well until a certain moment). See them below.
Your notary address amount is too low. Most probably you just do not have sufficient funds on your NN address. If this is the case you need just top up your address. (The address may run out of coins because it transacted too often and spend coins on miner fees.)
However, sometimes it may look that you have sufficient amount in the whole wallet but splitfunds returns the above error. I collected several cases that may lead to this issue:
Funds migrated to other addresses. The 'insufficient funds' problem may be also due to funds migrated to other addresses in the wallet. I have not found yet why this may happen but I noticed that for coins KMD, AYA, MIL, EMC2 the primary NN address had too low amount and new addresses appeared in the wallet which had the most of my amount. By the way, I believe I have not done any funds movement for coins like MIL and AYA and EMC2 myself and I think maybe iguana sometimes uses sendtoaddress rpc or some other rpc to move funds what causes the 'change' to go to a new address (TODO to myself: investigate why new addresses appear in the wallet).
How to fix: periodically send all funds to the NN address.
Funds are on coinbase P2PK output. The insufficient funds issue may be also due to most of your funds being on the coinbase P2PK address. Iguana splitfunds rpc does not use P2PK outputs to create 10000 sat utxos for notarisation (as it may think of them as utxos created for notarisations?). Check with listunspent rpc - maybe you have most of the funds on P2PK address.
How to fix: use sendtoaddress rpc to send all funds to P2PKH NN address, then use splitfunds rpc.
Empty listunspent. Another case with the insufficient funds issue happened in coins like AYA and EMC2. When the 'couldnt create duplicates tx' error appeared in those coins I again noticed several new addresses in the wallet. Nevertheless I managed to use sendtoaddress to send all funds to the actual NN address. After that to recover listunspent results on AYA I used its rescanblockchain rpc.
NOTE: listunspent rpc by default returns only transactions in blocks (so skips txns in mempool. Use listunspent 0 to check txns also in mempool). Apparently iguana splitfunds rpc also uses only confirmed transactions. So you need to check if you have 1000 utxos in mempool and wait until they are mined.
You recently sent some of your funds to other addresses and P2PKH output became empty. If you sent coins to other addresses your P2PKH account (used for notarisations) may be drained first, so you may have plenty of coins in your wallet but all on the coinbase P2PK output. Transfer some coins to P2PKH address for notarisation to work again.
NOTE: listunspent rpc may temporarily show an empty list or may not show some unspent outputs if some transactions are not yet confirmed. So wait until all txns are confirmed or use listunspent with minconf param set to -1 to show unconfirmed too.
NOTE Until this PR, for notary nodes, the transaction change (for e.g. when sendtoaddress rpc is called) was always sent to the P2PK output with the notary pubkey. This PR fixes this case by creating change outputs as P2PKH, reducing possibility of draining utxo for splitting.
Chain is not progressing. For MIL I also had the splitfunds error and I noticed that listunspent also showed an empty list. It turned out this chain had not been progressed in months. Apparently I forgot about a notification that it had gone to hibernation.
To diagnose more you may restart iguana with stdout and stderr output streams redirected into files like iguana.log and error.log for convenience (iguana ... 1>iguana.log 2>error.log). Additionally you may enable extra printf in the iguana C code (for this splitfunds error printfs of interest are in iguana_utxoduplicates() iguana_calcrawtx() iguana_signrawtx() functions) and rebuild it.
If to look into iguana logs, the 'insufficient funds' issue may expose itself as log messages like these:
- There is 'error signing raw utxoduplicates' log message.
- The created tx may have no vins (to see this you may enable a printf() to print vins in iguana_calcrawtx()).
- Also the log printf("avail %.8f satoshis %.8f, txfee %.8f burnamount %.8f vin0.scriptlen %d num.%d",...) may show less 'avail' value than needed 'satoshi + txfee' (f.e. if you have most of amount on P2PK)
Normally we use validateaddress address for that but for AYA this command does not show pubkey. They have getaddressinfo address instead.