-
Notifications
You must be signed in to change notification settings - Fork 1k
Error Codes
Here is an overview of the most common error codes and possible fixes.
The block with height <i> found share <x> which is < than <y> of block <j> | Sometimes stratum inserts
####Example
2013-10-24 9:03:43 - FATAL --> E0001: The block with height 107643 found share 18 which is < than 19 of block 107642
####Explanation
The findblocks cron has found a block which has a share ID lower than a previous block found. This can happen with fast block finding coins or if the shares tables ID column had a roll over (starting at ID of 1 again).
####Fix
This is not a fatal error yet unless the shares ID column rolled over. findblocks will try to autofix the issue by changing the shares. It will re-order the shares so a following run should not fail.
If a rollover is found, the fix is to run an update on the blocks table to mark all blocks with valid share_id fields to a share_id of 1. That way the cron will skip the E0001 detection and continue to run with the rolled over share_ids. The round boundaries might be wrong, so a possible solution would be to assign a share manually with the highest non-rolled over share_id. That way the round should be calculated properly.
There is no way for MPOS to auto-fix rolled over shares table since it would remove the ability for this and other error detection systems.
E0002: Failed to fetch both offending shares: <x> and <y>. . Block height: <n>
findblocks was not able to fetch both shares that are causing the Out of Order problem. Sometimes, if blocks have been accounted for already, shares can't be fetched because they were deleted.
Since a block has already been accounted for, there will need to be manually assigned a share to the found block <n>. Some things to keep in mind:
- Do not use a valid
upstream_resultshare - Do not include a valid
upstream_resultshare in that round - Use more than just a few shares if possible to make it a round
- Add valid block finder ID for block
<n> - Add valid total shares for block
<n>
E0003: Failed to change shares order!
findblocks tried to fix the shares order by updating both shares. This failed for some reason.
Nothing automated can be done here. Check for those blocks in question which share_ids did not get updated. Once found the solution field between those two shares can be manually switched. That will bring the blocks back in order with the shares.
Failed to reset block in database: <n>
findblocks successfully changed the out of order shares but failed to update the shares, share_id and account_id information in the block <n>.
Manually reset that block and try again.
E0005: Unable to fetch blocks upstream share, aborted: <error>
None of the available shares were matched for this block. This should never happen due to the fallbacks implemented to match a block against a share. If such an issue arrises, try assigning shares manually.
See E0002
E0015: Potential Double Payout detected
For each block that is going to be paid out, a new value is stored in the settings table: last_accounted_block_id. Note that the system stores the ID of the previously paid out block, not the height. Usually this procedure is pretty solid, but occasionally users run into this error. Most of the time, it was caused by an orphaned block not marked as orphaned at the time the payouts started to run. This is not something MPOS can catch, blockupdate is run prior to doing a payout, but with some coins blocks are not marked as orphaned in the upstream RPC so MPOS wouldn't know about it. Rare occasions showed that crons did not run through, then manual intervention is required.
If a payout failed in mid-run, first check for previous errors. If the block has not been confirmed yet, a potentially fix would be to delete all transactions related to this block and try to re-run it. If the block did confirm and has already paid out to users, all that can be done is to mark it as accounted. The last block being paid out can be found in the settings table under last_accounted_block_id. That block is probably not marked as accounted. Change the flag.
If users did not get paid out, the cron will not be able to recover if shares have been deleted already.
In case of an orphan block not marked properly, just re-run the cron with the force option.
E0062: Block has no share_id, not running payouts
This is usually a follow up error caused by findblocks not running properly. Check the findblocks logfile and fix those errors, then payouts should continue once you force the run. If not, the same error message will appear.
See above errors for a fix in findblocks cron.
E0063: Upstream share already assigned to previous block
A share was assigned to two blocks simultaneously due to a fast find rate for the pool.
Two ways, the easy and the hard way:
EASY
- Remove the block in question and force runs
- If payouts were done already, those are probably wrong
- If no payouts were done, this block is lost for payouts
HARD
- Assign the correct share to the new block
- Delete all transactions (if any) from that block
- This clears out any wrong transactions for this block
- Mark the block as unaccounted, if not already
- Allow another payout run to pick it up as unapid
- Change the
last_accounted_block_idin thesettingstable to the block ID before - We want this block to payout now, so we have to go back one block
- Rerun the run-payouts.sh cron
E0078: RPC method did not return 200 OK
This is caused by the RPC service not accepting the sendtoaddress call for a payout with a proper http 200 OK code. Some coins are causing issues by accepting the sendtoaddress call internally but sending a http 500 Internal server error back to MPOS during processing. For MPOS it looked like the transaction failed but it was indeed accepted by the RPC. To be sure, we are now failing the cronjob and give Pool Ops a chance to check.
Workaround: Please check your coind transactions for the payout that you can find in the payouts logfile. You can use the supplied scripts/validate_payouts.php script to find any missing records in your coind. Please double check those, the script is not 100% perfect and may sometimes not pick up a transaction when it should. If you are missing that users payout that caused the error, manually trigger a sendtoaddress call from the wallet to pay them out, then force a cron run to restart the payout process.
Keep in mind: This is a workaround for an issue upstream. It does the trick and protects pool operators from double payouts.
E0081: Failed to insert new block into database
During high load on a database findblocks could run into timeouts when waiting for a transaction lock. You will find this error in your logfile, finding new blocks is aborted until this block has been ported into the database.
You could try running the findblock cron again. Since the block was not added to the database, another run should try to pick it up again and add it. If this does not succeed find your bottleneck. It could be related to the archive cleanup not working properly and your archive growing way too large. This can be an issue when running PPLNS based payouts.
If that fails and forcing the cron did not fix it, you can insert it manually into the database. The values required are visible in the logfile. The blocks information is printed before the error message in a table format. Add those into your blocks table and try running your cron again.
-
- installation of Redis requires TCL 8.5 or newer TCL Download Page