Ethash frontier update

[zelig]

This issue collected all ethash related issues. @LefterisJP
we need to make sure all these tick off before frontier.

• integrate minerthreads in Ethereum.StartMining
• pass minerthreads admin JS admin.miner.start
• pass minerthreads in CLI flag
• ? miner tests using the test mini-DAG
• update CLI makedag subcommand to support generating DAG for any epoch
• check blocktests running time and unskip them if ok

---

• Move DAG file handling to ethash enhancement question Move DAG file handling to ethash #555
• ethash crashes on epoch when mining bug ethash crashes on epoch when mining #705
• Eventer blocked by miner bug Eventer blocked by miner  #697
• Mining on private network & wrong DAG epoch Mining on private network & wrong DAG epoch #674
• Ethereum CLI - Windows - runtime: out of memory: cannot allocate 1073741824-byte block Ethereum CLI - Windows - runtime: out of memory: cannot allocate 1073741824-byte block #562
• fatal error: unexpected signal during runtime execution on Windows 64bit bug fatal error: unexpected signal during runtime execution on Windows 64bit #524
• SEC-24 Parent issue for ethash / PoW security issues vuln SEC-24 Parent issue for ethash / PoW security issues #499
• Mining progress can possibly be stunted by determined attacker if difficulty gets high enough Mining progress can possibly be stunted by determined attacker if difficulty gets high enough #274
• SIGABRT: abort during cgo execution SIGABRT: abort during cgo execution #746

[LefterisJP]

I will help as much as I can, but golang is not my specialty. I am currently working on making the GO bridge use the new API. I will open a Pull Request and request feedback tomorrow probably.

[fjl]

The new bridge is almost done, see ethereum/ethash#41 for progress.

[Gustav-Simonsson]

Integration PR: #808

[Gustav-Simonsson]

Update: After debugging a bunch of bugs with @LefterisJP we now have a version where DAG creation works through Go bridge and mining works both on private chain and on testnet. Will verify it for block tests as well (their startup needs some updates to work with current develop branch)

One issue we found is that this call: https://github.com/LefterisJP/ethash/blob/finalize_and_use_new_api/ethash.go#L79 causes the Go garbage collector to run https://github.com/LefterisJP/ethash/blob/finalize_and_use_new_api/ethash.go#L67 before the C function is done with building the DAG.

This results in the C code running into segfault when trying to reference the cache in https://github.com/LefterisJP/ethash/blob/finalize_and_use_new_api/src/libethash/internal.c#L93

Commenting out the finalizer call avoids deallocating the cache before DAG building is done.

We should find out why the Go runtime / GC considers this cache to be unreachable before this call returns: https://github.com/LefterisJP/ethash/blob/finalize_and_use_new_api/ethash.go#L169

[Gustav-Simonsson]

Edit: moving the finalizer until after the call also works and should have same intended behavior.

[Gustav-Simonsson]

Another update: @fjl suggested a better way to clear the cache by moving the finalizer call to getCache and explicitly free the temp cache in makeDAG

[obscuren]

All that's left to do is add miner control over the CPU's being used. It will require some significant re-write on the miner's part since it initialises the amount of agents on init.

[zelig]

now only 1 issue left, support miniDAG for testing. but this is not a frontier show stopper

[fjl]

                                                                                  You can do this now. Just use ethash.NewForTesting.

[zelig]

i know we can, we just dont yet :) but on second thought that shouldnt be an ethash issue any more. closing this ftw