Replace the fake-bitcoin-cli with a proxy that can be instrumented from tests

[cdecker]

Fixes #1858

[rustyrussell]

Really want to get the release out soon....

[cdecker]

Really want to get the release out soon....

This wasn't intended to hold up the release, just some cleanups while you were addressing the WIP flag of #1846. Sorry for the confusion :-)

[cdecker]

This is ready for review, but we should hold off from merging it until 0.6.1 is released.

[rustyrussell]

The commits are a bit sloppy :) Seeming that younger ones fix up older commits a few times, though.

But I was trting to do before/after timing tests, and test_logging fails:

[gw7] [ 91%] FAILED tests/test_misc.py::test_logging 
===Flaky Test Report===


===End Flaky Test Report===

=================================== FAILURES ===================================
_________________________________ test_logging _________________________________
[gw7] linux -- Python 3.6.5 /usr/bin/python3

node_factory = <utils.NodeFactory object at 0x7fde146dc748>

    def test_logging(node_factory):
        # Since we redirect, node.start() will fail: do manually.
        l1 = node_factory.get_node(options={'log-file': 'logfile'}, may_fail=True, start=False)
        logpath = os.path.join(l1.daemon.lightning_dir, 'logfile')
        logpath_moved = os.path.join(l1.daemon.lightning_dir, 'logfile_moved')
    
        TailableProc.start(l1.daemon)
        wait_for(lambda: os.path.exists(logpath))
    
        shutil.move(logpath, logpath_moved)
        l1.daemon.proc.send_signal(signal.SIGHUP)
        wait_for(lambda: os.path.exists(logpath_moved))
>       wait_for(lambda: os.path.exists(logpath))

tests/test_misc.py:937: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

success = <function test_logging.<locals>.<lambda> at 0x7fde14653950>
timeout = 60, interval = 0.1

    def wait_for(success, timeout=TIMEOUT, interval=0.1):
        start_time = time.time()
        while not success() and time.time() < start_time + timeout:
            time.sleep(interval)
        if time.time() > start_time + timeout:
>           raise ValueError("Error waiting for {}", success)
E           ValueError: ('Error waiting for {}', <function test_logging.<locals>.<lambda> at 0x7fde14653950>)

tests/utils.py:47: ValueError
------------------------------ Captured log setup ------------------------------
utils.py                    99 DEBUG    Starting 'bitcoind -datadir=/tmp/ltests-g4iq2pzy/test_logging_1 -printtoconsole -server -logtimestamps -nolisten'
utils.py                   179 DEBUG    Waiting for ['Done loading'] in the logs
utils.py                   201 DEBUG    Found 're.compile('Done loading')' in logs
utils.py                   281 INFO     BitcoinD started
----------------------------- Captured stderr call -----------------------------
Could not connect to bitcoind using bitcoin-cli. Is bitcoind running?

Make sure you have bitcoind running and that bitcoin-cli is able to connect to bitcoind.

You can verify that your Bitcoin Core installation is ready for use by running:

    $ bitcoin-cli -regtest -rpcuser=rpcuser -rpcpassword=rpcpass echo 'hello world'
------------------------------ Captured log call -------------------------------
utils.py                    99 DEBUG    Starting 'lightningd/lightningd --addr=127.0.0.1:34337 --allow-deprecated-apis=false --bitcoin-rpcpassword=rpcpass --bitcoin-rpcuser=rpcuser --cltv-delta=6 --cltv-final=5 --dev-bitcoind-poll=1 --dev-broadcast-interval=1000 --dev-fail-on-subdaemon-fail --dev-no-reconnect --disable-dns --ignore-fee-limits=false --lightning-dir=/tmp/ltests-g4iq2pzy/test_logging_1/lightning-1/ --log-file=logfile --log-level=debug --network=regtest --rescan=1 --watchtime-blocks=5'
!!!!!!!!!!!! xdist.dsession.Interrupted: stopping after 1 failures !!!!!!!!!!!!!
==================== 1 failed, 115 passed in 258.70 seconds ====================

[rustyrussell]

Ack, with the fixup.

Confused how this got past CI. Please fold if it's right, and apply...

[cdecker]

ACK 4a603da