Noticed by stracing for an unrelated problem.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

1. Freeing an unconfirmed channel already releases the subd, so don't
   do that explicitly.
2. Use channel->owner to transfer ownership where possible, using
   channel_set_owner() which handles all the cases.

This simplifies the code and makes it more readable, IMHO.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

We don't need the other "gossip" messages in our echo-suppression filter.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

They all returned the same thing anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

connectd is going to end up using this do demux; make it fast and complete.

Fixing this reveals a problem in openingd: it now extracts the channel_id
from funding_signed (which is where we transition off the temporary), and
gets upset.  So fix that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

The "read until closed" trick doesn't work if the other end doesn't
close (as found in the next patch, where we use DEV_DISCONNECT_DISABLE_AFTER).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

These would have to be done by connectd, not the local daemon, once it's
intermediating.  Otherwise the remote peer won't see any change.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Once connectd is doing this, we can't close as soon as we send,
and in fact we can't do 'fail write' either.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

…ommit

It was always a hack, but an impossible one once connectd will be
interpreting dev-disconnect!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Increasingly we want to know is it local, and get the direction: it's
more efficient to do both at once.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

local_chan was mainly around so we could "soft" disable channels (and
really disable them once we used the channel_update in an error
message).

Instead we introduce the idea of a "deferred_update": it's either
deferred indefinitely (a peer goes offline, if we need to send it in
an error we'll apply it immediatly), or simply delayed to avoid
spamming everyone.

The resulting rewrite is much clearer, IMHO.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

They all returned the next io_plan, but it was always the same.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

We want to have a real (persistent) struct peer eventually.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

connectd will be keeping the conn open, so it needs to free this
"conn_timeout" timer.  Pass it through, so we can do that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Instead of passing the incoming socket to lightningd for the
subdaemon, create a new one and simply shuffle data between them,
keeping connectd in the loop.

For the moment, we don't decrypt at all, just shuffle.  This means our
buffer code is kind of a hack, but that goes away once we start
actually decrypting and understanding message boundaries.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

This avoids changes to crypto_sync which are coming in next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

We temporarily hack to sync_crypto_write/sync_crypto_read functions to
not do any crypto, and do it all in connectd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Now connectd is doing the crypto, we can use normal wire io.  We
create helper functions to clearly differentiate between "peer" comms
and intra-daemon comms though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Now that connectd does the crypto, no need to hand around crypto_state.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

As connectd handles more packets itself, or diverts them to/from gossipd,
it's the only place we can implement the dev_disconnect logic.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

channeld can't do it any more: it's using local sockets.  Connectd
can do it, and simply does it by type.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

We actually intercept the gossip_timestamp_filter, so the gossip_store
mechanism inside the per-peer daemon never kicks off for normal connections.

The gossipwith tool doesn't set OPT_GOSSIP_QUERIES, so it gets both, but
that only effects one place.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

We now let gossipd do it.

This also means there's nothing left in 'struct per_peer_state' to
send across the wire (the fds are sent separately), so that gets
removed from wire messages too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Just hand around the two fds.  We have to take a little care to close them
manually now, though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

We're the only ones who use it now, so put our fields inside it and
make it local.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

It's weird to have connectd ask gossipd, when lightningd can just do it
and hand all the addresses together.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

This is in preparation for gossipd feeding us the latest channel_updates,
rather than having lightningd and channeld query gossipd when it wants
to send an onion error with an update included.

This means gossipd will start telling us the updates, so we need the
channels loaded first.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

We want it to keep the latest, so it can make its own error msgs without
asking us.  This installs (but does not use!) the message handler.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Even if we're deferring putting them in the store and broadcasting them,
we tell lightningd so it will use it in any error messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

This way it can flush it if it was pending.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

…g gossipd.

We also no longer strip the type off: everyone handles both forms, and
Eclair doesn't strip (and it's easier!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Now we don't ask gossipd, but lightningd keeps channeld up-to-date.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

This only happened once, but it's possible if the node gets the new
node announcement while we're querying, we could see both old and new.

```
        # Get node_announcements.
        msgs = l1.query_gossip('gossip_timestamp_filter',
                               genesis_blockhash,
                               '0', '0xFFFFFFFF',
                               # Filter out gossip_timestamp_filter,
                               # channel_announcement and channel_updates.
                               filters=['0109', '0102', '0100'])
    
>       assert len(msgs) == 2
E       assert 3 == 2
E         +3
E         -2
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

We're weaning per-peer daemons off having a direct gossipd connection.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

We're about to change this to go via lightningd, not direct to gossipd,
and connectd will send gossip direct to gossipd.

Once we do this, it introduces a race.  The peer sends funding_locked
then immediately sends an update_channel.  channeld used to process
the funding_locked from the peer, tell gossipd about the new
channel, then finally forward the channel_update.

With the new architecture, we can have the channel_update hit gossipd
before we've told it about the channel.  It ignores the channel_update.

Better is to tell gossipd as soon as *we* are ready to send
funding_locked; this gives a round trip time at least, before gossipd
could get the update.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Once we send funding_locked, gossipd could start seeing channel_updates
from the peer (which get sent so we can use the channel in routehints
even before it's announcable).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

We want to stream gossip through this, but currently connectd treats the
fd as synchronous.  While we work on getting rid of that, it's easiest to
have two fds.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON_RPC: `ping` now works with connected peers, even without a channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

This is neater than what we had before, and slightly more general.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON_RPC: `sendcustommsg` now works with any connected peer, even when shutting down a channel.

We currently die when gossipd vanishes, but our direct connection will
go away.  We then complain if the node is shutting down while we're talking
to hsmd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Gossipd now simply gets told by channeld when peers arrive or leave.
(it only needs to know for the seeker).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Now we only send and receive gossip messages on this fd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

…msg.

We don't need the connection to ourselves!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

I'm guessing doing a 16 bit swap on WIRE_GOSSIP_STORE_ENDED is the solution

Error on Mac OSX 11.6:
common/gossip_store.c:132:16: error: result of comparison of constant 152043520 with expression of type 'be16' (aka 'unsigned short') is always false [-Werror,-Wtautological-constant-out-of-range-compare]
                if (buf.type == CPU_TO_BE32(WIRE_GOSSIP_STORE_ENDED))

$ gcc -v
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
Apple clang version 13.0.0 (clang-1300.0.29.30)
Target: x86_64-apple-darwin20.6.0