fix: Fix reentrancy bug in read_event()

Per the external documentation, there is an invariant that read_event()
will not call back into the SocketDescriptor, but the actual code
flushed the outbound queue.

Fix the bug and use the type system to guarantee this behavior in
the future. By changing the function parameters to use `impl
PayloadQueuer` the function will only have access to functions on that
interface throughout the execution, but still be able to pass the
OutboundQueue object since it satisfies the trait bounds.

Functions such as enqueue_message() also take a `impl PayloadQueuer` so
they can be called from that context, but functions that need the
SocketDescriptorFlusher interface will fail at compile-time and point to
the issue before any tests need to be run.

This also fixes up tests and the tokio implementation that did not
implement the proper behavior and relied on read_event() calling
send_data().

Author: julianknutsen

lightning-net-tokio/src/lib.rs

@@ -189,17 +189,20 @@ impl Connection {
 					Ok(len) => {
 						prepare_read_write_call!();
 						let read_res = peer_manager.read_event(&mut our_descriptor, &buf[0..len]);
-						let mut us_lock = us.lock().unwrap();
-						match read_res {
-							Ok(pause_read) => {
-								if pause_read {
-									us_lock.read_paused = true;
-								}
-								Self::event_trigger(&mut us_lock);
-							},
-							Err(e) => shutdown_socket!(e, Disconnect::CloseConnection),
+						{
+							let mut us_lock = us.lock().unwrap();
+							match read_res {
+								Ok(pause_read) => {
+									if pause_read {
+										us_lock.read_paused = true;
+									}
+									Self::event_trigger(&mut us_lock);
+								},
+								Err(e) => shutdown_socket!(e, Disconnect::CloseConnection),
+							}
+							us_lock.block_disconnect_socket = false;
 						}
-						us_lock.block_disconnect_socket = false;
+						peer_manager.process_events();
 					},
 					Err(e) => shutdown_socket!(e, Disconnect::PeerDisconnected),
 				},