Skip to content

Commit

Permalink
Merge bitcoin/bitcoin#19160: multiprocess: Add basic spawn and IPC su…
Browse files Browse the repository at this point in the history
…pport

84934bf multiprocess: Add echoipc RPC method and test (Russell Yanofsky)
7d76cf6 multiprocess: Add comments and documentation (Russell Yanofsky)
ddf7ecc multiprocess: Add bitcoin-node process spawning support (Russell Yanofsky)
10afdf0 multiprocess: Add Ipc interface implementation (Russell Yanofsky)
745c9ce multiprocess: Add Ipc and Init interface definitions (Russell Yanofsky)
5d62d7f Update libmultiprocess library (Russell Yanofsky)

Pull request description:

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).

  ---

  This PR adds basic process spawning and IPC method call support to `bitcoin-node` executables built with `--enable-multiprocess`[*].

  These changes are used in bitcoin/bitcoin#10102 to let node, gui, and wallet functionality run in different processes, and extended in bitcoin/bitcoin#19460 and bitcoin/bitcoin#19461 after that to allow gui and wallet processes to be started and stopped independently and connect to the node over a socket.

  These changes can also be used to implement new functionality outside the `bitcoin-node` process like external indexes or pluggable transports (bitcoin/bitcoin#18988). The `Ipc::spawnProcess` and `Ipc::serveProcess` methods added here are entry points for spawning a child process and serving a parent process, and being able to make bidirectional, multithreaded method calls between the processes. A simple example of this is implemented in commit "Add echoipc RPC method and test."

  Changes in this PR aside from the echo test were originally part of #10102, but have been split and moved here for easier review, and so they can be used for other applications like external plugins.

  Additional notes about this PR can be found at https://bitcoincore.reviews/19160

  [*] Note: the `--enable-multiprocess` feature is still experimental, and not enabled by default, and not yet supported on windows. More information can be found in [doc/multiprocess.md](https://github.com/bitcoin/bitcoin/blob/master/doc/multiprocess.md)

ACKs for top commit:
  fjahr:
    re-ACK 84934bf
  ariard:
    ACK 84934bf. Changes since last ACK fixes the silent merge conflict about `EnsureAnyNodeContext()`. Rebuilt and checked again debug command `echoipc`.

Tree-SHA512: 52a948b5e18a26d7d7a09b83003eaae9b1ed2981978c36c959fe9a55abf70ae6a627c4ff913a3428be17400a3dace30c58b5057fa75c319662c3be98f19810c6
  • Loading branch information
laanwj committed Apr 27, 2021
2 parents 19a56d1 + 84934bf commit ac219dc
Show file tree
Hide file tree
Showing 30 changed files with 805 additions and 13 deletions.
3 changes: 3 additions & 0 deletions build_msvc/bitcoind/bitcoind.vcxproj
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@
</PropertyGroup>
<ItemGroup>
<ClCompile Include="..\..\src\bitcoind.cpp" />
<ClCompile Include="..\..\src\init\bitcoind.cpp">
<ObjectFileName>$(IntDir)init_bitcoind.obj</ObjectFileName>
</ClCompile>
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\libbitcoinconsensus\libbitcoinconsensus.vcxproj">
Expand Down
4 changes: 2 additions & 2 deletions depends/packages/native_libmultiprocess.mk
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
package=native_libmultiprocess
$(package)_version=5741d750a04e644a03336090d8979c6d033e32c0
$(package)_version=d576d975debdc9090bd2582f83f49c76c0061698
$(package)_download_path=https://github.com/chaincodelabs/libmultiprocess/archive
$(package)_file_name=$($(package)_version).tar.gz
$(package)_sha256_hash=ac848db49a6ed53e423c62d54bd87f1f08cbb0326254a8667e10bbfe5bf032a4
$(package)_sha256_hash=9f8b055c8bba755dc32fe799b67c20b91e7b13e67cadafbc54c0f1def057a370
$(package)_dependencies=native_capnp

define $(package)_config_cmds
Expand Down
39 changes: 38 additions & 1 deletion doc/multiprocess.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Specific next steps after [#10102](https://github.com/bitcoin/bitcoin/pull/10102

## Debugging

After [#10102](https://github.com/bitcoin/bitcoin/pull/10102), the `-debug=ipc` command line option can be used to see requests and responses between processes.
The `-debug=ipc` command line option can be used to see requests and responses between processes.

## Installation

Expand All @@ -33,3 +33,40 @@ BITCOIND=bitcoin-node test/functional/test_runner.py
The configure script will pick up settings and library locations from the depends directory, so there is no need to pass `--enable-multiprocess` as a separate flag when using the depends system (it's controlled by the `MULTIPROCESS=1` option).

Alternately, you can install [Cap'n Proto](https://capnproto.org/) and [libmultiprocess](https://github.com/chaincodelabs/libmultiprocess) packages on your system, and just run `./configure --enable-multiprocess` without using the depends system. The configure script will be able to locate the installed packages via [pkg-config](https://www.freedesktop.org/wiki/Software/pkg-config/). See [Installation](https://github.com/chaincodelabs/libmultiprocess#installation) section of the libmultiprocess readme for install steps. See [build-unix.md](build-unix.md) and [build-osx.md](build-osx.md) for information about installing dependencies in general.

## IPC implementation details

Cross process Node, Wallet, and Chain interfaces are defined in
[`src/interfaces/`](../src/interfaces/). These are C++ classes which follow
[conventions](developer-notes.md#internal-interface-guidelines), like passing
serializable arguments so they can be called from different processes, and
making methods pure virtual so they can have proxy implementations that forward
calls between processes.

When Wallet, Node, and Chain code is running in the same process, calling any
interface method invokes the implementation directly. When code is running in
different processes, calling an interface method invokes a proxy interface
implementation that communicates with a remote process and invokes the real
implementation in the remote process. The
[libmultiprocess](https://github.com/chaincodelabs/libmultiprocess) code
generation tool internally generates proxy client classes and proxy server
classes for this purpose that are thin wrappers around Cap'n Proto
[client](https://capnproto.org/cxxrpc.html#clients) and
[server](https://capnproto.org/cxxrpc.html#servers) classes, which handle the
actual serialization and socket communication.

As much as possible, calls between processes are meant to work the same as
calls within a single process without adding limitations or requiring extra
implementation effort. Processes communicate with each other by calling regular
[C++ interface methods](../src/interfaces/README.md). Method arguments and
return values are automatically serialized and sent between processes. Object
references and `std::function` arguments are automatically tracked and mapped
to allow invoked code to call back into invoking code at any time, and there is
a 1:1 threading model where any thread invoking a method in another process has
a corresponding thread in the invoked process responsible for executing all
method calls from the source thread, without blocking I/O or holding up another
call, and using the same thread local variables, locks, and callbacks between
calls. The forwarding, tracking, and threading is implemented inside the
[libmultiprocess](https://github.com/chaincodelabs/libmultiprocess) library
which has the design goal of making calls between processes look like calls in
the same process to the extent possible.
47 changes: 44 additions & 3 deletions src/Makefile.am
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ EXTRA_LIBRARIES += \
$(LIBBITCOIN_CONSENSUS) \
$(LIBBITCOIN_SERVER) \
$(LIBBITCOIN_CLI) \
$(LIBBITCOIN_IPC) \
$(LIBBITCOIN_WALLET) \
$(LIBBITCOIN_WALLET_TOOL) \
$(LIBBITCOIN_ZMQ)
Expand Down Expand Up @@ -158,7 +159,10 @@ BITCOIN_CORE_H = \
init.h \
init/common.h \
interfaces/chain.h \
interfaces/echo.h \
interfaces/handler.h \
interfaces/init.h \
interfaces/ipc.h \
interfaces/node.h \
interfaces/wallet.h \
key.h \
Expand Down Expand Up @@ -299,6 +303,8 @@ obj/build.h: FORCE
"$(abs_top_srcdir)"
libbitcoin_util_a-clientversion.$(OBJEXT): obj/build.h

ipc/capnp/libbitcoin_ipc_a-ipc.$(OBJEXT): $(libbitcoin_ipc_mpgen_input:=.h)

# server: shared between bitcoind and bitcoin-qt
# Contains code accessing mempool and chain state that is meant to be separated
# from wallet and gui code (see node/README.md). Shared code should go in
Expand Down Expand Up @@ -558,7 +564,9 @@ libbitcoin_util_a_SOURCES = \
compat/glibcxx_sanity.cpp \
compat/strnlen.cpp \
fs.cpp \
interfaces/echo.cpp \
interfaces/handler.cpp \
interfaces/init.cpp \
logging.cpp \
random.cpp \
randomenv.cpp \
Expand Down Expand Up @@ -634,17 +642,17 @@ bitcoin_bin_ldadd = \

bitcoin_bin_ldadd += $(BOOST_LIBS) $(BDB_LIBS) $(MINIUPNPC_LIBS) $(NATPMP_LIBS) $(EVENT_PTHREADS_LIBS) $(EVENT_LIBS) $(ZMQ_LIBS) $(SQLITE_LIBS)

bitcoind_SOURCES = $(bitcoin_daemon_sources)
bitcoind_SOURCES = $(bitcoin_daemon_sources) init/bitcoind.cpp
bitcoind_CPPFLAGS = $(bitcoin_bin_cppflags)
bitcoind_CXXFLAGS = $(bitcoin_bin_cxxflags)
bitcoind_LDFLAGS = $(bitcoin_bin_ldflags)
bitcoind_LDADD = $(LIBBITCOIN_SERVER) $(bitcoin_bin_ldadd)

bitcoin_node_SOURCES = $(bitcoin_daemon_sources)
bitcoin_node_SOURCES = $(bitcoin_daemon_sources) init/bitcoin-node.cpp
bitcoin_node_CPPFLAGS = $(bitcoin_bin_cppflags)
bitcoin_node_CXXFLAGS = $(bitcoin_bin_cxxflags)
bitcoin_node_LDFLAGS = $(bitcoin_bin_ldflags)
bitcoin_node_LDADD = $(LIBBITCOIN_SERVER) $(bitcoin_bin_ldadd)
bitcoin_node_LDADD = $(LIBBITCOIN_SERVER) $(bitcoin_bin_ldadd) $(LIBBITCOIN_IPC) $(LIBMULTIPROCESS_LIBS)

# bitcoin-cli binary #
bitcoin_cli_SOURCES = bitcoin-cli.cpp
Expand Down Expand Up @@ -808,6 +816,39 @@ if HARDEN
$(AM_V_at) OBJDUMP=$(OBJDUMP) OTOOL=$(OTOOL) $(PYTHON) $(top_srcdir)/contrib/devtools/security-check.py $(bin_PROGRAMS)
endif

libbitcoin_ipc_mpgen_input = \
ipc/capnp/echo.capnp \
ipc/capnp/init.capnp
EXTRA_DIST += $(libbitcoin_ipc_mpgen_input)
%.capnp:

if BUILD_MULTIPROCESS
LIBBITCOIN_IPC=libbitcoin_ipc.a
libbitcoin_ipc_a_SOURCES = \
ipc/capnp/init-types.h \
ipc/capnp/protocol.cpp \
ipc/capnp/protocol.h \
ipc/exception.h \
ipc/interfaces.cpp \
ipc/process.cpp \
ipc/process.h \
ipc/protocol.h
libbitcoin_ipc_a_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES)
libbitcoin_ipc_a_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) $(LIBMULTIPROCESS_CFLAGS)

include $(MPGEN_PREFIX)/include/mpgen.mk
libbitcoin_ipc_mpgen_output = \
$(libbitcoin_ipc_mpgen_input:=.c++) \
$(libbitcoin_ipc_mpgen_input:=.h) \
$(libbitcoin_ipc_mpgen_input:=.proxy-client.c++) \
$(libbitcoin_ipc_mpgen_input:=.proxy-server.c++) \
$(libbitcoin_ipc_mpgen_input:=.proxy-types.c++) \
$(libbitcoin_ipc_mpgen_input:=.proxy-types.h) \
$(libbitcoin_ipc_mpgen_input:=.proxy.h)
nodist_libbitcoin_ipc_a_SOURCES = $(libbitcoin_ipc_mpgen_output)
CLEANFILES += $(libbitcoin_ipc_mpgen_output)
endif

if EMBEDDED_LEVELDB
include Makefile.crc32c.include
include Makefile.leveldb.include
Expand Down
15 changes: 11 additions & 4 deletions src/bitcoind.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
#include <compat.h>
#include <init.h>
#include <interfaces/chain.h>
#include <interfaces/init.h>
#include <node/context.h>
#include <node/ui_interface.h>
#include <noui.h>
Expand Down Expand Up @@ -104,10 +105,8 @@ int fork_daemon(bool nochdir, bool noclose, TokenPipeEnd& endpoint)

#endif

static bool AppInit(int argc, char* argv[])
static bool AppInit(NodeContext& node, int argc, char* argv[])
{
NodeContext node;

bool fRet = false;

util::ThreadSetInternalName("init");
Expand Down Expand Up @@ -254,10 +253,18 @@ int main(int argc, char* argv[])
util::WinCmdLineArgs winArgs;
std::tie(argc, argv) = winArgs.get();
#endif

NodeContext node;
int exit_status;
std::unique_ptr<interfaces::Init> init = interfaces::MakeNodeInit(node, argc, argv, exit_status);
if (!init) {
return exit_status;
}

SetupEnvironment();

// Connect bitcoind signal handlers
noui_connect();

return (AppInit(argc, argv) ? EXIT_SUCCESS : EXIT_FAILURE);
return (AppInit(node, argc, argv) ? EXIT_SUCCESS : EXIT_FAILURE);
}
45 changes: 45 additions & 0 deletions src/init/bitcoin-node.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
// Copyright (c) 2021 The Bitcoin Core developers
// Distributed under the MIT software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.

#include <interfaces/echo.h>
#include <interfaces/init.h>
#include <interfaces/ipc.h>
#include <node/context.h>

#include <memory>

namespace init {
namespace {
const char* EXE_NAME = "bitcoin-node";

class BitcoinNodeInit : public interfaces::Init
{
public:
BitcoinNodeInit(NodeContext& node, const char* arg0)
: m_node(node),
m_ipc(interfaces::MakeIpc(EXE_NAME, arg0, *this))
{
m_node.init = this;
}
std::unique_ptr<interfaces::Echo> makeEcho() override { return interfaces::MakeEcho(); }
interfaces::Ipc* ipc() override { return m_ipc.get(); }
NodeContext& m_node;
std::unique_ptr<interfaces::Ipc> m_ipc;
};
} // namespace
} // namespace init

namespace interfaces {
std::unique_ptr<Init> MakeNodeInit(NodeContext& node, int argc, char* argv[], int& exit_status)
{
auto init = std::make_unique<init::BitcoinNodeInit>(node, argc > 0 ? argv[0] : "");
// Check if bitcoin-node is being invoked as an IPC server. If so, then
// bypass normal execution and just respond to requests over the IPC
// channel and return null.
if (init->m_ipc->startSpawnedProcess(argc, argv, exit_status)) {
return nullptr;
}
return init;
}
} // namespace interfaces
29 changes: 29 additions & 0 deletions src/init/bitcoind.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
// Copyright (c) 2021 The Bitcoin Core developers
// Distributed under the MIT software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.

#include <interfaces/init.h>
#include <node/context.h>

#include <memory>

namespace init {
namespace {
class BitcoindInit : public interfaces::Init
{
public:
BitcoindInit(NodeContext& node) : m_node(node)
{
m_node.init = this;
}
NodeContext& m_node;
};
} // namespace
} // namespace init

namespace interfaces {
std::unique_ptr<Init> MakeNodeInit(NodeContext& node, int argc, char* argv[], int& exit_status)
{
return std::make_unique<init::BitcoindInit>(node);
}
} // namespace interfaces
6 changes: 4 additions & 2 deletions src/interfaces/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ The following interfaces are defined here:

* [`Handler`](handler.h) — returned by `handleEvent` methods on interfaces above and used to manage lifetimes of event handlers.

* [`Init`](init.h) — used by multiprocess code to access interfaces above on startup. Added in [#10102](https://github.com/bitcoin/bitcoin/pull/10102).
* [`Init`](init.h) — used by multiprocess code to access interfaces above on startup. Added in [#19160](https://github.com/bitcoin/bitcoin/pull/19160).

The interfaces above define boundaries between major components of bitcoin code (node, wallet, and gui), making it possible for them to run in different processes, and be tested, developed, and understood independently. These interfaces are not currently designed to be stable or to be used externally.
* [`Ipc`](ipc.h) — used by multiprocess code to access `Init` interface across processes. Added in [#19160](https://github.com/bitcoin/bitcoin/pull/19160).

The interfaces above define boundaries between major components of bitcoin code (node, wallet, and gui), making it possible for them to run in [different processes](../../doc/multiprocess.md), and be tested, developed, and understood independently. These interfaces are not currently designed to be stable or to be used externally.
18 changes: 18 additions & 0 deletions src/interfaces/echo.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
// Copyright (c) 2021 The Bitcoin Core developers
// Distributed under the MIT software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.

#include <interfaces/echo.h>

#include <memory>

namespace interfaces {
namespace {
class EchoImpl : public Echo
{
public:
std::string echo(const std::string& echo) override { return echo; }
};
} // namespace
std::unique_ptr<Echo> MakeEcho() { return std::make_unique<EchoImpl>(); }
} // namespace interfaces
26 changes: 26 additions & 0 deletions src/interfaces/echo.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
// Copyright (c) 2021 The Bitcoin Core developers
// Distributed under the MIT software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.

#ifndef BITCOIN_INTERFACES_ECHO_H
#define BITCOIN_INTERFACES_ECHO_H

#include <memory>
#include <string>

namespace interfaces {
//! Simple string echoing interface for testing.
class Echo
{
public:
virtual ~Echo() {}

//! Echo provided string.
virtual std::string echo(const std::string& echo) = 0;
};

//! Return implementation of Echo interface.
std::unique_ptr<Echo> MakeEcho();
} // namespace interfaces

#endif // BITCOIN_INTERFACES_ECHO_H
17 changes: 17 additions & 0 deletions src/interfaces/init.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
// Copyright (c) 2021 The Bitcoin Core developers
// Distributed under the MIT software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.

#include <interfaces/chain.h>
#include <interfaces/echo.h>
#include <interfaces/init.h>
#include <interfaces/node.h>
#include <interfaces/wallet.h>

namespace interfaces {
std::unique_ptr<Node> Init::makeNode() { return {}; }
std::unique_ptr<Chain> Init::makeChain() { return {}; }
std::unique_ptr<WalletClient> Init::makeWalletClient(Chain& chain) { return {}; }
std::unique_ptr<Echo> Init::makeEcho() { return {}; }
Ipc* Init::ipc() { return nullptr; }
} // namespace interfaces
Loading

0 comments on commit ac219dc

Please sign in to comment.