Threading

[ashamis]

• When using Raft we now assign a TLS connection to a thread and have that thread run the entire tx execution pipeline.
• Fix several other miscellaneous concurrency issues.

This is part of #774

[ghost]

threading_aaaa_5@4589 aka 20200206.6 vs master ewma over 30 builds from 4233 to 4581

[codecov-io]

Codecov Report

❗ No coverage uploaded for pull request base (master@a8a97a0). Click here to learn what that means.
The diff coverage is 100%.

@@            Coverage Diff            @@
##             master     #807   +/-   ##
=========================================
  Coverage          ?   77.61%           
=========================================
  Files             ?      150           
  Lines             ?    11503           
  Branches          ?        0           
=========================================
  Hits              ?     8928           
  Misses            ?     2575           
  Partials          ?        0

Flag	Coverage Δ
#e2e_BFT	52.19% <100%> (?)
#e2e_CFT	70.02% <100%> (?)
#unit_BFT	53.79% <100%> (?)
#unit_CFT	68.77% <100%> (?)

Impacted Files	Coverage Δ
src/node/rpc/memberfrontend.h	79.86% <ø> (ø)
src/consensus/raft/raft.h	82.21% <ø> (ø)
src/kv/kv.h	85.98% <ø> (ø)
src/ds/messaging.h	100% <ø> (ø)
src/node/nodestate.h	82.1% <ø> (ø)
src/consensus/pbft/libbyz/Client_proxy.h	78.01% <ø> (ø)
src/host/nodeconnections.h	93.15% <ø> (ø)
src/apps/luageneric/luageneric.cpp	74.75% <ø> (ø)
src/tls/pem.h	80% <ø> (ø)
src/node/rpc/frontend.h	76.92% <ø> (ø)
... and 14 more

[jumaffre]

AFAICS, there's still an issue in the encryptor where a key could be deleted (rollback or compact) while a transaction is being encrypted. As discussed, we should make sure we do not return a reference to a key in get_encryption_key() but a shared pointer? We can definitely sort this out in a separate PR though!