Neo 3 Opmizations update list

[lock9]

NEO3 Development Module - Optmizations

This module covers improvements in the execution, verification, and propagation of data.

Ready to be implemented

Note: Some issues may have none or more than one proposed solution.

• Replace exceptwith with faster function (Replace function exceptwith and unionwith with faster functions  #1174)
• Keep track of sender fee for mempool verification(Keep track of sender fee #1183)
• Auto compression (TCP P2P optimization #710)
• Batch message (New P2P message for bulk inventories #1088, Consensus optimization during transaction request #1077)
• Remove duplicate connection (When one side connect actively, the other side will always keep trying a new connection #1123)
• Send disconnection reason (Send disconnection reason when closing connection with a node #1133 Optimisations For The Network Protocol #366)
• opitmize ProtocolHandlerMailbox ( Remove the case of GetData in ProtocolHandlerMailbox#ShallDrop #1201 ) [to be merged]
• Filter unconnected peers in connected list ( Fix p2p filter unconnected peers #1160 )
• Add GetFullBlocks P2P logic (fix Network protocol suggestion: getfullblocks #522 Add GetFullBlocks P2P logic (enable fixing #522) #1138)
• Improve syncing (3x) ( Fix/Improve syncing (3x): Adding ping mechanism to TaskManager for replacing StartHeight and PendingKnownHashes strategy #899 ) [to be merged]
• Store connected peers and try to connect them when restart (Auto-reconnect seed node when all seed nodes crash #1198) [doing]
• Don't relay the inventory of block, when block height below remote node's LastBlockIndex [to do, waiting for Fix/Improve syncing (3x): Adding ping mechanism to TaskManager for replacing StartHeight and PendingKnownHashes strategy #899]
• Keep track of sender fee for mempool verification(Keep track of sender fee #1183)
• opitmize ProtocolHandlerMailbox ( Remove the case of GetData in ProtocolHandlerMailbox#ShallDrop #1201 )
• Replace exceptwith with faster function (Replace function exceptwith and unionwith with faster functions  #1174)
• Tranasction parallel verifiers (Add multi parallel verifiers to verify witness when new transaction received in Blockchain #1228） [to be merged]

To be discussed

• Create performance metrics (Create performance metrics used to benchmark NEO 3 #920)
• Optimize akka serial execution (Optimization proposal for Akka Receiver message processing #1188)
• Auxiliary data structure for re-verification (Plugin dll load method change. (2x) #810)
• Detached application and verification (Dettached Verification + Application: ensure delay is at most one block #965 Detach state persistence and block persistence #302)
• Memory poool opmitization model (Mempool Reverification Model (MRM): Optimizing our nodes #1140 Investigating mempool behavior with similar transactions #1148)
• Avoid bad peers propagation (Avoid bad peers propagation #1134)
• Multichannel for Oracles (Create multi-channel for built-in oracle nodes and consensus nodes communication #1106)
• Use libp2p (why use akka.net for p2p?it is slow. #611)
• Avoid empty blocks (Feature for consensus mechanism for avoiding empty blocks #921)
• Adjust block time(dynamic block time) (Decrease block time to 5 sec #375)
• DBFT3.0 (dBFT 3.0 proposal: active backups strategy #1126)
• Consensus optmization (Consensus optimization: Novel network routing strategies, a trade-off between payloads and communication #519 Optimized Delegated Byzantine Fault Tolerance (ODBFT) - Part I: commit phase + regeneration strategy + minnor message p2p route optimizing #426)
• Improve taskmanager, sync block without header Network protocol suggestion: getfullblocks #522 [waiting for Fix/Improve syncing (3x): Adding ping mechanism to TaskManager for replacing StartHeight and PendingKnownHashes strategy #899 and Add GetFullBlocks P2P logic (enable fixing #522) #1138]
• Transfer transaction directly without inv message（no obvious effect）
• Add node health status, better to select peers for communication Network Improvements to TaskManager in Neo 3.0 - Prefer lower latency & higher availability nodes #542

This is a work in progress. Please add your comments so we can update the list

[lock9]

Hi @Tommo-L, Do you mean pin the issue or add it here?

[Tommo-L]

p2p module

add to be implemented:

• Improve syncing (3x) ( Fix/Improve syncing (3x): Adding ping mechanism to TaskManager for replacing StartHeight and PendingKnownHashes strategy #899 ) [to be merged]
• Store connected peers and try to connect them when restart (Auto-reconnect seed node when all seed nodes crash #1198) [doing]
• Don't relay the inventory of block, when block height below remote node's LastBlockIndex [to do, waiting for Fix/Improve syncing (3x): Adding ping mechanism to TaskManager for replacing StartHeight and PendingKnownHashes strategy #899]

Add to be discussed

• Improve taskmanager, sync block without header Network protocol suggestion: getfullblocks #522 [waiting for Fix/Improve syncing (3x): Adding ping mechanism to TaskManager for replacing StartHeight and PendingKnownHashes strategy #899 and Add GetFullBlocks P2P logic (enable fixing #522) #1138]
• Add node health status, better to select peers for communication Network Improvements to TaskManager in Neo 3.0 - Prefer lower latency & higher availability nodes #542

[Tommo-L]

increase tps

• opitmize ProtocolHandlerMailbox ( Remove the case of GetData in ProtocolHandlerMailbox#ShallDrop #1201 )
• Keep track of sender fee for mempool verification(Keep track of sender fee #1183)
• Tranasction parallel verifiers (Add multi parallel verifiers to verify witness when new transaction received in Blockchain #1228） [to be merged]
• Send inv messages in batches in RemoteNode.CheckMessageQueue( no obvious effect )
• Change invoking InteropService from the function pointer way to a normal way Change invoking InteropService from the function pointer way to a normal way #1247