feat(ws): Add flow control to the tx history streamer

[msbrogli]

Motivation

The current implementation of the tx history streaming does not have a flow control mechanism. After running some stress tests, we learned that wallets might not be able to process transactions at the same rate the full node is sending them. So, this PR introduces a flow control mechanism using a sequence number and an ACK number.

Here's an incomplete design of the flow control mechanism: https://github.com/HathorNetwork/internal-rfcs/pull/25

Acceptance Criteria

1. Add seq: int to each stream message (except begin, end and error).
2. The flow control is disabled by default.
3. Add StreamBeginMessage.sliding_window_size: Optional[int] where none means "no flow control".
4. Add an optional window-size attribute to the create stream websocket commands xpub and manual.
5. If window-size is missing, the default value is used.
6. If window-size is a number, the provided value is used.
7. If window-size is the number is negative, the flow control is disabled.
8. Add a command request:history:ack that sets the ack number and optionally modifies the window size.
9. If flow control is enabled, the streamer will send up to _sliding_window_size messages without receiving an ack. Then it will wait for an ack (or a change in the window size) to resume.
10. Fix a bug in which the streamer was skipping one message every time the streamer was paused.

Checklist

• If you are requesting a merge into master, confirm this code is production-ready and can be included in future releases as soon as it gets merged

[github-actions]

Bencher

Report	Tue, September 3, 2024 at 20:46:43 UTC
Project	hathor-core
Branch	feat/tx-history-streaming-flow-control
Testbed	ubuntu-22.04

Click to view all benchmark results

Benchmark	Latency	Latency Results nanoseconds (ns) | (Δ%)	Latency Lower Boundary nanoseconds (ns) | (%)	Latency Upper Boundary nanoseconds (ns) | (%)
sync-v2 (up to 20000 blocks)	✅ (view plot)	101,390,691,250.28 (-0.47%)	91,685,880,126.34 (90.43%)	112,060,520,154.42 (90.48%)

Bencher - Continuous Benchmarking
View Public Perf Page
Docs | Repo | Chat | Help

[codecov]

Codecov Report

Attention: Patch coverage is 34.44444% with 59 lines in your changes missing coverage. Please review.

Project coverage is 84.50%. Comparing base (4f80786) to head (f3e3ab5).
Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
hathor/websocket/protocol.py	5.55%	34 Missing ⚠️
hathor/websocket/streamer.py	50.00%	22 Missing and 3 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1126      +/-   ##
==========================================
- Coverage   84.61%   84.50%   -0.11%     
==========================================
  Files         317      317              
  Lines       24150    24231      +81     
  Branches     3662     3680      +18     
==========================================
+ Hits        20434    20477      +43     
- Misses       3009     3050      +41     
+ Partials      707      704       -3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[jansegre]

Looks good to me.