fix(ws): Add a graceful close mechanism to handle late messages and prevent errors

[msbrogli]

Motivation

When the stream ended, an ACK message arrived late, causing an error response to be generated.

Check out the state diagram in the Reference-level explanation here: https://github.com/HathorNetwork/internal-rfcs/pull/25

Acceptance Criteria

1. Do nothing if the ACK received from the client is duplicated.
2. Stop the streamer if the ACK received from the client decreases.
3. Stop the streamer if the ACK received from the client is higher than the last sent message sequence number.
4. Add some protection to the pauseProducing, stopProducing, and resumeProducing methods.
5. Replace flags _started, _paused, and _stop by a state machine.
6. Introduce the new state CLOSING which means the streaming has ended and it is just waiting for the last ACK.
7. Modify streamer to expect ack for the stream end message.

Checklist

• If you are requesting a merge into master, confirm this code is production-ready and can be included in future releases as soon as it gets merged

[github-actions]

Bencher

Report	Fri, September 6, 2024 at 17:02:35 UTC
Project	hathor-core
Branch	fix/tx-history-streaming-graceful-close
Testbed	ubuntu-22.04

Click to view all benchmark results

Benchmark	Latency	Latency Results nanoseconds (ns) | (Δ%)	Latency Lower Boundary nanoseconds (ns) | (%)	Latency Upper Boundary nanoseconds (ns) | (%)
sync-v2 (up to 20000 blocks)	✅ (view plot)	101,447,530,631.24 (-1.92%)	93,094,571,891.75 (91.77%)	113,782,254,534.36 (89.16%)

Bencher - Continuous Benchmarking
View Public Perf Page
Docs | Repo | Chat | Help

[codecov]

Codecov Report

Attention: Patch coverage is 56.62651% with 36 lines in your changes missing coverage. Please review.

Project coverage is 84.42%. Comparing base (ec2563e) to head (22fc4b9).
Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
hathor/websocket/streamer.py	56.25%	24 Missing and 11 partials ⚠️
hathor/websocket/protocol.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1129      +/-   ##
==========================================
+ Coverage   84.39%   84.42%   +0.03%     
==========================================
  Files         317      317              
  Lines       24240    24291      +51     
  Branches     3684     3695      +11     
==========================================
+ Hits        20457    20508      +51     
+ Misses       3071     3068       -3     
- Partials      712      715       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.