-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(wallet): Add a vertex history streamer to the wallet websocket API #1082
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
msbrogli
force-pushed
the
feat/ws-history-streamer
branch
3 times, most recently
from
July 11, 2024 20:53
b9937c8
to
6cd3fea
Compare
msbrogli
commented
Jul 11, 2024
msbrogli
force-pushed
the
feat/ws-history-streamer
branch
from
July 12, 2024 05:58
6cd3fea
to
4f71c51
Compare
msbrogli
commented
Jul 12, 2024
msbrogli
force-pushed
the
feat/ws-history-streamer
branch
7 times, most recently
from
July 12, 2024 07:21
a8a77d0
to
b4f9209
Compare
msbrogli
commented
Jul 12, 2024
msbrogli
force-pushed
the
feat/ws-history-streamer
branch
from
July 12, 2024 07:54
b4f9209
to
8ed7341
Compare
msbrogli
commented
Jul 12, 2024
msbrogli
force-pushed
the
feat/ws-history-streamer
branch
from
July 12, 2024 08:44
8ed7341
to
ce31121
Compare
msbrogli
commented
Jul 12, 2024
msbrogli
force-pushed
the
feat/ws-history-streamer
branch
5 times, most recently
from
July 12, 2024 16:26
38be174
to
2cb449e
Compare
r4mmer
reviewed
Jul 12, 2024
msbrogli
force-pushed
the
feat/ws-history-streamer
branch
4 times, most recently
from
July 16, 2024 17:16
887c9f4
to
82c9605
Compare
msbrogli
force-pushed
the
feat/ws-history-streamer
branch
3 times, most recently
from
July 18, 2024 19:42
238ec39
to
6a09d06
Compare
r4mmer
reviewed
Jul 19, 2024
msbrogli
force-pushed
the
feat/ws-history-streamer
branch
8 times, most recently
from
July 19, 2024 18:12
f73ae2a
to
5383196
Compare
jansegre
previously approved these changes
Jul 20, 2024
r4mmer
previously approved these changes
Jul 22, 2024
msbrogli
force-pushed
the
feat/ws-history-streamer
branch
from
July 22, 2024 20:01
5383196
to
f42d005
Compare
msbrogli
force-pushed
the
feat/ws-history-streamer
branch
5 times, most recently
from
July 23, 2024 21:45
52ae6e5
to
be56084
Compare
msbrogli
force-pushed
the
feat/ws-history-streamer
branch
from
July 23, 2024 22:52
2f7ee1e
to
631f2ca
Compare
glevco
approved these changes
Jul 24, 2024
r4mmer
approved these changes
Jul 24, 2024
This was referenced Jul 25, 2024
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Wallets might need to download their transaction history. The full node currently offers a polling API but this is too slow to both large wallets and high latency environments. This PR implements a streaming API that speeds up the download for wallets.
Streaming protocol
Client sends either
request:history:xpub
orrequest:history:manual
.Server open the streaming and follows the documented protocol (see the streamer docustring).
Performance
Testing with a wallet that has 1,060 addresses and 1,187 transactions with 480ms of latency between the wallet and the full node.
Download time
Total time (including a post-processing that occurs after the download):
Notice that the post-processing time is the same for all APIs (~10 seconds).
Acceptance Criteria
HathorAdminWebsocketFactory.handle_message()
, moving it toHathorAdminWebsocketProtocol
.HathorAdminWebsocketFactory._handle_ping()
, moving it toHathorAdminWebsocketProtocol
.HathorAdminWebsocketFactory.subscribe_address()
, which subscribes the connection to receive real time updates for all vertices related to an address.request:history:xpub
, which opens a streaming generating addresses from the xpub and stopping when the GAP limit is reached.request:history:manual
, which opens a streaming from a list of addresses and stopping when the GAP limit is reached or the list of addresses ran out.request:history:stop
, which stops the current streamer.request:history:manual
to 5,000 addresses per connection.--disable-ws-history-streaming
argument to therun_node
cli command.disable_history_streaming()
to the factory that disables history streaming in all connections too.disable_history_streaming()
to the protocol that stops the streaming if one is opened.Checklist
master
, confirm this code is production-ready and can be included in future releases as soon as it gets merged