Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Transfer cancellation and notifications #432

Merged
merged 84 commits into from
Jun 4, 2024
Merged

Transfer cancellation and notifications #432

merged 84 commits into from
Jun 4, 2024

Conversation

tmpfs
Copy link
Collaborator

@tmpfs tmpfs commented May 27, 2024 •
edited
Loading

  • Fix file_transfers_offline_* test specs
  • Fix file_transfers_late_upload test spec
  • Cancel all inflight transfers on shutdown
  • Move failed operations to a failures queue
  • Periodically attempt to restore from failures queue to the transfers queue
  • Deleting secret file(s) must cancel any inflight upload/download
  • Moving secret file(s) must cancel any inflight upload/download

Closes #434 and closes #433.

tmpfs added 14 commits May 26, 2024 16:52
@tmpfs
Prepare cancel channel for upload/download.
3b909d3
@tmpfs
Improve inflight request handling.
5207e1f
@tmpfs
Tidy obsolete code.
c31dad9
@tmpfs
Tweak tracing instrumentation.
9143de3
@tmpfs
The file transfer request_id should start at 1.
03df16a
@tmpfs
Support id for system messages.
24e8782
@tmpfs
System message content is optional.
5c2029a
@tmpfs
Add variant to SysMessageLevel.
ca1b979
@tmpfs
Prepare notifications channel for inflight transfers.
fe2f28b
@tmpfs
Use notifications channel for transfers.
53b8c62 
Note that this disables the file progress command from the CLI tools for
now, need to restore this command later.
@tmpfs
Improve concurrent_transfers handling.
1a6f54b 
When we have multiple clients then a concurrent_transfers settings is
multiplied by the number of clients for uploads so we divide by the
number of clients to get as close as possible to the target
concurrent_transfers.
@tmpfs
Fatal error on connection refused.
72fdef1
@tmpfs
Test for connection refused error.
06cb797
@tmpfs
Restore device_id to AccountData.
e3bccdb
@tmpfs tmpfs self-assigned this May 27, 2024
tmpfs added 15 commits May 27, 2024 17:09
@tmpfs
Disable failing file transfer test specs for now.
3cf9808
@tmpfs
Switch to mpsc channel and handle backpressure.
633870b
@tmpfs
Fix file transfer test spec.
66e2a50 
Fatal error on connection refused breaks the retry logic that we already
have test specs for so we need request retry logic with exponential
backoff.
@tmpfs
Add retry logic for upload operation.
a391e50
@tmpfs
Use NetworkRetry in websocket module.
ed92d6f 
Also handle backpressure on websocket notification channel.
@tmpfs
File download handles retries.
b54c4c3
@tmpfs
Reduce concurrent_transfers.
4847ded
@tmpfs
Fix bug with progress notifications.
13a1889 
Need to await the call to send() on the mpsc::Sender.
@tmpfs
Add transfer error variant.
cccf119
@tmpfs
Check against receiver_count().
22540ff
@tmpfs
Restore public notifications() method.
c3483d4
@tmpfs
Use semaphore for file transfer concurrency.
eefe7a4
@tmpfs
Refactor file transfer modules.
8b04145
@tmpfs
Improve file transfer notifications.
85a8317
@tmpfs
Tidy file transfer modules.
8121320
tmpfs added 29 commits May 30, 2024 13:32
@tmpfs
Improve logic for proxying remote bridge file transfer events.
17a1c7a
@tmpfs
Tidy websocket shutdown test logic.
b918ba6
@tmpfs
Pass id to wait_and_retry.
211a4fb
@tmpfs
Do not queue events when no clients.
f21beac
@tmpfs
Use polled failures queue for file transfers.
89e0b9e
@tmpfs
Change constructor for file transfer settings.
9326b48
@tmpfs
Adjust failure interval and expiry.
6dbff3a
@tmpfs
Improve inflight notification handling.
402eacf
@tmpfs
Include transfer_id along with request_id.
68cc558 
A transfer_id is consistent for a given file, operation and origin so
that callers can see if an inflight transfer corresponds to an existing
inflight transfer.

For example, this can be used by a UI to remove a previously failed
transfer notification when the same transfer is re-queued.
@tmpfs
Fix retries in transfer operations.
23fb018
@tmpfs
Tidy borrows for transfer operations.
ede5ff5
@tmpfs
Do not notify when no items on re-insertion.
dae0b52
@tmpfs
Rename to cancel_all(), add cancel_one().
cb4bcb9
@tmpfs
Support user_canceled flag for transfer cancellation.
bd0dbe3
@tmpfs
Rename variant, expose TransferError.
9b28acd
@tmpfs
Remove unnecessary Mutex.
2d29905
@tmpfs
Switch to watch channel for retry cancellation.
790f7bf
@tmpfs
Improve handling for the retry cancel channel.
6c41272
@tmpfs
Do not use read_timeout() for uploads.
1dd3ed2 
Otherwise uploading very large files will never complete.
@tmpfs
Ensure download sends content length.
ff5cef4
@tmpfs
Tidy imports.
fbb25da
@tmpfs
Fix queue processing order.
ec1950e
@tmpfs
Use CancelReason for more information about cancellations.
f1bd646
@tmpfs
Fix bug with canceling during normalization.
4db1db4
@tmpfs
Refactor websocket disconnect and broadcast.
aa69226
@tmpfs
Tweak comment.
bd4ba8e
@tmpfs
Add test specs for abort transfer on move/delete.
9968365
@tmpfs
Improve ReceiveGuard for file upload handler.
774a57d
@tmpfs
Bump minor version.
385b7d9
@tmpfs tmpfs merged commit 2b1d765 into main Jun 4, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@tmpfs tmpfs

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Modify file transfer clients as servers are added and removed Cancel websocket connection retry on sign out
1 participant
@tmpfs