Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

xftp: add sending and receiving via URI-encoded redirects #968

Merged
merged 38 commits into from
Feb 13, 2024

Conversation

dpwiz
Copy link
Contributor

@dpwiz dpwiz commented Jan 16, 2024

No description provided.

@dpwiz dpwiz changed the title xftp: add URI encoding for FileDescription xftp: add sending and receiving via URI-encoded redirects Jan 29, 2024
@dpwiz dpwiz marked this pull request as ready for review January 29, 2024 14:27
ALTER TABLE snd_files ADD COLUMN redirect_size INTEGER;
ALTER TABLE snd_files ADD COLUMN redirect_digest BLOB;

ALTER TABLE rcv_files ADD COLUMN redirect_id INTEGER REFERENCES rcv_files ON DELETE CASCADE;
Copy link
Collaborator

@spaced4ndy spaced4ndy Feb 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are temporary and unencrypted redirect descriptions files deleted?
There are two cases:

  • on cleanup - they may be deleted if are read from db first
  • on delete api call

It may be better to explicitly check in both cases if file has a redirect file and delete in this case

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also redirect file can be deleted after destination rcv file has been updated

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added received yaml deletion right after reading it.

Redirect records should be picked up by agent's clean up manager just like the ordinary files.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Redirect records will be transitively deleted in db due to ON DELETE CASCADE and will not be picked up by cleanup manager for file deletion.

For delete api record of the file itself is always deleted first and redirect record deleted transitively, so the problem is even more acute than with cleanup manager where order may differ.

@epoberezkin epoberezkin merged commit 2f7a288 into master Feb 13, 2024
3 checks passed
@epoberezkin epoberezkin deleted the ab/file-link branch February 13, 2024 14:08
epoberezkin added a commit that referenced this pull request Feb 21, 2024
* xftp: add URI encoding for FileDescription

* tweak URI

* allow smaller blocks

* draft xftpReceiveFileFollow' and xftpSendFilePublic'

* add sending with redirect

* allow 64k chunks

* add migrations with redirect fields

* add test case

* fix deadlock

* revert CLI code

* WIP: working send/receive via URI

* fix field ambiguity

* cleanup

* update agent db schema

* update minimal chunk size

* add rfc

* apply suggestions from code review

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>

* add createRcvFileRedirect

* extract Simplex.Messaging.ServiceScheme and reuse for files

* update db schema

* check size/digest on receive complete

* cleanup

* use SIZE/DIGEST errors for redirects too

* split digest/size errors from redirect checks

* fix redirect error encoding

* rename RedirectMeta to RedirectFileInfo

* use query encoding for file URI

* group maybe fields under RcvFileRedirect

* add extras field

* update rfc

* add extras encoding and no-redirect tests

* fix toStrict for old ghc

* extra client data in file descr URI

* remove decoded yaml file

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants