Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set Upload-Length when appropriate #744

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Set Upload-Length when appropriate #744

wants to merge 5 commits into from

Conversation

Vija02
Copy link

@Vija02 Vija02 commented Jan 13, 2025

This PR fixes issue #182 where user cannot upload against a deferred resource.

The 2 main changes are:

  1. Bypass the Upload-Length header check if Upload-Defer-Length exists in the HEAD response
  2. Sets the Upload-Length header in the next request if options.uploadLengthDeferred is false - because we would know the length straight away

The code is mostly taken from the relevant issue, so credits goes there. But I've also modified it slightly to simplify the logic.

Closes #182

…the server. And allow resumeUpload if Upload-Defer-Length is set from server.
Copy link
Member

@Acconut Acconut left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this PR! Overall, taking the information about deferred length from the HEAD response is a good approach to solving this problem. Looking at the implementation, there are a few minor things to improve. Tests are also still missing, which we need to add before this can be merged. Let me know if you need help with this.

lib/upload.js Outdated Show resolved Hide resolved
lib/upload.js Outdated Show resolved Hide resolved
lib/upload.js Outdated Show resolved Hide resolved
@Vija02
Copy link
Author

Vija02 commented Jan 13, 2025

Thanks for the review, I've updated them based on your comments. I'll add some tests soon

Copy link
Member

@Acconut Acconut left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more comment so far. I am looking forward to the tests. Let me know if you need any more help!

There are some merge conflicts with main now, but I can take care of them once this PR is ready to get merged.

@@ -821,9 +828,10 @@ class BaseUpload {
// If the upload length is deferred, the upload size was not specified during
// upload creation. So, if the file reader is done reading, we know the total
// upload size and can tell the tus server.
if (this.options.uploadLengthDeferred && done) {
if (this._uploadLengthDeferred && (!this.options.uploadLengthDeferred || done)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (this._uploadLengthDeferred && (!this.options.uploadLengthDeferred || done)) {
if (this._uploadLengthDeferred || done) {

I think we can just use this._uploadLengthDeferred here and don't need to consult the original option anymore.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That will change the behaviour quite a bit. As it is suggested right now, there's some problems:

  1. We don't always know the size when this._uploadLengthDeferred is true since options.uploadLengthDeferred could be true. In this case, we'd send wrong info.
  2. done will make it always pass the header at the end, which I believe isn't how it should work. It should only send the header on the next known request from what I understand.

If we change it to an AND, it'll kinda work, but only send the length header at the end.

The logic in the commit right now expands to:

  • (this._uploadLengthDeferred && !this.options.uploadLengthDeferred)
    Send header on next opportunity if the request is of known size
  • (this._uploadLengthDeferred && done)
    And send at the end if it's a stream

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, my suggestion is wrong. I used || instead of && on accident and my suggestion would be:

Suggested change
if (this._uploadLengthDeferred && (!this.options.uploadLengthDeferred || done)) {
if (this._uploadLengthDeferred && done) {

If we change it to an AND, it'll kinda work, but only send the length header at the end.

Yes, that's right and that's how tus-js-client has handled deferred lengths in the past. It always sent the length header once the file source is closed because then we know the file size for sure. I don't see a problem with keeping this behavior. Or do you?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly, I'm quite new to the protocol and library so I think you'd know more. But I was looking at this line over at the protocol documentation:
image

That led me to believe that it should be the first possible instance. Although, I guess both ways should work anyway?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @Acconut, just want to bump this PR, any thoughts on this? Thanks 😄

@Vija02
Copy link
Author

Vija02 commented Jan 14, 2025

@Acconut I've added 2 tests. And also updated a fix for calculating the total size on that code block. Before the fix, it'd calculate the size of the current request and send that rather than the total size

@Vija02 Vija02 requested a review from Acconut January 14, 2025 23:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Upload against a deferred resource does not work without options.uploadLengthDeferred
2 participants