-
Notifications
You must be signed in to change notification settings - Fork 159
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Access token expiry causes failed uploads on OCIS test server #10474
Comments
owncloud/ocis#5066 really seems related. After downgrading to 4.0.6 and observing the same behavior, I am now trying setting these values: IDP_ACCESS_TOKEN_EXPIRATION=2592000 Update: Doesn't resolve issue |
I tried to reproduce it on https://ocis.ocis-wopi.latest.owncloud.works/ but instance seems broken. no possible to create project space (500 error) so I tested it using our examples see here https://github.com/owncloud/ocis/blob/master/deployments/examples/ localy works fine to me:
Actual: token_refreshing_while_uploading.mov |
Hi @ScharfViktor thank you very much for taking the time to help troubleshoot this. I followed your steps exactly, except I am using the WOPI deployment. I noticed that setting IDP_ID_TOKEN_EXPIRATION in the .env file did not work, so I set it in docker-compose.yml. When I set the expiration to 30, I can observe the token being refreshed every 30 seconds during file upload. However, after a little while the uploads fail. This happens to me every single time, in every test environment I can try. I've tried different servers, versions, hosts, files, networks, everything. The only common denominator I can possibly think of is that I am using Mac Safari. The other difference I see in your test is that I am not dragging a single file, but a directory full of large files, although I cannot imagine that would matter. Here are the system logs:
|
Hey @prohtex thank you so much for looking into this issue with such dedication and attention to detail! The issue you are describing seems to be related to browsers throttling inactive tabs... I have this as a reproducer:
Could you please try the following?
Throttling in modern browsers exists so that you can keep 500 browser tabs "open" at the same time, not drain your device battery, keep the memory footprint and CPU usage low (haha, laughing in Chromium). However, if you start an upload and want it to finish, this is kind of annoying... More info: https://aboutfrontend.blog/tab-throttling-in-browsers/ There seems to be a way to disable inactive tab throttling in Safari, but that's really only for debugging purposes, not for your everyday browser usage. See https://apple.stackexchange.com/questions/280679/preventing-safari-background-tabs-disconnecting-from-server The only solution I currently see in our code base would be to implement long running actions via Web Workers, i.e. in separate threads. Looks like those are still not throttled by modern browsers... we already thought about that nearly 2 years ago in a different context, see #7039 For the token expiration you could try to add the cc @tbsbdr this needs PM attention. |
@kulmann I've played with this a bit as well. It is also pretty easily reproducible when just throttling the network bandwith (using the developer tools) to e.g. DSL or 4G/LTE (Wifi might also work). In that case the token refresh request seems to just take too long when an upload is happening in parallel and the token expires. |
... transferring to web |
We're requesting a new access token 10 seconds before the current one expires. Maybe we could do that earlier, that's an easy fix. Maybe give it 30 seconds if the access token lifetime is longer than e.g. 1 minute (1 minute is already a magic number in the oidc-client-ts lib that we use). |
All I can say is that uploads must continue in "background" tabs; assessing as p2 as uploading is essential for us. as mentioned by you, @kulmann solving this issue via web worker might be heavy lifting, so a effort estimation / reasearch beforehand should happen before we go in. |
Hi @kulmann, I am glad the issue was found! I can confirm that taking further actions keeps the tab active and the upload going. This also explains why the point at which the upload failed was rather inconsistent (as I was taking various actions with the browser, and not always keeping it in the background). I can also confirm that setting IDP_ID_TOKEN_EXPIRATION mitigates the issue; I had mistakenly tried to set the value in the .env file previously and it was not taking effect. Our use case is for photo retouching, and so we are frequently sending massive files over the web client. A big part of why we wanted to migrate to OCIS was to improve reliability sending large files this way; we often encounter timeouts and broken uploads with OC10. I am assuming sites like DropBox and Wetransfer use Web Workers for their uploads. For the meantime, we will likely deploy more than one instance of OCIS so we can continue to test, and to keep one with a very long token timeout for these purposes. It doesn't appear possible to get the WOPI deployment working without Docker so we will continue to use the Ubuntu VM. I can always test anything in a fresh Ubuntu VM or on macOS. Thanks again to the team of dedicated individuals working to make OCIS an incredible product. I'm a PHP developer so can't contribute much in the way of code, but am happy to do whatever I can to help. |
After further testing, I am noticing some quirks: #10492 |
I am receiving the same error, but then when trying to upload using the android client, rclone and web. I have also set the access tokens to expire after 24 hours in my external idp. using authentik. Errors I'm getting are: Rclone:
|
It sounds like a different error then. We have a similar issue open in the server repo, which makes more sense if you're experiencing these problems with other clients as well. Maybe related? owncloud/ocis#8372 Either way, there are definitely issues with tab throttling. #10955 addresses those. |
I've been scratching my head over token expiry errors that cause failed uploads on OCIS documented on owncloud/ocis#8368 and owncloud/ocis#8372. Today I reproduced these errors on OCIS continuous deployment test server.
I obviously don't have access to those logs, but screenshots are below.
Steps to reproduce:
The text was updated successfully, but these errors were encountered: