Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Linux network fixes #3859

Merged
merged 1 commit into from
Jul 21, 2023
Merged

Conversation

HebaruSan
Copy link
Member

Problems

  • Users on Linux have reported that downloads and ZIP validation both sometimes "freeze", and eventually the whole install process comes to a halt. I have seen this when trying to install RP-1; usually a few smaller mods get stuck:
    ckan_mono_frozen_downloads
    image
  • If a downloaded ZIP fails to validate (such as for a file downloaded from taniwha.org), CKAN attempts to continue the installation, which wastes time because it will ultimately fail and be rolled back (reported by @UltraJohn on Discord)

Causes

  • Individual downloads and validations were not actually freezing; rather, they completed normally, and the files were moved to the main cache folder, but the progress bars reappeared after being completed, due to the final 100% completion progress update being reordered to occur before some of the earlier in-progress progress updates.
  • The overall install was halting because once in a while, instead of starting the next queued download, CKAN was re-queueing it, because shouldQueue was true when it shouldn't have been, because bytesLeft wasn't 0 when it should have been in another completed download. This value gets set whenever a progress update is raised, so either the last one was missing, or it had wrong numbers.
  • Exceptions thrown by ZIP validation were being caught and ignored after deleting the invalid file, so the installer thought everything was fine and kept going.
  • The mutex guarding completed download processing was excessively broad and included ZIP validation, which can take on the order of a minute for large files, which is too long to hold a lock. This may have been causing issues since if one file completed while another was validating, the newly completed one would have to wait.

Changes

  • Now we have one more progress notification for completed downloads and validations, as well as explicitly setting bytesLeft to 0 in the completed download handler. This should prevent any problems with weird byte tracking.
  • Now ZIP validation errors will be treated the same as network errors—saved and provided to calling code—including the ability to abort or retry, and the install won't proceed if they occur
  • Now the mutex that guards processing of completed downloads is more granular (parts that don't access shared state are moved out of the lock block), to allow multiple concurrent ZIP validations to proceed without blocking one another.

After these changes, I'm able to download and install RP-1 in Linux, where before it would reliably get stuck.

Report progress better
Validate downloads outside mutex
Abort install if validation fails
Set bytes remaining to 0 on completion
@HebaruSan HebaruSan added Bug Something is not working as intended Core (ckan.dll) Issues affecting the core part of CKAN Linux Issues specific for Linux Mono Issues specific for Mono Network Issues affecting internet connections of CKAN labels Jul 20, 2023
@HebaruSan HebaruSan requested a review from techman83 July 20, 2023 21:57
Copy link
Member

@techman83 techman83 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Always enjoy you're clear explanations of what is going on! Nothing in the implementation stands out, LGTM!

@HebaruSan HebaruSan merged commit d1dd847 into KSP-CKAN:master Jul 21, 2023
@HebaruSan HebaruSan deleted the fix/network-quirks branch July 21, 2023 02:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something is not working as intended Core (ckan.dll) Issues affecting the core part of CKAN Linux Issues specific for Linux Mono Issues specific for Mono Network Issues affecting internet connections of CKAN
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants