Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for downloading with aria2 if it's already installed #111

Merged
merged 2 commits into from
Oct 21, 2020

Conversation

interstateone
Copy link
Contributor

@interstateone interstateone commented Oct 16, 2020

By default, xcodes will use a URLSession to download the specified version. If aria2 (available in Homebrew with brew install aria2) is installed, either at /usr/local/bin/aria2c or at the path specified by the --aria2 flag, then it will be used instead. aria2 will use up to 16 connections to download Xcode 3-5x faster (based on a couple tests on my residential internet). If you have aria2 installed and would prefer to not use it, you can use the --no-aria2 flag.

aria2 downloads should retry up to 3 times if it fails in the middle, just like URLSession. Errors that cause aria2c to exit with a non-zero status code will show an appropriate error message.

In order to support both aria2 and URLSession, I had to change the download URLs. The previous URL prefix still works, but I was having issues with it when using aria2 for downloads. This new URL prefix matches whats shown in a browser at developer.apple.com/download/more, and works with both URLSession and aria2.

Closes #93

Testing

Download a bunch of xcodes while you're on this branch 😄

First, clone this repo if necessary and check out the PR branch:

git clone https://github.com/RobotsAndPencils/xcodes.git
cd xcodes
git checkout -b interstateone-aria2 master
git pull https://github.com/interstateone/xcodes.git aria2

You can test this new behaviour with your own account using a command like:

swift run xcodes install 12.1 gm seed # or some other version that you don't have already

You should try installing both with and without aria2, so:

  • without aria2c
  • with aria2c
  • with aria2c and --no-aria2 flag to use URLSession

You can see if aria2c is being used during install by opening Activity Monitor and filtering for aria2c in the top right.

@thii
Copy link
Contributor

thii commented Oct 16, 2020

Nice. It'd be nice to make aria2 a dependency in the Homebrew formula so that it'd be using it by default.

@interstateone
Copy link
Contributor Author

@thii That’s a good idea, I hadn’t thought of that! I’m leaning towards making it optional in the formula, at least to begin with, so that this is an incremental change.

(Part of my hesitation is that installing aria2 took hours despite providing a bottle, because it had to build a bunch of its dependencies. I think this might have been because of the versions of some formulas that I already had installed, and so not everyone will have this experience, but I’d like to see how it works for more people first.)

@interstateone
Copy link
Contributor Author

interstateone commented Oct 16, 2020

If we do want to make aria2 the default download option in the future I'd like to look into statically linking libaria2 instead of relying on the separate executable. I don't know if this would actually work yet, though 😄

Edit: Oh, right, except it's GPLv2, so xcodes would need to be the same if we linked the library.

@thii
Copy link
Contributor

thii commented Oct 19, 2020

I just noticed the installation slowness too. Let's keep it optional for now.

The previous URL still works, but I was having issues with it when using aria2 for downloads. This new URL matches whats shown in a browser at developer.apple.com/download/more, and works with both URLSession and aria2.
@interstateone interstateone marked this pull request as ready for review October 21, 2020 00:35
Copy link

@civatrix civatrix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few questions about the Aria2 code but otherwise looks good

Sources/XcodesKit/Environment.swift Outdated Show resolved Hide resolved
Sources/XcodesKit/Environment.swift Outdated Show resolved Hide resolved
@interstateone interstateone merged commit e8a846b into XcodesOrg:master Oct 21, 2020
@interstateone interstateone deleted the aria2 branch October 21, 2020 02:46
@interstateone
Copy link
Contributor Author

I tried adding depends_on to the formula, but it seems to only work when building from source, not when installing the bottle. I think it's okay if users have to brew install robotsandpencils/made/xcodes aria2 instead though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Investigate performing downloads with aria2
3 participants