Skip to content

Retry downloading a package#1198

Merged
dlang-bot merged 5 commits intodlang:masterfrom
grogancolin:grogancolin/#1167
Aug 26, 2017
Merged

Retry downloading a package#1198
dlang-bot merged 5 commits intodlang:masterfrom
grogancolin:grogancolin/#1167

Conversation

@grogancolin
Copy link
Contributor

Try 3 times to download a package/metadata

Packagesupplier.d now tries 3 times to download a package and metadata.
This should aid travis to not be as flaky as it is currently.

Implements enhancement request #1167

@wilzbach
Copy link
Contributor

See also: #1190 though not conflicting with this PR.

foreach(i; 0..3) {
try{
download(url, path);
break;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

break; <-> return;

logDebug("Failed to download package %s from %s (Attempt %s of 3)", packageId, url, i + 1);
continue;
}
throw new Exception("Failed to download package %s from %s".format(packageId, url));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be moved outside of the loop.

break;
}
catch(HTTPStatusException e) {
logDebug("Failed to download package %s from %s (Attempt %s of 3)", packageId, url, i + 1);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be a check for 404 errors here, or maybe even limit the retry behavior to 500 errors and connection failures. Retrying for 404 errors would mean that package download in scenarios with multiple registries (that contain different sets of packages) potentially gets slowed down considerably.

Packagesupplier.d now tries 3 times to download a package and metadata.
This should aid travis to not be as flaky as it is currently.

Implements enhancement request dlang#1167
Copy link
Member

@s-ludwig s-ludwig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably refactor this, so that the retry-logic is contained in the download function, but that doesn't have to be done as part of the PR.

catch(HTTPStatusException e) {
if (e.status == 404) {
logDebug("Failed to download package %s from %s (404)", packageId, url);
return;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how the outer code will behave if nothing downloaded, but to be consistent with the previous behavior, this would also have to throw instead of returning.

@dlang-bot
Copy link
Collaborator

Thanks for your pull request, @grogancolin! We are looking forward to reviewing it, and you should be hearing from a maintainer soon.

Some tips to help speed things up:

  • smaller, focused PRs are easier to review than big ones

  • try not to mix up refactoring or style changes with bug fixes or feature enhancements

  • provide helpful commit messages explaining the rationale behind each change

Bear in mind that large or tricky changes may require multiple rounds of review and revision.

Please see CONTRIBUTING.md for more information.

Copy link
Member

@s-ludwig s-ludwig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added the throw for the 404 case.

@grogancolin
Copy link
Contributor Author

Thanks!
Apologies - I never got time to get back to this quickly :-)

@PetarKirov
Copy link
Member

PetarKirov commented Jul 31, 2017

@s-ludwig maybe it's time to drop support for accient frontends versions of dmd like 2.068 which currently prevents the merging of this PR?

@wilzbach
Copy link
Contributor

Looks more like a random failure - let's have a look whether restarting works

@s-ludwig
Copy link
Member

AFAIR, 2.068 is the frontend that the latest released version of GDC uses, so I think we should keep it for a while. But it's indeed strange that it fails for this PR, I already restarted it, but it still failed.

@dlang-bot dlang-bot merged commit db075a8 into dlang:master Aug 26, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants