Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync: Try to fix the GitHub abuse mechanism coping code #978

Merged
merged 4 commits into from
Mar 30, 2021
Merged

Sync: Try to fix the GitHub abuse mechanism coping code #978

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
37ba793
Sync: Try to fix the GitHub abuse mechanism coping code
ewels Mar 26, 2021
0971ef9
Changelog
ewels Mar 26, 2021
df9061f
Merge branch 'dev' into fix-970
ewels Mar 29, 2021
a21fa21
Merge branch 'dev' into fix-970
ewels Mar 30, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
### Tools

* Strip values from `nf-core launch` web response which are False and have no default in the schema [[#976](https://github.com/nf-core/tools/issues/976)]
* Try to fix the fix for the automated sync when we submit too many PRs at once [[#970](https://github.com/nf-core/tools/issues/970)]

### Template

Expand Down
14 changes: 10 additions & 4 deletions nf_core/sync.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
import json
import logging
import os
import random
import re
import requests
import requests_cache
Expand Down Expand Up @@ -331,29 +332,34 @@ def make_pull_request(self):
try:
self.gh_pr_returned_data = json.loads(r.content)
returned_data_prettyprint = json.dumps(self.gh_pr_returned_data, indent=4)
r_headers_pp = json.dumps(r.headers, indent=4)
except:
self.gh_pr_returned_data = r.content
returned_data_prettyprint = r.content
r_headers_pp = r.headers

# PR worked
if r.status_code == 201:
self.pr_url = self.gh_pr_returned_data["html_url"]
log.debug(f"GitHub API PR worked:\n{returned_data_prettyprint}")
log.debug(f"GitHub API PR worked:\n{returned_data_prettyprint}\n\n{r_headers_pp}")
log.info(f"GitHub PR created: {self.gh_pr_returned_data['html_url']}")
break

# Returned 403 error - too many simultaneous requests
# https://github.com/nf-core/tools/issues/911
if r.status_code == 403:
log.debug(f"GitHub API PR failed with 403 error:\n{returned_data_prettyprint}\n\n{r.headers}")
wait_time = float(re.sub("[^0-9]", "", r.headers.get("Retry-After", 30)))
log.debug(f"GitHub API PR failed with 403 error:\n{returned_data_prettyprint}\n\n{r_headers_pp}")
wait_time = float(re.sub("[^0-9]", "", str(r.headers.get("Retry-After", 0))))
if wait_time == 0:
log.debug("Couldn't find 'Retry-After' header, guessing a length of time to wait")
wait_time = random.randrange(10, 60)
log.warning(f"Got 403 code - probably the abuse protection. Trying again after {wait_time} seconds..")
time.sleep(wait_time)

# Something went wrong
else:
raise PullRequestException(
f"GitHub API returned code {r.status_code}: \n\n{returned_data_prettyprint}\n\n{r.headers}"
f"GitHub API returned code {r.status_code}: \n\n{returned_data_prettyprint}\n\n{r_headers_pp}"
)

def close_open_template_merge_prs(self):
Expand Down