Skip to content

Commit

Permalink
Azure retry on result timeout / server busy (#950)
Browse files Browse the repository at this point in the history
# Pull Request

## Azure retry on result timeout / server busy

This fixes a small issue where Azure ARM requests could intermittently
fail if the corresponding control plane is too busy.


______________________________________________________________________

## Description

This manifests as a HTTP 503 response code, with a message like:
```
ERROR Response: <Response [503]> :: {"error":{"code":"ServerTimeout","message":"The request timed out. Diagnostic information: timestamp '20250209T125446Z', subscription id '...', tracking id '...', request correlation id '...'."}}
```

According to the official docs, these responses indicate the request
should be retried:
https://learn.microsoft.com/en-us/rest/api/storageservices/common-rest-api-error-codes

______________________________________________________________________

## Type of Change

- 🛠️ Bug fix

______________________________________________________________________

## Testing

Manual testing, fix that we have been using for a while in experiments
but have not merged yet.

______________________________________________________________________

## Additional Notes (optional)


______________________________________________________________________

Co-authored-by: Eu Jing Chua <eujingchua@microsoft.com>
  • Loading branch information
eujing and Eu Jing Chua authored Feb 11, 2025
1 parent e94989e commit d168d87
Showing 1 changed file with 5 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,11 @@ def _get_session(self, params: dict) -> requests.Session:
session = requests.Session()
session.mount(
"https://",
HTTPAdapter(max_retries=Retry(total=total_retries, backoff_factor=backoff_factor)),
HTTPAdapter(
max_retries=Retry(
total=total_retries, backoff_factor=backoff_factor, status_forcelist=[503]
)
),
)
session.headers.update(self._get_headers())
return session
Expand Down

0 comments on commit d168d87

Please sign in to comment.