Skip to content
This repository has been archived by the owner on Aug 8, 2023. It is now read-only.

[core] rate limit handling on online file source #6223

Merged
merged 5 commits into from
Sep 13, 2016

Conversation

ivovandongen
Copy link
Contributor

Fixes #5821

@ivovandongen ivovandongen added ⚠️ DO NOT MERGE Work in progress, proof of concept, or on hold Core The cross-platform C++ core, aka mbgl iOS Mapbox Maps SDK for iOS macOS Mapbox Maps SDK for macOS Android Mapbox Maps SDK for Android Linux Qt Mapbox Maps SDK for Qt, aka Qt Location Mapbox GL and removed ⚠️ DO NOT MERGE Work in progress, proof of concept, or on hold labels Aug 31, 2016
@ivovandongen
Copy link
Contributor Author

I've implemented the core and platform specifics. All platforms, expect for Android, have the same unit test. It would be nice to also test this against an actual server with a low rate limit as well, just to be sure.

Error(Reason, std::string = "", optional<RetryAfter> = {});

private:
optional<Timestamp> parseRetryAfter(const optional<RetryAfter> retryAfter_) const;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you move this to http_header.hpp and make the signature:

optional<Timestamp> parseRetryHeaders(const optional<std::string>& retryAfter,
                                      const optional<std::string>& xRateLimitReset);

Advantages:

  • No need for RetryAfter type
  • Factor conditional and integer conversion logic from individual SDK implementations

@ivovandongen
Copy link
Contributor Author

@jfirebaugh Thanks for taking a look. I've changed the implementations and will rework the tests.

@ivovandongen
Copy link
Contributor Author

@jfirebaugh

  • I've extracted the timeout functions to a header in src/util so it is testable, didn't want to add it to the public header as it is not quite public api. Let me know if this isn't what you had in mind.
  • I kept the integration tests so that the platform bindings are tested as well.
  • I've added unit tests on the timeout functions directly.

When writing the unit tests I ran into a little issue where the the timeout became negative on high number of errors as the sign bit flipped (here among others). Changed all three exponential backoffs so that doesn't happen (1 -> 1u) here


optional<Timestamp> parseRetryHeaders(const optional<std::string>& retryAfter,
const optional<std::string>& xRateLimitReset) {
optional<Timestamp> result;
Copy link
Contributor

@jfirebaugh jfirebaugh Sep 6, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prefer early returns to temporary variables:

if (retryAfter) {
    try {
        return ...;
    } catch ... {
        return ...;
    }
}

if (xRateLimitReset) {
    return ...;
}

return {};


optional<Timestamp> parseRetryHeaders(const optional<std::string>& retryAfter,
const optional<std::string>& xRateLimitReset) {
if(retryAfter) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit, missing space after if

@tmpsantos
Copy link
Contributor

Thanks for adding more tests!

@ivovandongen
Copy link
Contributor Author

@tmpsantos Thanks for the review! Made the changes as suggested. Waiting on CI and then I'll merge.

@ivovandongen ivovandongen merged commit ed4e37b into master Sep 13, 2016
@ivovandongen ivovandongen deleted the 5821-filesource-rate-limit branch September 13, 2016 12:45
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Android Mapbox Maps SDK for Android Core The cross-platform C++ core, aka mbgl iOS Mapbox Maps SDK for iOS Linux macOS Mapbox Maps SDK for macOS Qt Mapbox Maps SDK for Qt, aka Qt Location Mapbox GL
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants