Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🎉 Source Mixpanel: Fix Mixpanel 429 status code is not handled #12286

Closed
wants to merge 5 commits into from
Closed

🎉 Source Mixpanel: Fix Mixpanel 429 status code is not handled #12286

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
b22e041
fix mixpanel rate limiting
itaseskii Apr 22, 2022
c4528e2
Merge branch 'master' into mixpanel-rate-limit-fix
itaseskii May 2, 2022
60fa965
update version
itaseskii May 2, 2022
d2eb925
resolve conflicts & remove duplicate reqs limit
itaseskii May 3, 2022
1acf63b
resolve conflicts
itaseskii May 11, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion airbyte-config/init/src/main/resources/seed/source_definitions.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -473,7 +473,7 @@
- name: Mixpanel
sourceDefinitionId: 12928b32-bf0a-4f1e-964f-07e12e37153a
dockerRepository: airbyte/source-mixpanel
dockerImageTag: 0.1.14
dockerImageTag: 0.1.15
documentationUrl: https://docs.airbyte.io/integrations/sources/mixpanel
icon: mixpanel.svg
sourceType: api
Expand Down
2 changes: 1 addition & 1 deletion airbyte-config/init/src/main/resources/seed/source_specs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5038,7 +5038,7 @@
path_in_connector_config:
- "credentials"
- "client_secret"
- dockerImage: "airbyte/source-mixpanel:0.1.14"
- dockerImage: "airbyte/source-mixpanel:0.1.15"
spec:
documentationUrl: "https://docs.airbyte.io/integrations/sources/mixpanel"
connectionSpecification:
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-mixpanel/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,5 @@ ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]


LABEL io.airbyte.version=0.1.14
LABEL io.airbyte.version=0.1.15
LABEL io.airbyte.name=airbyte/source-mixpanel
6 changes: 3 additions & 3 deletions airbyte-integrations/connectors/source-mixpanel/source_mixpanel/source.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ class MixpanelStream(HttpStream, ABC):
"""
Formatted API Rate Limit (https://help.mixpanel.com/hc/en-us/articles/115004602563-Rate-Limits-for-API-Endpoints):
A maximum of 5 concurrent queries
400 queries per hour.
60 queries per hour.

API Rate Limit Handler: after each request freeze for the time period: 3600/reqs_per_hour_limit seconds
"""
Expand All @@ -36,7 +36,7 @@ def url_base(self):
return f"https://{prefix}mixpanel.com/api/2.0/"

# https://help.mixpanel.com/hc/en-us/articles/115004602563-Rate-Limits-for-Export-API-Endpoints#api-export-endpoint-rate-limits
reqs_per_hour_limit: int = 400 # 1 req in 9 secs
reqs_per_hour_limit: int = 60 # 1 query per minute

def __init__(
self,
Expand Down Expand Up @@ -755,8 +755,8 @@ class Export(DateSlicesMixin, IncrementalMixpanelStream):
"""

primary_key: str = None

cursor_field: str = "time"
reqs_per_hour_limit: str = 60 # 1 query per minute

@property
def url_base(self):
Expand Down
3 changes: 2 additions & 1 deletion docs/integrations/sources/mixpanel.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,8 @@ Select the correct region \(EU or US\) for your Mixpanel project. See detail [he

| Version | Date | Pull Request | Subject |
|:---------|:-----------|:---------------------------------------------------------|:-----------------------------------------------------------------------------------------------------|
| `0.1.14` | 2022-05-02 | [11501](https://github.com/airbytehq/airbyte/pull/11501) | Improve incremental sync method to streams |
| `0.1.15` | 2022-05-03 | [12286](https://github.com/airbytehq/airbyte/pull/12286) | Fix Mixpanel 429 status code is not handled |
| `0.1.14` | 2022-05-02 | [11501](https://github.com/airbytehq/airbyte/pull/11501) | Improve incremental sync method to streams |
| `0.1.13` | 2022-04-27 | [12335](https://github.com/airbytehq/airbyte/pull/12335) | Adding fixtures to mock time.sleep for connectors that explicitly sleep |
| `0.1.12` | 2022-03-31 | [11633](https://github.com/airbytehq/airbyte/pull/11633) | Increase unit test coverage |
| `0.1.11` | 2022-04-04 | [11318](https://github.com/airbytehq/airbyte/pull/11318) | Change Response Reading |
Expand Down