Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Source Google Analytics UA: fix check connection #20858

Merged
merged 3 commits into from
Jan 4, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -607,7 +607,7 @@
- name: Google Analytics (Universal Analytics)
sourceDefinitionId: eff3616a-f9c3-11eb-9a03-0242ac130003
dockerRepository: airbyte/source-google-analytics-v4
dockerImageTag: 0.1.32
dockerImageTag: 0.1.33
documentationUrl: https://docs.airbyte.com/integrations/sources/google-analytics-universal-analytics
icon: google-analytics.svg
sourceType: api
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5218,7 +5218,7 @@
oauthFlowOutputParameters:
- - "access_token"
- - "refresh_token"
- dockerImage: "airbyte/source-google-analytics-v4:0.1.32"
- dockerImage: "airbyte/source-google-analytics-v4:0.1.33"
spec:
documentationUrl: "https://docs.airbyte.com/integrations/sources/google-analytics-universal-analytics"
connectionSpecification:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ COPY main.py ./
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=0.1.32
LABEL io.airbyte.version=0.1.33
LABEL io.airbyte.name=airbyte/source-google-analytics-v4
Original file line number Diff line number Diff line change
Expand Up @@ -544,6 +544,10 @@ def stream_slices(self, stream_state: Mapping[str, Any] = None, **kwargs: Any) -
end_date = pendulum.now().date()
return [{"startDate": self.to_datetime_str(start_date), "endDate": self.to_datetime_str(end_date)}]

def parse_response(self, response: requests.Response, **kwargs: Any) -> Iterable[Mapping]:
res = response.json()
return res.get("reports", {})[0].get("data")


class SourceGoogleAnalyticsV4(AbstractSource):
"""Google Analytics lets you analyze data about customer engagement with your website or application."""
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -94,25 +94,29 @@ def test_no_regressions_for_result_is_sampled_and_data_is_golden_warnings(
def test_check_connection_fails_jwt(
jwt_encode_mock,
test_config_auth_service,
mocker,
requests_mock,
mock_metrics_dimensions_type_list_link,
mock_auth_call,
mock_api_returns_no_records,
mock_auth_call
):
"""
check_connection fails because of the API returns no records,
then we assume than user doesn't have permission to read requested `view`
"""
source = SourceGoogleAnalyticsV4()
requests_mock.register_uri("POST", "https://analyticsreporting.googleapis.com/v4/reports:batchGet",
[{"status_code": 403,
"json": {"results": [],
"error": "User does not have sufficient permissions for this profile."}}])

is_success, msg = source.check_connection(MagicMock(), test_config_auth_service)
assert is_success is False
assert (
msg
== f"Please check the permissions for the requested view_id: {test_config_auth_service['view_id']}. Cannot retrieve data from that view ID."
== f"Please check the permissions for the requested view_id: {test_config_auth_service['view_id']}. "
f"User does not have sufficient permissions for this profile."
)
jwt_encode_mock.encode.assert_called()
assert mock_auth_call.called
assert mock_api_returns_no_records.called


@patch("source_google_analytics_v4.source.jwt")
Expand Down Expand Up @@ -141,28 +145,31 @@ def test_check_connection_success_jwt(
def test_check_connection_fails_oauth(
jwt_encode_mock,
test_config,
mocker,
mock_metrics_dimensions_type_list_link,
mock_auth_call,
mock_api_returns_no_records,
requests_mock
):
"""
check_connection fails because of the API returns no records,
then we assume than user doesn't have permission to read requested `view`
"""
source = SourceGoogleAnalyticsV4()
requests_mock.register_uri("POST", "https://analyticsreporting.googleapis.com/v4/reports:batchGet",
[{"status_code": 403,
"json": {"results": [],
"error": "User does not have sufficient permissions for this profile."}}])
is_success, msg = source.check_connection(MagicMock(), test_config)
assert is_success is False
assert (
msg == f"Please check the permissions for the requested view_id: {test_config['view_id']}. Cannot retrieve data from that view ID."
msg == f"Please check the permissions for the requested view_id: {test_config['view_id']}."
f" User does not have sufficient permissions for this profile."
)
jwt_encode_mock.encode.assert_not_called()
assert "https://www.googleapis.com/auth/analytics.readonly" in unquote(mock_auth_call.last_request.body)
assert "client_id_val" in unquote(mock_auth_call.last_request.body)
assert "client_secret_val" in unquote(mock_auth_call.last_request.body)
assert "refresh_token_val" in unquote(mock_auth_call.last_request.body)
assert mock_auth_call.called
assert mock_api_returns_no_records.called


@patch("source_google_analytics_v4.source.jwt")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -159,8 +159,9 @@ Incremental sync is supported only if you add `ga:date` dimension to your custom

| Version | Date | Pull Request | Subject |
|:--------|:-----------|:---------------------------------------------------------|:---------------------------------------------------------------------------------------------|
| 0.1.32 | 2022-11-04 | [18965](https://github.com/airbytehq/airbyte/pull/18965) | Fix for `discovery` stage, when `custom_reports` are provided with single stream as `dict` |
| 0.1.31 | 2022-10-30 | [18670](https://github.com/airbytehq/airbyte/pull/18670) | Add `Custom Reports` schema validation on `check connection` |
| 0.1.33 | 2022-12-23 | [20858](https://github.com/airbytehq/airbyte/pull/20858) | Fix check connection |
| 0.1.32 | 2022-11-04 | [18965](https://github.com/airbytehq/airbyte/pull/18965) | Fix for `discovery` stage, when `custom_reports` are provided with single stream as `dict` |
| 0.1.31 | 2022-10-30 | [18670](https://github.com/airbytehq/airbyte/pull/18670) | Add `Custom Reports` schema validation on `check connection` |
| 0.1.30 | 2022-10-13 | [17943](https://github.com/airbytehq/airbyte/pull/17943) | Fix pagination |
| 0.1.29 | 2022-10-12 | [17905](https://github.com/airbytehq/airbyte/pull/17905) | Handle exceeded daily quota gracefully |
| 0.1.28 | 2022-09-24 | [16920](https://github.com/airbytehq/airbyte/pull/16920) | Added segments and filters to custom reports |
Expand Down