Source: Amazon Ads - Add pk validator for reports when expected pk is…

… not returned (airbytehq#25146)

* Add pk validator for reports when expected pk is not returned

* Update PR number

* Fix formating

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>

airbyte-config-oss/init-oss/src/main/resources/seed/oss_catalog.json

@@ -8481,7 +8481,7 @@
     "sourceDefinitionId": "c6b0a29e-1da9-4512-9002-7bfd0cba2246",
     "name": "Amazon Ads",
     "dockerRepository": "airbyte/source-amazon-ads",
-    "dockerImageTag": "1.0.2",
+    "dockerImageTag": "1.0.3",
     "documentationUrl": "https://docs.airbyte.com/integrations/sources/amazon-ads",
     "icon": "amazonads.svg",
     "sourceType": "api",

airbyte-config-oss/init-oss/src/main/resources/seed/source_definitions.yaml

@@ -65,7 +65,7 @@
 - name: Amazon Ads
   sourceDefinitionId: c6b0a29e-1da9-4512-9002-7bfd0cba2246
   dockerRepository: airbyte/source-amazon-ads
-  dockerImageTag: 1.0.2
+  dockerImageTag: 1.0.3
   documentationUrl: https://docs.airbyte.com/integrations/sources/amazon-ads
   icon: amazonads.svg
   sourceType: api

airbyte-config-oss/init-oss/src/main/resources/seed/source_specs.yaml

@@ -822,7 +822,7 @@
     supportsNormalization: false
     supportsDBT: false
     supported_destination_sync_modes: []
-- dockerImage: "airbyte/source-amazon-ads:1.0.2"
+- dockerImage: "airbyte/source-amazon-ads:1.0.3"
   spec:
     documentationUrl: "https://docs.airbyte.com/integrations/sources/amazon-ads"
     connectionSpecification:

airbyte-integrations/connectors/source-amazon-ads/Dockerfile

@@ -14,5 +14,5 @@ ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]
 
 
 
-LABEL io.airbyte.version=1.0.2
+LABEL io.airbyte.version=1.0.3
 LABEL io.airbyte.name=airbyte/source-amazon-ads

airbyte-integrations/connectors/source-amazon-ads/source_amazon_ads/streams/report_streams/report_streams.py

@@ -4,6 +4,7 @@
 
 import json
 import re
+import uuid
 from abc import ABC, abstractmethod
 from copy import deepcopy
 from dataclasses import dataclass
@@ -165,7 +166,7 @@ def read_records(
                     profileId=report_info.profile_id,
                     recordType=report_info.record_type,
                     reportDate=report_date,
-                    recordId=metric_object[self.metrics_type_to_id_map[report_info.record_type]],
+                    recordId=metric_object.get(self.metrics_type_to_id_map[report_info.record_type], str(uuid.uuid4())),
                     metric=metric_object,
                 ).dict()
 

airbyte-integrations/connectors/source-amazon-ads/unit_tests/test_report_streams.py

@@ -103,6 +103,15 @@
 )
 METRICS_COUNT = 5
 
+METRIC_RESPONSE_WITHOUT_ASIN_PK = b64decode(
+    """
+    H4sIAAAAAAAAClXMvQrCMBQF4L1PETIbyM1v4+giLr6AOAQTStH+ECsi4r
+    t7aZOKwx3u+TjnVBHyxiOEXnw3+rbpD4FuSe205IYrvvnHo+8iMr3jf4us
+    xKzHnK0lmls+7NPwGOdFapTjINXPcmy0FByELjD51MRpQesAUSooeI2v55
+    DCrp1ZAwipnF1HMy9lZ7StJRdAET/V+Qv1VRgd7AAAAA==
+"""
+)
+
 
 def setup_responses(init_response=None, init_response_products=None, init_response_brands=None, status_response=None, metric_response=None):
     if init_response:
@@ -206,6 +215,23 @@ def test_products_report_stream(config):
     assert len(metrics) == METRICS_COUNT * len(stream.metrics_map)
 
 
+@responses.activate
+def test_products_report_stream_without_pk(config):
+    setup_responses(
+        init_response_products=REPORT_INIT_RESPONSE,
+        status_response=REPORT_STATUS_RESPONSE,
+        metric_response=METRIC_RESPONSE_WITHOUT_ASIN_PK,
+    )
+
+    profiles = make_profiles(profile_type="vendor")
+
+    stream = SponsoredProductsReportStream(config, profiles, authenticator=mock.MagicMock())
+    stream_slice = {"profile": profiles[0], "reportDate": "2021-07-25", "retry_count": 3}
+    metrics = [m for m in stream.read_records(SyncMode.incremental, stream_slice=stream_slice)]
+
+    assert len(metrics) == len(stream.metrics_map)
+
+
 @responses.activate
 def test_brands_report_stream(config):
     setup_responses(

connectors.md

@@ -11,7 +11,7 @@
 | **Airtable** | <img alt="Airtable icon" src="https://raw.githubusercontent.com/airbytehq/airbyte/master/airbyte-config-oss/init-oss/src/main/resources/icons/airtable.svg" height="30" height="30"/> | Source | airbyte/source-airtable:3.0.0 | generally_available | [link](https://docs.airbyte.com/integrations/sources/airtable) | [code](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-airtable) | <small>`14c6e7ea-97ed-4f5e-a7b5-25e9a80b8212`</small> |
 | **AlloyDB for PostgreSQL** | <img alt="AlloyDB for PostgreSQL icon" src="https://raw.githubusercontent.com/airbytehq/airbyte/master/airbyte-config-oss/init-oss/src/main/resources/icons/alloydb.svg" height="30" height="30"/> | Source | airbyte/source-alloydb:2.0.19 | generally_available | [link](https://docs.airbyte.com/integrations/sources/alloydb) | [code](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-alloydb) | <small>`1fa90628-2b9e-11ed-a261-0242ac120002`</small> |
 | **Alpha Vantage** | <img alt="Alpha Vantage icon" src="https://raw.githubusercontent.com/airbytehq/airbyte/master/airbyte-config-oss/init-oss/src/main/resources/icons/alpha-vantage.svg" height="30" height="30"/> | Source | airbyte/source-alpha-vantage:0.1.1 | alpha | [link](https://docs.airbyte.com/integrations/sources/alpha-vantage) | [code](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-alpha-vantage) | <small>`db385323-9333-4fec-bec3-9e0ca9326c90`</small> |
-| **Amazon Ads** | <img alt="Amazon Ads icon" src="https://raw.githubusercontent.com/airbytehq/airbyte/master/airbyte-config-oss/init-oss/src/main/resources/icons/amazonads.svg" height="30" height="30"/> | Source | airbyte/source-amazon-ads:1.0.2 | generally_available | [link](https://docs.airbyte.com/integrations/sources/amazon-ads) | [code](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-amazon-ads) | <small>`c6b0a29e-1da9-4512-9002-7bfd0cba2246`</small> |
+| **Amazon Ads** | <img alt="Amazon Ads icon" src="https://raw.githubusercontent.com/airbytehq/airbyte/master/airbyte-config-oss/init-oss/src/main/resources/icons/amazonads.svg" height="30" height="30"/> | Source | airbyte/source-amazon-ads:1.0.3 | generally_available | [link](https://docs.airbyte.com/integrations/sources/amazon-ads) | [code](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-amazon-ads) | <small>`c6b0a29e-1da9-4512-9002-7bfd0cba2246`</small> |
 | **Amazon SQS** | <img alt="Amazon SQS icon" src="https://raw.githubusercontent.com/airbytehq/airbyte/master/airbyte-config-oss/init-oss/src/main/resources/icons/awssqs.svg" height="30" height="30"/> | Source | airbyte/source-amazon-sqs:0.1.0 | alpha | [link](https://docs.airbyte.com/integrations/sources/amazon-sqs) | [code](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-amazon-sqs) | <small>`983fd355-6bf3-4709-91b5-37afa391eeb6`</small> |
 | **Amazon Seller Partner** | <img alt="Amazon Seller Partner icon" src="https://raw.githubusercontent.com/airbytehq/airbyte/master/airbyte-config-oss/init-oss/src/main/resources/icons/amazonsellerpartner.svg" height="30" height="30"/> | Source | airbyte/source-amazon-seller-partner:1.0.1 | alpha | [link](https://docs.airbyte.com/integrations/sources/amazon-seller-partner) | [code](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-amazon-seller-partner) | <small>`e55879a8-0ef8-4557-abcf-ab34c53ec460`</small> |
 | **Amplitude** | <img alt="Amplitude icon" src="https://raw.githubusercontent.com/airbytehq/airbyte/master/airbyte-config-oss/init-oss/src/main/resources/icons/amplitude.svg" height="30" height="30"/> | Source | airbyte/source-amplitude:0.2.0 | generally_available | [link](https://docs.airbyte.com/integrations/sources/amplitude) | [code](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-amplitude) | <small>`fa9f58c6-2d03-4237-aaa4-07d75e0c1396`</small> |

docs/integrations/sources/amazon-ads.md

@@ -94,6 +94,7 @@ Information about expected report generation waiting time you may find [here](ht
 
 | Version | Date       | Pull Request                                             | Subject                                                                                                         |
 |:--------|:-----------|:---------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------|
+| 1.0.3   | 2023-04-13 | [25146](https://github.com/airbytehq/airbyte/pull/25146) | Validate pk for reports when expected pk is not returned                                                        |
 | 1.0.2   | 2023-02-03 | [22355](https://github.com/airbytehq/airbyte/pull/22355) | Migrate `products_report` stream to API v3                                                                      |
 | 1.0.1   | 2022-11-01 | [18677](https://github.com/airbytehq/airbyte/pull/18677) | Add optional config report_record_types                                                                         |
 | 1.0.0   | 2023-01-30 | [21677](https://github.com/airbytehq/airbyte/pull/21677) | Fix bug with non-unique primary keys in report streams. Add asins_keywords and asins_targets                    |