🐛Source Amazon Seller Partner: Fix GET_VENDOR_TRAFFIC_REPORT when reportPeriod=DAY

[tianchen98]

What

issue: #37729

How

Added availablity_sla_days=3 and fixed_period_in_days=1 to ensure that all data in the lookback window can be fetched and that dataEndTime doesn't violate Amazon Vendor Central's SLA, which is 72 hours after the close of the period.

Review guide

streams.py

User Impact

Syncing GET_VENDOR_TRAFFIC_REPORT will not result in errors

Can this PR be safely reverted and rolled back?

• YES 💚
• NO ❌

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
airbyte-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jun 7, 2024 4:40pm

[CLAassistant]

All committers have signed the CLA.

[tianchen98]

what's the action that I need to take now? Should I update connector version?

[darynaishchenko]

what's the action that I need to take now? Should I update connector version?
@tianchen98
Thanks for the changes.

Did you test it on your data, everything is fine?

Please update connector version in metedata.yaml and pyproject.toml and add changelog record in docs/integrations/sources/amazon-seller-partner.md. Thanks.

[tianchen98]

Does PR branch need to stay up-to-date with master in order to merge? @darynaishchenko

[marcosmarxm]

Running tests @tianchen98

[marcosmarxm]

@tianchen98 it looks some unit tests are failing. Can you fix them?

[tianchen98]

@tianchen98 it looks some unit tests are failing. Can you fix them?

I can't really see the failed tests. It's returning Forbidden 403

[darynaishchenko]

@tianchen98 it looks some unit tests are failing. Can you fix them?

I can't really see the failed tests. It's returning Forbidden 403

@tianchen98, You can run poetry install --with dev and poetry run pytest unit_tests from connector's folder to run unit tests. Most of tests failed due to changes in request url so old request mocks are not match actual requests.

FAILED unit_tests/integration/test_report_based_streams.py::TestFullRefresh::test_given_report_when_read_then_return_records[GET_VENDOR_TRAFFIC_REPORT-json]
FAILED unit_tests/integration/test_report_based_streams.py::TestFullRefresh::test_given_compressed_report_when_read_then_return_records[GET_VENDOR_TRAFFIC_REPORT-json]
FAILED unit_tests/integration/test_report_based_streams.py::TestFullRefresh::test_given_http_status_500_then_200_when_create_report_then_retry_and_return_records[GET_VENDOR_TRAFFIC_REPORT-json]
FAILED unit_tests/integration/test_report_based_streams.py::TestFullRefresh::test_given_http_status_500_then_200_when_retrieve_report_then_retry_and_return_records[GET_VENDOR_TRAFFIC_REPORT-json]
FAILED unit_tests/integration/test_report_based_streams.py::TestFullRefresh::test_given_http_status_500_then_200_when_get_document_url_then_retry_and_return_records[GET_VENDOR_TRAFFIC_REPORT-json]
FAILED unit_tests/integration/test_report_based_streams.py::TestFullRefresh::test_given_http_status_500_then_200_when_download_document_then_retry_and_return_records[GET_VENDOR_TRAFFIC_REPORT-json]
FAILED unit_tests/integration/test_report_based_streams.py::TestFullRefresh::test_given_report_access_forbidden_when_read_then_no_records_and_error_logged[GET_VENDOR_TRAFFIC_REPORT-json]
FAILED unit_tests/integration/test_report_based_streams.py::TestFullRefresh::test_given_report_status_cancelled_when_read_then_stream_completed_successfully_and_warn_about_cancellation[GET_VENDOR_TRAFFIC_REPORT-json]
FAILED unit_tests/integration/test_report_based_streams.py::TestFullRefresh::test_given_report_status_fatal_when_read_then_exception_raised[GET_VENDOR_TRAFFIC_REPORT-json]
FAILED unit_tests/integration/test_report_based_streams.py::TestFullRefresh::test_given_http_error_500_on_create_report_when_read_then_no_records_and_error_logged[GET_VENDOR_TRAFFIC_REPORT-json]
FAILED unit_tests/integration/test_report_based_streams.py::TestIncremental::test_given_report_when_read_then_default_cursor_field_added_to_every_record[GET_VENDOR_TRAFFIC_REPORT-json]
FAILED unit_tests/integration/test_report_based_streams.py::TestIncremental::test_given_report_when_read_then_state_message_produced_and_state_match_latest_record[GET_VENDOR_TRAFFIC_REPORT-json]

[tianchen98]

Test can be passed if I remove fixed_period_in_days = 1 from the stream config, but that'll break the stream when reportPeriod is set to DAY

[darynaishchenko]

Test can be passed if I remove fixed_period_in_days = 1 from the stream config, but that'll break the stream when reportPeriod is set to DAY

Yes, I understood. You are allowed to change unit tests implementation if source code was changed.

[marcosmarxm]

Thanks for your contribution @tianchen98 the code was introduced in #38210. I'm going to close this now.