Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check existence of export file with request before resuming sync #62

Merged
merged 3 commits into from
Mar 4, 2020

Conversation

dmosorast
Copy link
Contributor

Description of change

It appears that there's a situation where Marketo marks an export as "Completed", but returns a 404 when requesting the file itself. This is an error scenario as far as the tap is concerned.

This PR adjusts the tap's behavior when resuming to check for a 404 when requesting a 0 byte range of the file, and reporting it to the sync code as not existing if that check returns a 404.

Manual QA steps

  • Ran through a situation with a bookmarked file ID that didn't exist, and ensured it started a new sync.
  • Ran through a new sync and ensured that the records were requested and emitted properly
  • Ran through end to end with the current code to ensure that it still works

Risks

  • Low, currently the tap will just keep persisting the same state when a bunk export_id gets in there.

Rollback steps

  • revert this branch, bump patch version, release new version

tap_marketo/client.py Outdated Show resolved Hide resolved
tap_marketo/client.py Show resolved Hide resolved
tap_marketo/client.py Outdated Show resolved Hide resolved
@dmosorast dmosorast merged commit 58fc698 into master Mar 4, 2020
@dmosorast dmosorast deleted the fix/404-error-on-exports branch March 4, 2020 22:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants