Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle exceptions outside of retryable exceptions #65

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

luandy64
Copy link
Contributor

Description of change

The tap uses a client to talk to Zuora and it only looks at HTTP 429 and HTTP 500 error codes.

When there is a 401 error for example, the error is obscured by some generic error the tap throws.

This PR forces any non-200, non-retryable response to terminate the tap.

Before the change

CRITICAL Could not discover US-based REST Production data center url out of ['https://rest.na.zuora.com/', 'https://rest.zuora.com/']
Traceback (most recent call last):
  File "/usr/local/share/virtualenvs/tap-zuora/bin/tap-zuora", line 33, in <module>
    sys.exit(load_entry_point('tap-zuora', 'console_scripts', 'tap-zuora')())
  File "/usr/local/share/virtualenvs/tap-zuora/lib/python3.8/site-packages/singer/utils.py", line 229, in wrapped
    return fnc(*args, **kwargs)
  File "/home/vagrant/git/tap-zuora/tap_zuora/__init__.py", line 114, in main
    client = Client.from_config(args.config)
  File "/home/vagrant/git/tap-zuora/tap_zuora/client.py", line 47, in from_config
    return Client(config['username'], config['password'], partner_id, sandbox, european)
  File "/home/vagrant/git/tap-zuora/tap_zuora/client.py", line 36, in __init__
    self.rest_url = self.get_url(rest=True)
  File "/home/vagrant/git/tap-zuora/tap_zuora/client.py", line 69, in get_url
    raise Exception("Could not discover {} {} {} data center url out of {}"
Exception: Could not discover US-based REST Production data center url out of ['https://rest.na.zuora.com/', 'https://rest.zuora.com/']

After the change

CRITICAL 401 Client Error: Unauthorized for url: https://rest.na.zuora.com/v1/describe/Account
Traceback (most recent call last):
  File "/usr/local/share/virtualenvs/tap-zuora/bin/tap-zuora", line 33, in <module>
    sys.exit(load_entry_point('tap-zuora', 'console_scripts', 'tap-zuora')())
  File "/usr/local/share/virtualenvs/tap-zuora/lib/python3.8/site-packages/singer/utils.py", line 229, in wrapped
    return fnc(*args, **kwargs)
  File "/home/vagrant/git/tap-zuora/tap_zuora/__init__.py", line 114, in main
    client = Client.from_config(args.config)
  File "/home/vagrant/git/tap-zuora/tap_zuora/client.py", line 47, in from_config
    return Client(config['username'], config['password'], partner_id, sandbox, european)
  File "/home/vagrant/git/tap-zuora/tap_zuora/client.py", line 36, in __init__
    self.rest_url = self.get_url(rest=True)
  File "/home/vagrant/git/tap-zuora/tap_zuora/client.py", line 54, in get_url
    resp = self._retryable_request("GET","{}v1/describe/{}".format(url_prefix, stream_name), headers=self.rest_headers)
  File "/usr/local/share/virtualenvs/tap-zuora/lib/python3.8/site-packages/backoff/_sync.py", line 94, in retry
    ret = target(*args, **kwargs)
  File "/home/vagrant/git/tap-zuora/tap_zuora/client.py", line 103, in _retryable_request
    resp.raise_for_status()
  File "/usr/local/share/virtualenvs/tap-zuora/lib/python3.8/site-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://rest.na.zuora.com/v1/describe/Account

Manual QA steps

  • Ran the tap manually

Risks

  • Low. All errors raised from this change should have caused the tap to fail anyway. This change just helps a user understand the error better.

Rollback steps

  • revert this branch, bump the version

@luandy64 luandy64 mentioned this pull request Aug 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant