Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://storage.scrapinghub.com/collections/233792/s/nikoncoolpix #2

Open
ghost opened this issue Sep 7, 2017 · 0 comments

Comments

@ghost
Copy link

ghost commented Sep 7, 2017

Following the Scrapy Price Monitor I encountered an error after successfully deploying project to Scrapy Cloud. Running for example amazon.com spider job, it is completed with 0 items and 5 errors (1 for each 'product name'). In job log I get (for 'product_name': 'nikoncoolpix'):
[scrapy.core.scraper] Error processing {'retailer': 'amazon.com', 'product_name': 'nikoncoolpix', 'when': '2017/09/07 03:57:21', 'price': 256.95, 'title': 'Nikon COOLPIX B500 Digital Camera (Red)', 'url': 'https://www.amazon.com/Nikon-COOLPIX-B500-Digital-Camera/dp/B01C3LEE9G'}

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/site-packages/twisted/internet/defer.py", line 588, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/app/__main__.egg/price_monitor/pipelines.py", line 20, in process_item
  File "/usr/local/lib/python3.5/site-packages/scrapinghub/hubstorage/collectionsrt.py", line 152, in set
    return self._collections.set(self.coltype, self.colname, *args, **kwargs)
  File "/usr/local/lib/python3.5/site-packages/scrapinghub/hubstorage/collectionsrt.py", line 56, in set
    return self.apipost((_type, _name), is_idempotent=True, jl=_values)
  File "/usr/local/lib/python3.5/site-packages/scrapinghub/hubstorage/resourcetype.py", line 74, in apipost
    return self.apirequest(_path, method='POST', **kwargs)
  File "/usr/local/lib/python3.5/site-packages/scrapinghub/hubstorage/resourcetype.py", line 71, in apirequest
    return jldecode(self._iter_lines(_path, **kwargs))
  File "/usr/local/lib/python3.5/site-packages/scrapinghub/hubstorage/resourcetype.py", line 60, in _iter_lines
    r = self.client.request(**kwargs)
  File "/usr/local/lib/python3.5/site-packages/scrapinghub/hubstorage/client.py", line 107, in request
    return self.retrier.call(invoke_request)
  File "/usr/local/lib/python3.5/site-packages/retrying.py", line 206, in call
    return attempt.get(self._wrap_exception)
  File "/usr/local/lib/python3.5/site-packages/retrying.py", line 247, in get
    six.reraise(self.value[0], self.value[1], self.value[2])
  File "/usr/local/lib/python3.5/site-packages/six.py", line 686, in reraise
    raise value
  File "/usr/local/lib/python3.5/site-packages/retrying.py", line 200, in call
    attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
  File "/usr/local/lib/python3.5/site-packages/scrapinghub/hubstorage/client.py", line 100, in invoke_request
    r.raise_for_status()
  File "/usr/local/lib/python3.5/site-packages/requests/models.py", line 844, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://storage.scrapinghub.com/collections/233792/s/nikoncoolpix

Same error is encountered when running spider in local environment. I would really appreciate any help.

System specifications:

  • OS Windows 10
  • Python 3.6.1.
  • Scrapy 1.4.0.
@ghost ghost changed the title requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://storage.scrapinghub.com/collections/233792/s/amazonechodot requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://storage.scrapinghub.com/collections/233792/s/nikoncoolpix Sep 7, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

0 participants