Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proxylist cant be loaded on Scrapy Cloud #29

Open
robelozano opened this issue May 28, 2017 · 1 comment
Open

proxylist cant be loaded on Scrapy Cloud #29

robelozano opened this issue May 28, 2017 · 1 comment

Comments

@robelozano
Copy link

I try several time diferente ways

I add the file "proxylist.txt" in the same folder than setting than the project in addition i upload it to "https://dl.dropboxusercontent.com/s/esdm19mnvz2yguf/proxylist.txt"

I substitute the name in the:
PROXY_LIST = 'https://dl.dropboxusercontent.com/s/esdm19mnvz2yguf/proxylist.txt'
or
PROXY_LIST = 'proxylist.txt'
or
PROXY_LIST = '/proxylist.txt'
PROXY_LIST = '../proxylist.txt'

if i do it like PROXY_LIST = 'proxylist.txt' in my PC, it works like a charm but not once i load it in Scrapy Cloud.

Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1299, in _inlineCallbacks
result = g.send(result)
File "/usr/local/lib/python2.7/site-packages/scrapy/crawler.py", line 90, in crawl
six.reraise(*exc_info)
File "/usr/local/lib/python2.7/site-packages/scrapy/crawler.py", line 72, in crawl
self.engine = self._create_engine()
File "/usr/local/lib/python2.7/site-packages/scrapy/crawler.py", line 97, in _create_engine
return ExecutionEngine(self, lambda _: self.stop())
File "/usr/local/lib/python2.7/site-packages/scrapy/core/engine.py", line 69, in init
self.downloader = downloader_cls(crawler)
File "/usr/local/lib/python2.7/site-packages/scrapy/core/downloader/init.py", line 88, in init
self.middleware = DownloaderMiddlewareManager.from_crawler(crawler)
File "/usr/local/lib/python2.7/site-packages/scrapy/middleware.py", line 58, in from_crawler
return cls.from_settings(crawler.settings, crawler)
File "/usr/local/lib/python2.7/site-packages/scrapy/middleware.py", line 36, in from_settings
mw = mwcls.from_crawler(crawler)
File "/app/python/lib/python2.7/site-packages/scrapy_proxies/randomproxy.py", line 55, in from_crawler
return cls(crawler.settings)
File "/app/python/lib/python2.7/site-packages/scrapy_proxies/randomproxy.py", line 35, in init
fin = open(self.proxy_list)
IOError: [Errno 2] No such file or directory: '../proxylist.txt'

please i need some help.

@MrAutoIt
Copy link

MrAutoIt commented Nov 3, 2019

Did you ever get this to work? I am having the same issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants