Fix missing dependencies

[ariarijp]

Step to reproduce the problem.

$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Importing base box 'redash/dev'...
==> default: Matching MAC address for NAT networking...

...snip...

==> default:   Running setup.py install for cryptography
==> default:
==> default:     Installed /tmp/pip_build_root/cryptography/cffi-1.8.3-py2.7-linux-x86_64.egg
==> default:     Traceback (most recent call last):
==> default:       File "<string>", line 1, in <module>
==> default:       File "/tmp/pip_build_root/cryptography/setup.py", line 333, in <module>
==> default:         **keywords_with_side_effects(sys.argv)
==> default:       File "/usr/lib/python2.7/distutils/core.py", line 111, in setup
==> default:         _setup_distribution = dist = klass(attrs)
==> default:       File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 243, in __init__
==> default:         _Distribution.__init__(self,attrs)
==> default:       File "/usr/lib/python2.7/distutils/dist.py", line 287, in __init__
==> default:         self.finalize_options()
==> default:       File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 277, in finalize_options
==> default:         ep.load()(self, ep.name, value)
==> default:       File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2088, in load
==> default:         entry = __import__(self.module_name, globals(),globals(), ['__name__'])
==> default:     ImportError: No module named setuptools_ext
==> default:     Complete output from command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/cryptography/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-HIo4YP-record/install-record.txt --single-version-externally-managed --compile:
==> default:
==> default:
==> default: Installed /tmp/pip_build_root/cryptography/cffi-1.8.3-py2.7-linux-x86_64.egg
==> default:
==> default: Traceback (most recent call last):
==> default:
==> default:   File "<string>", line 1, in <module>
==> default:
==> default:   File "/tmp/pip_build_root/cryptography/setup.py", line 333, in <module>
==> default:
==> default:     **keywords_with_side_effects(sys.argv)
==> default:
==> default:   File "/usr/lib/python2.7/distutils/core.py", line 111, in setup
==> default:
==> default:     _setup_distribution = dist = klass(attrs)
==> default:
==> default:   File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 243, in __init__
==> default:
==> default:     _Distribution.__init__(self,attrs)
==> default:
==> default:   File "/usr/lib/python2.7/distutils/dist.py", line 287, in __init__
==> default:
==> default:     self.finalize_options()
==> default:
==> default:   File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 277, in finalize_options
==> default:
==> default:     ep.load()(self, ep.name, value)
==> default:
==> default:   File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2088, in load
==> default:
==> default:     entry = __import__(self.module_name, globals(),globals(), ['__name__'])
==> default:
==> default: ImportError: No module named setuptools_ext
==> default:
==> default: ----------------------------------------
==> default:   Rolling back uninstall of cryptography
==> default: Cleaning up...
==> default: Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/cryptography/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-HIo4YP-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/cryptography
==> default: Storing debug log for failure in /home/vagrant/.pip/pip.log
==> default: Downloading/unpacking pymongo==3.2.1
==> default:   Running setup.py (path:/tmp/pip_build_root/pymongo/setup.py) egg_info for package pymongo
==> default:
==> default: Installing collected packages: pymongo
==> default:   Found existing installation: pymongo 2.7.2
==> default:     Uninstalling pymongo:
==> default:       Successfully uninstalled pymongo
==> default:   Running setup.py install for pymongo
==> default:     building 'bson._cbson' extension
==> default:     x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Ibson -I/usr/include/python2.7 -c bson/_cbsonmodule.c -o build/temp.linux-x86_64-2.7/bson/_cbsonmodule.o
==> default:     x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Ibson -I/usr/include/python2.7 -c bson/time64.c -o build/temp.linux-x86_64-2.7/bson/time64.o
==> default:     x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Ibson -I/usr/include/python2.7 -c bson/buffer.c -o build/temp.linux-x86_64-2.7/bson/buffer.o
==> default:     x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Ibson -I/usr/include/python2.7 -c bson/encoding_helpers.c -o build/temp.linux-x86_64-2.7/bson/encoding_helpers.o
==> default:     x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/bson/_cbsonmodule.o build/temp.linux-x86_64-2.7/bson/time64.o build/temp.linux-x86_64-2.7/bson/buffer.o build/temp.linux-x86_64-2.7/bson/encoding_helpers.o -o build/lib.linux-x86_64-2.7/bson/_cbson.so
==> default:     building 'pymongo._cmessage' extension
==> default:     x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Ibson -I/usr/include/python2.7 -c pymongo/_cmessagemodule.c -o build/temp.linux-x86_64-2.7/pymongo/_cmessagemodule.o
==> default:     x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Ibson -I/usr/include/python2.7 -c bson/buffer.c -o build/temp.linux-x86_64-2.7/bson/buffer.o
==> default:     x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/pymongo/_cmessagemodule.o build/temp.linux-x86_64-2.7/bson/buffer.o -o build/lib.linux-x86_64-2.7/pymongo/_cmessage.so
==> default:
==> default: Successfully installed pymongo
==> default: Cleaning up...
==> default: [2016-09-20 12:42:32,884][PID:1840][WARNING][redash.query_runner] GoogleSpreadsheet query runner enabled but not supported, not registering. Either disable or install missing dependencies.
==> default: [2016-09-20 12:42:32,938][PID:1840][WARNING][redash.query_runner] InfluxDB query runner enabled but not supported, not registering. Either disable or install missing dependencies.
==> default: [2016-09-20 12:42:32,948][PID:1840][WARNING][redash.query_runner] Presto query runner enabled but not supported, not registering. Either disable or install missing dependencies.
==> default: [2016-09-20 12:42:32,952][PID:1840][WARNING][redash.query_runner] Hive query runner enabled but not supported, not registering. Either disable or install missing dependencies.
==> default: [2016-09-20 12:42:32,959][PID:1840][WARNING][redash.query_runner] Vertica query runner enabled but not supported, not registering. Either disable or install missing dependencies.
==> default: [2016-09-20 12:42:32,962][PID:1840][WARNING][redash.query_runner] TreasureData query runner enabled but not supported, not registering. Either disable or install missing dependencies.
==> default: [2016-09-20 12:42:32,975][PID:1840][WARNING][redash.query_runner] Microsoft SQL Server query runner enabled but not supported, not registering. Either disable or install missing dependencies.
==> default: Traceback (most recent call last):
==> default:   File "/opt/redash/current/manage.py", line 10, in <module>
==> default:
==> default: from redash.wsgi import app
==> default:   File "/opt/redash/current/redash/wsgi.py", line 3, in <module>
==> default:
==> default: app = create_app()
==> default:   File "/opt/redash/current/redash/__init__.py", line 84, in create_app
==> default:
==> default: from redash import handlers
==> default:   File "/opt/redash/current/redash/handlers/__init__.py", line 5, in <module>
==> default:
==> default: from redash.authentication.org_resolving import current_org
==> default:   File "/opt/redash/current/redash/authentication/__init__.py", line 11, in <module>
==> default:
==> default: from redash.authentication import google_oauth, saml_auth, remote_user_auth
==> default:   File "/opt/redash/current/redash/authentication/google_oauth.py", line 5, in <module>
==> default:
==> default: from flask_oauthlib.client import OAuth
==> default:   File "/usr/local/lib/python2.7/dist-packages/flask_oauthlib/client.py", line 12, in <module>
==> default:
==> default: import oauthlib.oauth1
==> default: ImportError
==> default: :
==> default: No module named oauthlib.oauth1
==> default: [2016-09-20 12:42:33,713][PID:1847][WARNING][redash.query_runner] GoogleSpreadsheet query runner enabled but not supported, not registering. Either disable or install missing dependencies.
==> default: [2016-09-20 12:42:33,760][PID:1847][WARNING][redash.query_runner] InfluxDB query runner enabled but not supported, not registering. Either disable or install missing dependencies.
==> default: [2016-09-20 12:42:33,771][PID:1847][WARNING][redash.query_runner] Presto query runner enabled but not supported, not registering. Either disable or install missing dependencies.
==> default: [2016-09-20 12:42:33,773][PID:1847][WARNING][redash.query_runner] Hive query runner enabled but not supported, not registering. Either disable or install missing dependencies.
==> default: [2016-09-20 12:42:33,782][PID:1847][WARNING][redash.query_runner] Vertica query runner enabled but not supported, not registering. Either disable or install missing dependencies.
==> default: [2016-09-20 12:42:33,786][PID:1847][WARNING][redash.query_runner] TreasureData query runner enabled but not supported, not registering. Either disable or install missing dependencies.
==> default: [2016-09-20 12:42:33,799][PID:1847][WARNING][redash.query_runner] Microsoft SQL Server query runner enabled but not supported, not registering. Either disable or install missing dependencies.
==> default: Traceback (most recent call last):
==> default:   File "/opt/redash/current/manage.py", line 10, in <module>
==> default:
==> default: from redash.wsgi import app
==> default:   File "/opt/redash/current/redash/wsgi.py", line 3, in <module>
==> default:
==> default: app = create_app()
==> default:   File "/opt/redash/current/redash/__init__.py", line 84, in create_app
==> default:
==> default: from redash import handlers
==> default:   File "/opt/redash/current/redash/handlers/__init__.py", line 5, in <module>
==> default:
==> default: from redash.authentication.org_resolving import current_org
==> default:   File "/opt/redash/current/redash/authentication/__init__.py", line 11, in <module>
==> default:
==> default: from redash.authentication import google_oauth, saml_auth, remote_user_auth
==> default:   File "/opt/redash/current/redash/authentication/google_oauth.py", line 5, in <module>
==> default:
==> default: from flask_oauthlib.client import OAuth
==> default:   File "/usr/local/lib/python2.7/dist-packages/flask_oauthlib/client.py", line 12, in <module>
==> default:
==> default: import oauthlib.oauth1
==> default: ImportError
==> default: :
==> default: No module named oauthlib.oauth1
==> default: [2016-09-20 12:42:34,554][PID:1853][WARNING][redash.query_runner] GoogleSpreadsheet query runner enabled but not supported, not registering. Either disable or install missing dependencies.
==> default: [2016-09-20 12:42:34,599][PID:1853][WARNING][redash.query_runner] InfluxDB query runner enabled but not supported, not registering. Either disable or install missing dependencies.
==> default: [2016-09-20 12:42:34,609][PID:1853][WARNING][redash.query_runner] Presto query runner enabled but not supported, not registering. Either disable or install missing dependencies.
==> default: [2016-09-20 12:42:34,614][PID:1853][WARNING][redash.query_runner] Hive query runner enabled but not supported, not registering. Either disable or install missing dependencies.
==> default: [2016-09-20 12:42:34,620][PID:1853][WARNING][redash.query_runner] Vertica query runner enabled but not supported, not registering. Either disable or install missing dependencies.
==> default: [2016-09-20 12:42:34,623][PID:1853][WARNING][redash.query_runner] TreasureData query runner enabled but not supported, not registering. Either disable or install missing dependencies.
==> default: [2016-09-20 12:42:34,635][PID:1853][WARNING][redash.query_runner] Microsoft SQL Server query runner enabled but not supported, not registering. Either disable or install missing dependencies.
==> default: Traceback (most recent call last):
==> default:   File "/opt/redash/current/manage.py", line 10, in <module>
==> default:
==> default: from redash.wsgi import app
==> default:   File "/opt/redash/current/redash/wsgi.py", line 3, in <module>
==> default:
==> default: app = create_app()
==> default:   File "/opt/redash/current/redash/__init__.py", line 84, in create_app
==> default:
==> default: from redash import handlers
==> default:   File "/opt/redash/current/redash/handlers/__init__.py", line 5, in <module>
==> default:
==> default: from redash.authentication.org_resolving import current_org
==> default:   File "/opt/redash/current/redash/authentication/__init__.py", line 11, in <module>
==> default:
==> default: from redash.authentication import google_oauth, saml_auth, remote_user_auth
==> default:   File "/opt/redash/current/redash/authentication/google_oauth.py", line 5, in <module>
==> default:
==> default: from flask_oauthlib.client import OAuth
==> default:   File "/usr/local/lib/python2.7/dist-packages/flask_oauthlib/client.py", line 12, in <module>
==> default:
==> default: import oauthlib.oauth1
==> default: ImportError
==> default: :
==> default: No module named oauthlib.oauth1

...snip...

[arikfr]

I'm not sure what's going on here, because cffi is a dependency of cryptography and oauthlib is dependency of flask-oauthlib. Both of them already specified in requirements.txt.

So I'm not sure what's going on here... :-\ Was this a "new" Vagrant box?

[TonyStarkBy]

vagrant ssh
sudo pip install cffi
sudo pip install cryptography
sudo pip install oauthlib

This solution is works for me.

[ariarijp]

@arikfr Yes, this box is new one.

So, I tried again ./bin/vagrant_ctl.sh start instead of vagrant ssh on latest master branch.

But I got error similar to before error :-(

$ ./bin/vagrant_ctl.sh start
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Checking if box 'redash/dev' is up to date...
==> default: Couldn't find Cheffile at ./Cheffile.
==> default: Machine already provisioned. Run `vagrant provision` or use the `--provision`
==> default: flag to force provisioning. Provisioners marked to run always will still run.
12:25:36 web.1    | started with pid 1972
12:25:36 worker.1 | started with pid 1973
12:25:37 worker.1 | [2016-10-08 12:25:37,020][PID:1977][INFO][requests.packages.urllib3.connectionpool] Starting new HTTP connection (1): metadata.google.internal
12:25:37 worker.1 | [2016-10-08 12:25:37,042][PID:1977][WARNING][redash.query_runner] BigQueryGCE query runner enabled but not supported, not registering. Either disable or install missing dependencies.
12:25:37 worker.1 | [2016-10-08 12:25:37,179][PID:1977][WARNING][redash.query_runner] GoogleSpreadsheet query runner enabled but not supported, not registering. Either disable or install missing dependencies.
12:25:37 web.1    | [2016-10-08 12:25:37,375][PID:1975][INFO][requests.packages.urllib3.connectionpool] Starting new HTTP connection (1): metadata.google.internal
12:25:37 web.1    | [2016-10-08 12:25:37,384][PID:1975][WARNING][redash.query_runner] BigQueryGCE query runner enabled but not supported, not registering. Either disable or install missing dependencies.
12:25:37 web.1    | [2016-10-08 12:25:37,397][PID:1975][WARNING][redash.query_runner] GoogleSpreadsheet query runner enabled but not supported, not registering. Either disable or install missing dependencies.
12:25:37 worker.1 | [2016-10-08 12:25:37,404][PID:1977][WARNING][redash.query_runner] InfluxDB query runner enabled but not supported, not registering. Either disable or install missing dependencies.
12:25:37 worker.1 | [2016-10-08 12:25:37,429][PID:1977][WARNING][redash.query_runner] Presto query runner enabled but not supported, not registering. Either disable or install missing dependencies.
12:25:37 worker.1 | [2016-10-08 12:25:37,433][PID:1977][WARNING][redash.query_runner] Hive query runner enabled but not supported, not registering. Either disable or install missing dependencies.
12:25:37 worker.1 | [2016-10-08 12:25:37,448][PID:1977][WARNING][redash.query_runner] Vertica query runner enabled but not supported, not registering. Either disable or install missing dependencies.
12:25:37 worker.1 | [2016-10-08 12:25:37,452][PID:1977][WARNING][redash.query_runner] TreasureData query runner enabled but not supported, not registering. Either disable or install missing dependencies.
12:25:37 web.1    | [2016-10-08 12:25:37,453][PID:1975][WARNING][redash.query_runner] InfluxDB query runner enabled but not supported, not registering. Either disable or install missing dependencies.
12:25:37 web.1    | [2016-10-08 12:25:37,468][PID:1975][WARNING][redash.query_runner] Presto query runner enabled but not supported, not registering. Either disable or install missing dependencies.
12:25:37 worker.1 | [2016-10-08 12:25:37,469][PID:1977][WARNING][redash.query_runner] Microsoft SQL Server query runner enabled but not supported, not registering. Either disable or install missing dependencies.
12:25:37 web.1    | [2016-10-08 12:25:37,474][PID:1975][WARNING][redash.query_runner] Hive query runner enabled but not supported, not registering. Either disable or install missing dependencies.
12:25:37 web.1    | [2016-10-08 12:25:37,487][PID:1975][WARNING][redash.query_runner] Vertica query runner enabled but not supported, not registering. Either disable or install missing dependencies.
12:25:37 web.1    | [2016-10-08 12:25:37,496][PID:1975][WARNING][redash.query_runner] TreasureData query runner enabled but not supported, not registering. Either disable or install missing dependencies.
12:25:37 web.1    | [2016-10-08 12:25:37,518][PID:1975][WARNING][redash.query_runner] Microsoft SQL Server query runner enabled but not supported, not registering. Either disable or install missing dependencies.
12:25:37 worker.1 | [2016-10-08 12:25:37,654: WARNING/MainProcess] /usr/local/lib/python2.7/dist-packages/celery/apps/worker.py:161: CDeprecationWarning:
12:25:37 worker.1 | Starting from version 3.2 Celery will refuse to accept pickle by default.

12:25:37 worker.1 | The pickle serializer is a security concern as it may give attackers
12:25:37 worker.1 | the ability to execute any command.  It's important to secure
12:25:37 worker.1 | your broker from unauthorized access when using pickle, so we think
12:25:37 worker.1 | that enabling pickle should require a deliberate action and not be
12:25:37 worker.1 | the default choice.

12:25:37 worker.1 | If you depend on pickle then you should set a setting to disable this
12:25:37 worker.1 | warning and to be sure that everything will continue working
12:25:37 worker.1 | when you upgrade to Celery 3.2::

12:25:37 worker.1 |     CELERY_ACCEPT_CONTENT = ['pickle', 'json', 'msgpack', 'yaml']

12:25:37 worker.1 | You must only enable the serializers that you will actually use.


12:25:37 worker.1 |   warnings.warn(CDeprecationWarning(W_PICKLE_DEPRECATED))
12:25:37 web.1    | Traceback (most recent call last):
12:25:37 web.1    |   File "./manage.py", line 10, in <module>
12:25:37 web.1    |     from redash.wsgi import app
12:25:37 web.1    |   File "/opt/redash/current/redash/wsgi.py", line 3, in <module>
12:25:37 web.1    |     app = create_app()
12:25:37 web.1    |   File "/opt/redash/current/redash/__init__.py", line 76, in create_app
12:25:37 web.1    |     from redash import handlers
12:25:37 web.1    |   File "/opt/redash/current/redash/handlers/__init__.py", line 5, in <module>
12:25:37 web.1    |     from redash.authentication.org_resolving import current_org
12:25:37 web.1    |   File "/opt/redash/current/redash/authentication/__init__.py", line 11, in <module>
12:25:37 web.1    |     from redash.authentication import google_oauth, saml_auth, remote_user_auth
12:25:37 web.1    |   File "/opt/redash/current/redash/authentication/google_oauth.py", line 5, in <module>
12:25:37 web.1    |     from flask_oauthlib.client import OAuth
12:25:37 web.1    |   File "/usr/local/lib/python2.7/dist-packages/flask_oauthlib/client.py", line 12, in <module>
12:25:37 web.1    |     import oauthlib.oauth1
12:25:37 web.1    | ImportError: No module named oauthlib.oauth1
12:25:37 web.1    | process terminated
12:25:37 system   | sending SIGTERM to all processes
12:25:37 system   | sending SIGTERM to pid 1973
12:25:37 worker.1 | process terminated

[arikfr]

This is an issue with the Vagrant box... I opened #1480 to track the work on the new dev guide & probably new Vagrant box.

[ariarijp]

Sorry for late reply. I will track #1480.