From b7b75592f5c8e92d6f7b8da5dc6b1ede2e21dbb3 Mon Sep 17 00:00:00 2001 From: qimingj Date: Thu, 19 Jan 2017 13:59:06 -0800 Subject: [PATCH] Mergemaster/cloudml (#134) * Add gcs_copy_file() that is missing but is referenced in a couple of places. (#110) * Add gcs_copy_file() that is missing but is referenced in a couple of places. * Add DataFlow to pydatalab dependency list. * Fix travis test errors by reimplementing gcs copy. * Remove unnecessary shutil import. * Flake8 configuration. Set max line length to 100. Ignore E111, E114 (#102) * Add datalab user agent to CloudML trainer and predictor requests. (#112) * Update oauth2client to 2.2.0 to satisfy cloudml in Cloud Datalab (#111) * Update README.md (#114) Added docs link. * Generate reST documentation for magic commands (#113) Auto generate docs for any added magics by searching through the source files for lines with register_line_cell_magic, capturing the names for those magics, and calling them inside an ipython kernel with the -h argument, then storing that output into a generated datalab.magics.rst file. * Fix an issue that %%chart failed with UDF query. (#116) * Fix an issue that %%chart failed with UDF query. The problem is that the query is submitted to BQ without replacing variable values from user namespace. * Fix chart tests by adding ip.user_ns mock. * Fix charting test. * Add missing import "mock". * Fix chart tests. * Fix "%%bigquery schema" issue -- the command generates nothing in output. (#119) * Add some missing dependencies, remove some unused ones (#122) * Remove scikit-learn and scipy as dependencies * add more required packages * Add psutil as dependency * Update packages versions * Cleanup (#123) * Remove unnecessary semicolons * remove unused imports * remove unncessary defined variable * Fix query_metadata tests (#128) Fix query_metadata tests * Make the library pip-installable (#125) This PR adds tensorflow and cloudml in setup.py to make the lib pip-installable. I had to install them explicitly using pip from inside the setup.py script, even though it's not a clean way to do it, it gets around the two issues we have at the moment with these two packags: - Pypi has Tensorflow version 0.12, while we need 0.11 for the current version of pydatalab. According to the Cloud ML docs, that version exists as a pip package for three supported platforms. - Cloud ML SDK exists as a pip package, but also not on Pypi, and while we could add it as a dependency link, there exists another package on Pypi called cloudml, and pip ends up installing that instead (see #124). I cannot find a way to force pip to install the package from the link I included. * Set command description so it is displayed in --help. argparser's format_help() prints description but not help. (#131) --- setup.py | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/setup.py b/setup.py index 739eace2a..8bbeb5067 100644 --- a/setup.py +++ b/setup.py @@ -113,3 +113,24 @@ ] } ) + +# for python2 only, install tensorflow and cloudml +if sys.version_info[0] == 2: + tensorflow_path = None + if platform.system() == 'Darwin': + tensorflow_path = 'https://storage.googleapis.com/tensorflow/mac/cpu/tensorflow-0.11.0-py2-none-any.whl' + elif platform.system() == 'Linux': + if platform.linux_distribution()[0] == 'Ubuntu': + tensorflow_path = 'https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.11.0-cp27-none-linux_x86_64.whl' + elif platform.linux_distribution()[0] == 'Debian': + tensorflow_path = 'https://storage.googleapis.com/tensorflow/linux/cpu/debian/jessie/tensorflow-0.11.0-cp27-none-linux_x86_64.whl' + + # install tensorflow + if not tensorflow_path: + print("""Warning: could not find tensorflow build for your OS. + Please go to https://www.tensorflow.org/get_started/os_setup to see install options""") + else: + pip.main(['install', tensorflow_path]) + + # install cloud ml sdk + pip.main(['install', 'https://storage.googleapis.com/cloud-ml/sdk/cloudml.latest.tar.gz'])