You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Python SDK client library is not compatible with the most popular DS installations in GCP.
I was using !pip install feast together with standard Jupyter notebook installations for TensorFlow 1.15.0 and 2.2.0.
In both cases the installations end up with the following errors:
Collecting feast
Downloading feast-0.6.2-py3-none-any.whl (116 kB)
|████████████████████████████████| 116 kB 7.1 MB/s eta 0:00:01
Collecting google-cloud-bigquery==1.18.*
Downloading google_cloud_bigquery-1.18.1-py2.py3-none-any.whl (146 kB)
|████████████████████████████████| 146 kB 21.2 MB/s eta 0:00:01
Requirement already satisfied: googleapis-common-protos==1.* in /opt/conda/lib/python3.7/site-packages (from feast) (1.51.0)
Requirement already satisfied: Click==7.* in /opt/conda/lib/python3.7/site-packages (from feast) (7.1.2)
Collecting google
Downloading google-3.0.0-py2.py3-none-any.whl (45 kB)
|████████████████████████████████| 45 kB 4.3 MB/s eta 0:00:01
Collecting google-api-core==1.20.*
Downloading google_api_core-1.20.1-py2.py3-none-any.whl (90 kB)
|████████████████████████████████| 90 kB 9.2 MB/s eta 0:00:01
Collecting kafka-python==1.*
Downloading kafka_python-1.4.7-py2.py3-none-any.whl (266 kB)
|████████████████████████████████| 266 kB 16.6 MB/s eta 0:00:01
Requirement already satisfied: tabulate==0.8.* in /opt/conda/lib/python3.7/site-packages (from feast) (0.8.7)
Requirement already satisfied: google-auth<2.0dev,>=1.14.0 in /opt/conda/lib/python3.7/site-packages (from feast) (1.19.2)
Collecting google-cloud-storage==1.20.*
Downloading google_cloud_storage-1.20.0-py2.py3-none-any.whl (70 kB)
|████████████████████████████████| 70 kB 10.7 MB/s eta 0:00:01
Collecting google-cloud-bigquery-storage==0.7.*
Downloading google_cloud_bigquery_storage-0.7.0-py2.py3-none-any.whl (55 kB)
|████████████████████████████████| 55 kB 5.3 MB/s eta 0:00:01
Requirement already satisfied: pandas~=1.0.0 in /opt/conda/lib/python3.7/site-packages (from feast) (1.0.5)
Collecting google-cloud-core==1.0.*
Downloading google_cloud_core-1.0.3-py2.py3-none-any.whl (26 kB)
Requirement already satisfied: protobuf>=3.10 in /opt/conda/lib/python3.7/site-packages (from feast) (3.12.3)
Requirement already satisfied: grpcio==1.* in /opt/conda/lib/python3.7/site-packages (from feast) (1.30.0)
Requirement already satisfied: numpy in /opt/conda/lib/python3.7/site-packages (from feast) (1.19.1)
Requirement already satisfied: toml==0.10.* in /opt/conda/lib/python3.7/site-packages (from feast) (0.10.1)
Collecting confluent-kafka
Downloading confluent_kafka-1.5.0-cp37-cp37m-manylinux1_x86_64.whl (8.1 MB)
|████████████████████████████████| 8.1 MB 15.8 MB/s eta 0:00:01
Collecting PyYAML==5.1.*
Downloading PyYAML-5.1.2.tar.gz (265 kB)
|████████████████████████████████| 265 kB 74.1 MB/s eta 0:00:01
Requirement already satisfied: tqdm==4.* in /opt/conda/lib/python3.7/site-packages (from feast) (4.48.0)
Collecting fastavro<0.23,>=0.22.11
Downloading fastavro-0.22.13-cp37-cp37m-manylinux2010_x86_64.whl (1.3 MB)
|████████████████████████████████| 1.3 MB 43.8 MB/s eta 0:00:01
Collecting pandavro==1.5.*
Downloading pandavro-1.5.2.tar.gz (3.8 kB)
Collecting pyarrow<0.16.0,>=0.15.1
Downloading pyarrow-0.15.1-cp37-cp37m-manylinux2010_x86_64.whl (59.2 MB)
|████████████████████████████████| 59.2 MB 2.4 MB/s eta 0:00:011:00:08.2 MB 2.4 MB/s eta 0:00:06
Collecting google-resumable-media<0.5.0dev,>=0.3.1
Downloading google_resumable_media-0.4.1-py2.py3-none-any.whl (38 kB)
Requirement already satisfied: beautifulsoup4 in /opt/conda/lib/python3.7/site-packages (from google->feast) (4.9.1)
Requirement already satisfied: setuptools>=34.0.0 in /opt/conda/lib/python3.7/site-packages (from google-api-core==1.20.*->feast) (49.2.0.post20200712)
Requirement already satisfied: six>=1.10.0 in /opt/conda/lib/python3.7/site-packages (from google-api-core==1.20.*->feast) (1.15.0)
Requirement already satisfied: requests<3.0.0dev,>=2.18.0 in /opt/conda/lib/python3.7/site-packages (from google-api-core==1.20.*->feast) (2.24.0)
Requirement already satisfied: pytz in /opt/conda/lib/python3.7/site-packages (from google-api-core==1.20.*->feast) (2020.1)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in /opt/conda/lib/python3.7/site-packages (from google-auth<2.0dev,>=1.14.0->feast) (4.1.1)
Requirement already satisfied: rsa<5,>=3.1.4; python_version >= "3" in /opt/conda/lib/python3.7/site-packages (from google-auth<2.0dev,>=1.14.0->feast) (4.6)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /opt/conda/lib/python3.7/site-packages (from google-auth<2.0dev,>=1.14.0->feast) (0.2.7)
Requirement already satisfied: python-dateutil>=2.6.1 in /opt/conda/lib/python3.7/site-packages (from pandas~=1.0.0->feast) (2.8.1)
Requirement already satisfied: soupsieve>1.2 in /opt/conda/lib/python3.7/site-packages (from beautifulsoup4->google->feast) (2.0.1)
Requirement already satisfied: chardet<4,>=3.0.2 in /opt/conda/lib/python3.7/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core==1.20.*->feast) (3.0.4)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/lib/python3.7/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core==1.20.*->feast) (1.25.10)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.7/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core==1.20.*->feast) (2020.6.20)
Requirement already satisfied: idna<3,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core==1.20.*->feast) (2.10)
Requirement already satisfied: pyasn1>=0.1.3 in /opt/conda/lib/python3.7/site-packages (from rsa<5,>=3.1.4; python_version >= "3"->google-auth<2.0dev,>=1.14.0->feast) (0.4.8)
Building wheels for collected packages: PyYAML, pandavro
Building wheel for PyYAML (setup.py) ... done
Created wheel for PyYAML: filename=PyYAML-5.1.2-cp37-cp37m-linux_x86_64.whl size=44104 sha256=f3ae30f6e563d48459717b27bfca01eedcda098554cceaa7816e747de448fc76
Stored in directory: /home/jupyter/.cache/pip/wheels/23/b9/73/57aaccb6957d94ed63f474b51a9f7f992c5eff4635052c0557
Building wheel for pandavro (setup.py) ... done
Created wheel for pandavro: filename=pandavro-1.5.2-py3-none-any.whl size=2954 sha256=d02a6dea6e304befccbc4432336798a567f3de4ef90cf593906c37701fb9d3c5
Stored in directory: /home/jupyter/.cache/pip/wheels/33/3f/96/9f1b46a9f7f6043ff4741b1aa1a7b249ba33be4dc1d08843e4
Successfully built PyYAML pandavro
ERROR: tfx 0.22.1 has requirement pyarrow<0.17,>=0.16, but you'll have pyarrow 0.15.1 which is incompatible.
ERROR: tfx-bsl 0.22.1 has requirement pyarrow<0.17,>=0.16.0, but you'll have pyarrow 0.15.1 which is incompatible.
ERROR: tensorflow-model-analysis 0.22.2 has requirement pyarrow<0.17,>=0.16, but you'll have pyarrow 0.15.1 which is incompatible.
ERROR: tensorflow-model-analysis 0.22.2 has requirement scipy==1.4.1; python_version >= "3", but you'll have scipy 1.5.1 which is incompatible.
ERROR: tensorflow-data-validation 0.22.2 has requirement joblib<0.15,>=0.12, but you'll have joblib 0.16.0 which is incompatible.
ERROR: tensorflow-data-validation 0.22.2 has requirement pyarrow<0.17,>=0.16, but you'll have pyarrow 0.15.1 which is incompatible.
ERROR: google-cloud-translate 2.0.1 has requirement google-cloud-core<2.0dev,>=1.1.0, but you'll have google-cloud-core 1.0.3 which is incompatible.
ERROR: google-cloud-logging 1.15.0 has requirement google-cloud-core<2.0dev,>=1.1.0, but you'll have google-cloud-core 1.0.3 which is incompatible.
ERROR: apache-beam 2.22.0 has requirement httplib2<0.18.0,>=0.8, but you'll have httplib2 0.18.1 which is incompatible.
ERROR: apache-beam 2.22.0 has requirement mock<3.0.0,>=1.0.1, but you'll have mock 4.0.2 which is incompatible.
Installing collected packages: google-api-core, google-cloud-core, google-resumable-media, google-cloud-bigquery, google, kafka-python, google-cloud-storage, google-cloud-bigquery-storage, confluent-kafka, PyYAML, fastavro, pandavro, pyarrow, feast
Attempting uninstall: google-api-core
Found existing installation: google-api-core 1.22.0
Uninstalling google-api-core-1.22.0:
Successfully uninstalled google-api-core-1.22.0
Attempting uninstall: google-cloud-core
Found existing installation: google-cloud-core 1.3.0
Uninstalling google-cloud-core-1.3.0:
Successfully uninstalled google-cloud-core-1.3.0
Attempting uninstall: google-resumable-media
Found existing installation: google-resumable-media 0.5.1
Uninstalling google-resumable-media-0.5.1:
Successfully uninstalled google-resumable-media-0.5.1
Attempting uninstall: google-cloud-bigquery
Found existing installation: google-cloud-bigquery 1.26.0
Uninstalling google-cloud-bigquery-1.26.0:
Successfully uninstalled google-cloud-bigquery-1.26.0
Attempting uninstall: google-cloud-storage
Found existing installation: google-cloud-storage 1.29.0
Uninstalling google-cloud-storage-1.29.0:
Successfully uninstalled google-cloud-storage-1.29.0
Attempting uninstall: PyYAML
Found existing installation: PyYAML 5.3.1
ERROR: Cannot uninstall 'PyYAML'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
The reason is simple - the range of supported libraries in: https://github.com/feast-dev/feast/blob/v0.6-branch/sdk/python/setup.py is too small. This contradicts the best practices of maintaining dependencies described in this article (Indicate transitive dependencies using install_requires): https://medium.com/knerd/best-practices-for-python-dependency-management-cc8d1913db82
Listing only direct dependencies and permitting the largest possible range of dependencies will simplify the feast client installation process a lot.
The text was updated successfully, but these errors were encountered:
Thanks for raising this @abfreiheit, I think this is a known issue. It's also becoming a problem when using the statistics API where the use of Google libraries can cause issues that aren't always noticed. Let's keep this issue open until we have relaxed the dependencies.
Python SDK client library is not compatible with the most popular DS installations in GCP.
I was using
!pip install feast
together with standard Jupyter notebook installations for TensorFlow 1.15.0 and 2.2.0.In both cases the installations end up with the following errors:
The reason is simple - the range of supported libraries in:
https://github.com/feast-dev/feast/blob/v0.6-branch/sdk/python/setup.py
is too small. This contradicts the best practices of maintaining dependencies described in this article (Indicate transitive dependencies using install_requires):https://medium.com/knerd/best-practices-for-python-dependency-management-cc8d1913db82
Listing only direct dependencies and permitting the largest possible range of dependencies will simplify the feast client installation process a lot.
The text was updated successfully, but these errors were encountered: