can't install tensorflow==2.0.0rc0 with poetry (python3.6.5) #1330

zachmoshe · 2019-08-26T14:56:51Z

I am on the latest Poetry version.
- (I'm actually on 1.0.0b1)
I have searched the issues of this repo and believe that this is not a duplicate.
If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

OS version and name: MacOS 10.14.5
Poetry version: 1.0.0b1
Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/zachmoshe/36ec3cc128545aa8ab49a2d4dcd9cd52

Issue

Open a new project (poetry new .)
run poetry add tensorflow==2.0.0rc0

Fails because it can't install functools32 (for python2 only).
debug logs - https://gist.github.com/zachmoshe/8bd72026dd727380f4b88dc4ef55ff4f

Installing previous versions works and installing rc0 with pip also works.
I understand functools32 is a new dependency and that somehow we try to install a version which only work with python2. I just can't understand why do we try this specific version.

Any solution or a workaround?

Thanks!

The text was updated successfully, but these errors were encountered:

Caligatio · 2019-08-26T17:18:50Z

It appears that tensorflow 2.0.0-rc0's setup.py dynamically changes its requirements depending on the environment by executing code. My guess is that Poetry does not handle this case well for some reason as it appears ALL of the Python2-linked dependencies are being grabbed.

I've never really played with the ast module and I'm only taking guesses here...

zachmoshe · 2019-08-26T18:21:10Z

Makes sense, but I'm not familiar with the internals as well.. definitely not a standard setup.py.
Can anyone suggest at least a workaround until a real solution is deployed?

On hold due to python-poetry/poetry#1330

mike239x · 2019-09-02T20:42:54Z

Yeap, I am yet another one with the same issue.
@zachmoshe just use pip for now?
I also was thinking - one could get the exact versions one needs from the pip-install (or from $ poetry debug:resolve tensorflow==2.0.0rc0), and then hard code them into poetry.lock file and run $ poetry install. I sadly do not know how exactly to operate directly on the lock-file though...

aflc · 2019-09-19T09:15:07Z

I found some reasons that functools32 dependency found even if Python3 projects you have:

poetry find a dependencies from PyPI's "requires_dist", and tensorflow 2.0.0rc0 or rc1 has functools32 in it:

https://pypi.org/ pypi/tensorflow/2.0.0rc0/json:

"requires_dist": [
      "absl-py (>=0.7.0)",
      "astor (>=0.6.0)",
      "gast (>=0.2.0)",
      "google-pasta (>=0.1.6)",
      "keras-applications (>=1.0.8)",
      "keras-preprocessing (>=1.0.5)",
      "numpy (<2.0,>=1.16.0)",
      "opt-einsum (>=2.3.2)",
      "six (>=1.10.0)",
      "protobuf (>=3.6.1)",
      "tb-nightly (<1.15.0a20190807,>=1.15.0a20190806)",
      "tf-estimator-nightly (<1.14.0.dev2019080602,>=1.14.0.dev2019080601)",
      "termcolor (>=1.1.0)",
      "wrapt (>=1.11.1)",
      "grpcio (>=1.8.6)",
      "wheel",
      "mock (>=2.0.0)",
      "functools32 (>=3.2.3)",
      "backports.weakref (>=1.0rc1); python_version < \"3.4\"",
      "enum34 (>=1.1.6); python_version < \"3.4\""
    ],

ref: https://github.com/sdispater/poetry/blob/cb8dc0d94e68042f120441b4e30d89c92598b40e/poetry/repositories/pypi_repository.py#L286

So at first, we need to modify tensorflow package metadata. I think it's a tensorflow team matter.

Even if requires_dist is null, poetry tries to get dependencies from universal bdist_wheel. and tensorflow doesn't have a universal wheel, it chooses a first wheel to inspect (it's a cp27 wheel).

ref: https://github.com/sdispater/poetry/blob/cb8dc0d94e68042f120441b4e30d89c92598b40e/poetry/repositories/pypi_repository.py#L350

So poetry need to be fixed these logics, IMO, it should respect Python version of venv.

gorjanradevski · 2019-10-02T15:27:19Z

Is there a particular workaround at the moment so that TF 2.0 can be installed?

sdispater · 2019-10-06T16:31:31Z

Unfortunately, this is not on Poetry's end. If you take a look at https://pypi.org/pypi/tensorflow/json in the requires_dist section of the info property, functools32 is specified as a dependency without any environment marker so Poetry rightly assumes that it's an unconditional dependency.

So, this issue must be fixed upstream by the tensorflow maintainers

aflc · 2019-10-07T10:33:05Z

So, this issue must be fixed upstream by the tensorflow maintainers

tensorflow/tensorflow#32758
It may fixes the issue, but need to wait until 2.1 release.

ColonelThirtyTwo · 2020-01-05T01:26:51Z

so Poetry rightly assumes

Uh, if pip installs it fine, but poetry screws up, is it really "rightly"?

grisaitis · 2020-01-06T18:00:23Z

poetry add tensorflow@^2.1.0rc0

works for me

tensorflow 2.1.0 should be released soon.

Magnati · 2020-01-07T10:22:19Z

In addition to @grisaitis answer
I had to fulfill some other prerequisites:

poetry > 0.12 (used 1.0.0)
python < 3.8 (tensorflow does support only up to 3.7, yeah a no brainer :P)

then his command poetry add tensorflow@^2.1.0rc0 worked for me too.

mjlbach · 2020-02-02T19:20:29Z

I'm also running into problems due to this this: It seems to be failing to pull in dependencies matching *

❯ poetry add tf-nightly@2.2.0-alpha.20200202

Updating dependencies
Resolving dependencies... (0.1s)

[SolverProblemError]
Because tf-nightly (2.2.0.dev20200202) depends on tf-estimator-nightly (*) which doesn't match any versions, tf-nightly is forbidden.
So, because model_repo depends on tf-nightly (2.2.0-alpha.20200202), version solving failed.

If you look in:

at https://pypi.org/pypi/tf-nightly/json in the requires_dist section of the info property you can see that tf-estimator-nightly has no specified version.

One workaround is to explicitly install tf-estimator-nightly, but it would be nice if poetry could patch around this for now...

mohelm · 2020-03-26T19:08:03Z

@mjlbach : i could not get this work. is the succession of commands you are proposing

poetry add tf-estimator-nightly
poetry add tf-nightly

The first one works, the second one does not. I get this error:

[EnvCommandError]
Command ['/Users/moritz.helm/Projects/recsys/oyster/.venv/bin/pip', 'install', '--no-deps', 'tf-nightly==2.2.0.dev20200202'] errored with the following return code 1, and output:
ERROR: Could not find a version that satisfies the requirement tf-nightly==2.2.0.dev20200202 (from versions: none)
ERROR: No matching distribution found for tf-nightly==2.2.0.dev20200202

mjlbach · 2020-03-26T20:16:59Z

@mohelm I don't think the release I used in my specific command is available anymore. Using the commands

poetry add tf-estimator-nightly
poetry add tf-nightly

I was able to install tf-nightly on a fresh vm.

mohelm · 2020-03-26T20:20:08Z

@mjlbach : Many thanks! Do you remember the Python version? I am actually trying this with Python 3.8.

mjlbach · 2020-03-26T20:21:29Z

@mohelm Ah! That's your problem, tensorflow is only packaged for 3.6/3.7 currently (I believe)

mohelm · 2020-03-26T20:24:26Z

Hm, these nightlies should work with 3.8 I believe. Michael Lingelbach <notifications@github.com> schrieb am Do., 26. März 2020, 21:21:

…

@mohelm <https://github.com/mohelm> Ah! That's your problem, tensorflow is only packaged for 3.6/3.7 currently (I believe) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1330 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEJC4NK7BXIEI35QTZDTIV3RJO2NRANCNFSM4IPQRYUA> .

mjlbach · 2020-03-26T20:46:41Z

Hm, these nightlies should work with 3.8 I believe. Michael Lingelbach notifications@github.com schrieb am Do., 26. März 2020, 21:21:
…
@mohelm https://github.com/mohelm Ah! That's your problem, tensorflow is only packaged for 3.6/3.7 currently (I believe) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1330 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEJC4NK7BXIEI35QTZDTIV3RJO2NRANCNFSM4IPQRYUA .

You're correct, I just managed to get tf-nightly installed on 3.8... Not sure why you are having the issue.

mohelm · 2020-03-26T20:50:21Z

Interesting. I tried on Mac, but I also saw just now the caveats (go to caveats section) here https://github.com/tensorflow/tensorflow/releases Which I guess do apply?? May I ask you on which operating system you succeeded? Michael Lingelbach <notifications@github.com> schrieb am Do., 26. März 2020, 21:46:

…

Hm, these nightlies should work with 3.8 I believe. Michael Lingelbach ***@***.*** schrieb am Do., 26. März 2020, 21:21: … <#m_-403593830552407337_m_-732227986706241720_> @mohelm <https://github.com/mohelm> https://github.com/mohelm Ah! That's your problem, tensorflow is only packaged for 3.6/3.7 currently (I believe) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1330 (comment) <#1330 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEJC4NK7BXIEI35QTZDTIV3RJO2NRANCNFSM4IPQRYUA . You're correct, I just managed to get tf-nightly installed on 3.8... Not sure why you are having the issue. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1330 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEJC4NOBQNBYRXXLXIC7QMTRJO5MBANCNFSM4IPQRYUA> .

mjlbach · 2020-03-26T20:51:35Z

@mohelm nixOS, works on ubuntu as well

mohelm · 2020-03-26T20:52:43Z

Awesome will try tomorrow thanks. Michael Lingelbach <notifications@github.com> schrieb am Do., 26. März 2020, 21:51:

…

@mohelm <https://github.com/mohelm> nixOS, works on ubuntu as well — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1330 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEJC4NLQ5YC7D4FRHLIDOJDRJO56PANCNFSM4IPQRYUA> .

logicpeters · 2020-04-03T21:29:31Z

This is also a problem with Tensorflow 1.15, and some python libraries require the 1.x version of Tensorflow.

kevinthenet · 2020-04-28T18:48:24Z

For anyone having similar issues, I'd recommend trying poetry shell then pip install -U pip to upgrade pip. I was having issues with poetry finding tensorflow@2.1.0, and found that the poetry virtualenv pip was on version 17, even though mine was on 20.

Give up on testing against TF 2.0 because Poetry can't install it well (see python-poetry/poetry#1330)

lmarsden · 2020-07-16T15:41:28Z

This is also a problem with Tensorflow 1.15, and some python libraries require the 1.x version of Tensorflow.

I also ran into this issue with Tensorflow 1.15.0, but Tensorflow 1.14.0 installs OK.

tsabsch · 2020-09-24T12:08:37Z

Installing Tensorflow 1.15.2 or 1.15.3 works fine for me.

augustebaum · 2022-10-11T14:05:26Z

Hi, I still have this exact problem. Are there any reliable workarounds?

neersighted · 2022-10-11T15:44:48Z

This exact problem is related to a Tensorflow release candidate having bad metadata. If you're having issues installing Tensorflow, I suggest you go to Discussions or Discord to get help, as there are many common misconceptions and pitfalls, and we don't yet have a FAQ for involved binary packages like Tensorflow and PyTorch.

notramo · 2023-05-04T13:34:41Z

This still doesn't work.

github-actions · 2024-02-29T14:06:10Z

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

Notgnoshi added a commit to Notgnoshi/research that referenced this issue Aug 27, 2019

WIP: Attempt to use poetry for dependency management

caebe82

On hold due to python-poetry/poetry#1330

Notgnoshi added a commit to Notgnoshi/research that referenced this issue Aug 28, 2019

WIP: Attempt to use poetry for dependency management

4a44bc2

On hold due to python-poetry/poetry#1330

Notgnoshi added a commit to Notgnoshi/research that referenced this issue Aug 28, 2019

WIP: Attempt to use poetry for dependency management

a36e711

On hold due to python-poetry/poetry#1330

brycedrennan mentioned this issue Aug 28, 2019

Poetry tries to install functools32 in a Python 3.x based virtual environment #1334

Closed

3 tasks

aflc mentioned this issue Sep 19, 2019

[TF 2.0.0rc0] dependency on functools32 prevents python3.7 installation tensorflow/tensorflow#31983

Closed

fakyras mentioned this issue Sep 30, 2019

Ability to override/ignore sub-dependencies #697

Closed

1 task

sdispater closed this as completed Oct 6, 2019

sdispater mentioned this issue Oct 18, 2019

Support of manylinux2010 #1373

Closed

dustinlacewell mentioned this issue Dec 29, 2019

Update Tensorflow storybro/storybro#2

Closed

chongkong mentioned this issue Feb 24, 2020

Python version incorrect by default. Add checks and add docs. tensorflow/tfx#1312

Closed

ymodak mentioned this issue Mar 16, 2020

AttributeError: module 'tensorflow' has no attribute 'compat' when importing tensorflow tensorflow/tensorflow#37525

Closed

pshved added a commit to pshved/tensorbank that referenced this issue Jul 8, 2020

Poetry doesn't play nicely with TF 2.0 so give up

13ed2a5

Give up on testing against TF 2.0 because Poetry can't install it well (see python-poetry/poetry#1330)

SuryanarayanaY mentioned this issue Nov 17, 2022

Tensorflow 2.10 cannot be installed using poetry on linux aarch64 tensorflow/tensorflow#58602

Closed

github-actions bot locked as resolved and limited conversation to collaborators Feb 29, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

can't install tensorflow==2.0.0rc0 with poetry (python3.6.5) #1330

can't install tensorflow==2.0.0rc0 with poetry (python3.6.5) #1330

zachmoshe commented Aug 26, 2019

Caligatio commented Aug 26, 2019

zachmoshe commented Aug 26, 2019

mike239x commented Sep 2, 2019

aflc commented Sep 19, 2019

gorjanradevski commented Oct 2, 2019

sdispater commented Oct 6, 2019

aflc commented Oct 7, 2019 •

edited

Loading

ColonelThirtyTwo commented Jan 5, 2020

grisaitis commented Jan 6, 2020

Magnati commented Jan 7, 2020 •

edited

Loading

mjlbach commented Feb 2, 2020 •

edited

Loading

mohelm commented Mar 26, 2020

mjlbach commented Mar 26, 2020

mohelm commented Mar 26, 2020

mjlbach commented Mar 26, 2020

mohelm commented Mar 26, 2020 via email

mjlbach commented Mar 26, 2020

mohelm commented Mar 26, 2020 via email

mjlbach commented Mar 26, 2020

mohelm commented Mar 26, 2020 via email

logicpeters commented Apr 3, 2020

kevinthenet commented Apr 28, 2020

lmarsden commented Jul 16, 2020

tsabsch commented Sep 24, 2020

augustebaum commented Oct 11, 2022

neersighted commented Oct 11, 2022

notramo commented May 4, 2023

github-actions bot commented Feb 29, 2024

can't install tensorflow==2.0.0rc0 with poetry (python3.6.5) #1330

can't install tensorflow==2.0.0rc0 with poetry (python3.6.5) #1330

Comments

zachmoshe commented Aug 26, 2019

Issue

Caligatio commented Aug 26, 2019

zachmoshe commented Aug 26, 2019

mike239x commented Sep 2, 2019

aflc commented Sep 19, 2019

gorjanradevski commented Oct 2, 2019

sdispater commented Oct 6, 2019

aflc commented Oct 7, 2019 • edited Loading

ColonelThirtyTwo commented Jan 5, 2020

grisaitis commented Jan 6, 2020

Magnati commented Jan 7, 2020 • edited Loading

mjlbach commented Feb 2, 2020 • edited Loading

mohelm commented Mar 26, 2020

mjlbach commented Mar 26, 2020

mohelm commented Mar 26, 2020

mjlbach commented Mar 26, 2020

mohelm commented Mar 26, 2020 via email

mjlbach commented Mar 26, 2020

mohelm commented Mar 26, 2020 via email

mjlbach commented Mar 26, 2020

mohelm commented Mar 26, 2020 via email

logicpeters commented Apr 3, 2020

kevinthenet commented Apr 28, 2020

lmarsden commented Jul 16, 2020

tsabsch commented Sep 24, 2020

augustebaum commented Oct 11, 2022

neersighted commented Oct 11, 2022

notramo commented May 4, 2023

github-actions bot commented Feb 29, 2024

aflc commented Oct 7, 2019 •

edited

Loading

Magnati commented Jan 7, 2020 •

edited

Loading

mjlbach commented Feb 2, 2020 •

edited

Loading