Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

install-poetry.py: Use for 1.1.9 and up #14

Closed
wants to merge 2 commits into from

Conversation

tony
Copy link
Contributor

@tony tony commented Nov 25, 2021

Part 1 of 2 fix for #10 / python-poetry/poetry#3345

Error:
ModuleNotFoundError: No module named 'cleo', Versions: Python 3.10 and poetry 1.1.9 - 1.1.11 during poetry invocation
'Link' object has no attribute 'is_absolute' poetry 1.1.7 - 1.1.8 during poetry install of pycparser==0.20.0

See also:

@tony tony requested a review from a team as a code owner November 25, 2021 15:43
tony added a commit to tony/asdf-poetry that referenced this pull request Nov 25, 2021
This operates independently of poetry releases so it can pull
bugfixes merged outside of poetry's own release cadence.

See also:
- https://github.com/python-poetry/install.python-poetry.org
- python-poetry/poetry#3345 (comment)

In conjunction with asdf-community#14 this gets python 3.10 working with poetry
1.1.11 where ModuleNotFoundError: No module named 'cleo' was raised.
@tony tony changed the title install-poetry.py: Use for 1.1.7 and up install-poetry.py: Use for 1.1.9 and up Nov 25, 2021
Copy link
Contributor

@Kurt-von-Laven Kurt-von-Laven left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not affiliated with asdf, asdf-poetry, or Poetry, but this looks good to me.

tony added a commit to tony/asdf-poetry that referenced this pull request Nov 25, 2021
This operates independently of poetry releases so it can pull
bugfixes merged outside of poetry's own release cadence.

See also:
- https://github.com/python-poetry/install.python-poetry.org
- python-poetry/poetry#3345 (comment)

In conjunction with asdf-community#14 this gets python 3.10 working with poetry
1.1.11 where ModuleNotFoundError: No module named 'cleo' was raised.
@tony
Copy link
Contributor Author

tony commented Nov 26, 2021

@crflynn Are either / both of these (including #15) worth merging?

#14 - expands install-poetry.py from poetry 2.0 -> 1.1.9
#15 - uses new official install location for install-poetry.py

I'm using both of them in a fork and it's working well for me.

@crflynn
Copy link
Member

crflynn commented Nov 26, 2021

I'm testing some changes locally starting with this PR.

ModuleNotFoundError: No module named 'cleo', Versions: Python 3.10 and poetry 1.1.9 - 1.1.11

I'm able to repro this, and while it does fix the cleo error, it also expands issue #10 for ^1.1.9 on python <3.10.

'Link' object has no attribute 'is_absolute' poetry 1.1.7 - 1.1.8

Not able to repro this. Is there a specific python version I should be using/installing with?

@tony
Copy link
Contributor Author

tony commented Nov 27, 2021

@crflynn Taking a look

@tony
Copy link
Contributor Author

tony commented Nov 27, 2021

@crflynn

'Link' object has no attribute 'is_absolute' poetry 1.1.7 - 1.1.8
Not able to repro this. Is there a specific python version I should be using/installing with?

Happened with with pyparser==2.20

I can also set it to 1.1.7 in this PR if it was just me

❯ poetry install
Creating virtualenv myproject in /home/t/work/project/myproject/.venv
Installing dependencies from lock file

Package operations: 229 installs, 0 updates, 0 removals

  • Installing pycparser (2.20): Failed

  AttributeError

  'Link' object has no attribute 'is_absolute'

  at ~/.config/.asdf/installs/poetry/1.1.7/venv/lib/python3.7/site-packages/poetry/core/packages/file_dependency.py:33 in __init__
       29│         self._path = path
       30│         self._base = base or Path.cwd()
       31│         self._full_path = path
       32│
    →  33│         if not self._path.is_absolute():
       34│             try:
       35│                 self._full_path = self._base.joinpath(self._path).resolve()
       36│             except FileNotFoundError:
       37│                 raise ValueError("Directory {} does not exist".format(self._path))

You can reproduce with this pyproject.toml:

[tool.poetry]
name = "my project"
version = "1.0.0"
description = "None"
authors = ["Anonymous"]

[tool.poetry.dependencies]
python = "==3.*.*,>=3.7.0"
pycparser = "==2.20"

The bug happens here: python-poetry/poetry#4409 (comment)

tony added a commit to tony/asdf-poetry that referenced this pull request Nov 27, 2021
This operates independently of poetry releases so it can pull
bugfixes merged outside of poetry's own release cadence.

See also:
- https://github.com/python-poetry/install.python-poetry.org
- python-poetry/poetry#3345 (comment)

In conjunction with asdf-community#14 this gets python 3.10 working with poetry
1.1.11 where ModuleNotFoundError: No module named 'cleo' was raised.
tony added a commit to tony/asdf-poetry that referenced this pull request Nov 27, 2021
This operates independently of poetry releases so it can pull
bugfixes merged outside of poetry's own release cadence.

See also:
- https://github.com/python-poetry/install.python-poetry.org
- python-poetry/poetry#3345 (comment)

In conjunction with asdf-community#14 this gets python 3.10 working with poetry
1.1.11 where ModuleNotFoundError: No module named 'cleo' was raised.
@tony
Copy link
Contributor Author

tony commented Nov 27, 2021

@crflynn Rebased with #15

Any preference on whether to stick with 1.1.9 or go to 1.1.7?

@crflynn
Copy link
Member

crflynn commented Nov 27, 2021

I'm reluctant to merge this because:

while it does fix the cleo error, it also expands issue #10 for ^1.1.9 on python <3.10.

and I would rather not further break the functionality of this plugin on what I would guess is currently a more common situation for users than ^1.1.9 on python=3.10.

You point out some discussion here #15 (comment) which seems to be related to #10 and indicates an incoming fix in poetry 1.2. Since we have 1.2+ on the new installer perhaps it's better wait for that release and inform users that in order to use python 3.10 they must be using at least poetry 1.2.

@tony
Copy link
Contributor Author

tony commented Nov 27, 2021

The plugin is broke 1.1.9 - 1.1.11 with python 3.10 currently, this is a patch to add support for those systems.

Since we have 1.2+ on the new installer perhaps it's better wait for that release and inform users that in order to use python 3.10 they must be using at least poetry 1.2.

Can you rephrase? Is it a plugins place to tell the user / developers what python version and poetry version to use?

1.1.9 - 1.1.11 is broke for me without this patch. I am not sure how this would further break functionality.

1.1.9 - As-is:
asdf install poetry 1.1.9
poRetrieving Poetry metadata
et
This installer is deprecated. Poetry versions installed using this script will not be able to use 'self update' command to upgrade to 1.2.0a1 or later.
# Welcome to Poetry!

This will download and install the latest version of Poetry,
a dependency and package manager for Python.

It will add the `poetry` command to Poetry's bin directory, located at:

$HOME/.config/.asdf/installs/poetry/1.1.9/bin

This path needs to be in your `PATH` environment variable,
but will not be added automatically.

You can uninstall at any time by executing this script with the --uninstall option,
and these changes will be reverted.

Installing version: 1.1.9
r  - Downloading poetry-1.1.9-linux.tar.gz (62.07MB)

Poetry (1.1.9) is installed now. Great!

To get started you need Poetry's bin directory ($HOME/.config/.asdf/installs/poetry/1.1.9/bin) in your `PATH`
environment variable.

To configure your current shell run `source $HOME/.config/.asdf/installs/poetry/1.1.9/env`

project on  python-3.10 [!] ❯ poetry
Traceback (most recent call last):
  File "~/.config/.asdf/installs/poetry/1.1.9/bin/poetry", line 17, in <module>
    from poetry.console import main
  File "~/.config/.asdf/installs/poetry/1.1.9/lib/poetry/console/__init__.py", line 1, in <module>
    from .application import Application
  File "~/.config/.asdf/installs/poetry/1.1.9/lib/poetry/console/application.py", line 3, in <module>
    from cleo import Application as BaseApplication
ModuleNotFoundError: No module named 'cleo'
1.1.9 - WIth PR:
asdf install poetry 1.1.9
Retrieving Poetry metadata

# Welcome to Poetry!

This will download and install the latest version of Poetry,
a dependency and package manager for Python.

It will add the `poetry` command to Poetry's bin directory, located at:

~/.config/.asdf/installs/poetry/1.1.9/bin

You can uninstall at any time by executing this script with the --uninstall option,
and these changes will be reverted.

Installing Poetry (1.1.9): Done

Poetry (1.1.9) is installed now. Great!

To get started you need Poetry's bin directory (~/.config/.asdf/installs/poetry/1.1.9/bin) in your `PATH`
environment variable.

Add `export PATH="~/.config/.asdf/installs/poetry/1.1.9/bin:$PATH"` to your shell configuration file.

Alternatively, you can call Poetry explicitly with `~/.config/.asdf/installs/poetry/1.1.9/bin/poetry`.

You can test that everything is set up by executing:

`poetry --version`

project  on  python-3.10 [!] ❯ poetry
Poetry version 1.1.9

USAGE
  poetry [-h] [-q] [-v [<...>]] [-V] [--ansi] [--no-ansi] [-n] <command> [<arg1>] ... [<argN>]

ARGUMENTS
  <command>              The command to execute
  <arg>                  The arguments of the command

GLOBAL OPTIONS
  -h (--help)            Display this help message
  -q (--quiet)           Do not output any message
  -v (--verbose)         Increase the verbosity of messages: "-v" for normal output, "-vv" for more verbose output and "-vvv" for debug
  -V (--version)         Display this application version
  --ansi                 Force ANSI output
  --no-ansi              Disable ANSI output
  -n (--no-interaction)  Do not ask any interactive question

AVAILABLE COMMANDS
  about                  Shows information about Poetry.
  add                    Adds a new dependency to pyproject.toml.
  build                  Builds a package, as a tarball and a wheel by default.
  cache                  Interact with Poetry's cache
  check                  Checks the validity of the pyproject.toml file.
  config                 Manages configuration settings.
  debug                  Debug various elements of Poetry.
  env                    Interact with Poetry's project environments.
  export                 Exports the lock file to alternative formats.
  help                   Display the manual of a command
  init                   Creates a basic pyproject.toml file in the current directory.
  install                Installs the project dependencies.
  lock                   Locks the project dependencies.
  new                    Creates a new Python project at <path>.
  publish                Publishes a package to a remote repository.
  remove                 Removes a package from the project dependencies.
  run                    Runs a command in the appropriate environment.
  search                 Searches for packages on remote repositories.
  self                   Interact with Poetry directly.
  shell                  Spawns a shell within the virtual environment.
  show                   Shows information about packages.
  update                 Update the dependencies as according to the pyproject.toml file.
  version                Shows the version of the project or bumps it when a valid bump rule is provided.
1.1.11 - Without this patch (as-is):
asdf unisntall 1.1.11
asdf install 1.1.11
Traceback (most recent call last):
  File "~/.config/.asdf/installs/poetry/1.1.11/bin/poetry", line 17, in <module>
    from poetry.console import main
  File "~/.config/.asdf/installs/poetry/1.1.11/lib/poetry/console/__init__.py", line 1, in <module>
    from .application import Application
  File "~/.config/.asdf/installs/poetry/1.1.11/lib/poetry/console/application.py", line 3, in <module>
    from cleo import Application as BaseApplication
ModuleNotFoundError: No module named 'cleo'
1.11 - With this patch
asdf uninstall poetry 1.1.11
asdf install poetry 1.1.9
Poetry version 1.1.11

USAGE
  poetry [-h] [-q] [-v [<...>]] [-V] [--ansi] [--no-ansi] [-n] <command> [<arg1>] ... [<argN>]

ARGUMENTS
  <command>              The command to execute
  <arg>                  The arguments of the command

GLOBAL OPTIONS
  -h (--help)            Display this help message
  -q (--quiet)           Do not output any message
  -v (--verbose)         Increase the verbosity of messages: "-v" for normal output, "-vv" for more verbose output and "-vvv" for debug
  -V (--version)         Display this application version
  --ansi                 Force ANSI output
  --no-ansi              Disable ANSI output
  -n (--no-interaction)  Do not ask any interactive question

AVAILABLE COMMANDS
  about                  Shows information about Poetry.
  add                    Adds a new dependency to pyproject.toml.
  build                  Builds a package, as a tarball and a wheel by default.
  cache                  Interact with Poetry's cache
  check                  Checks the validity of the pyproject.toml file.
  config                 Manages configuration settings.
  debug                  Debug various elements of Poetry.
  env                    Interact with Poetry's project environments.
  export                 Exports the lock file to alternative formats.
  help                   Display the manual of a command
  init                   Creates a basic pyproject.toml file in the current directory.
  install                Installs the project dependencies.
  lock                   Locks the project dependencies.
  new                    Creates a new Python project at <path>.
  publish                Publishes a package to a remote repository.
  remove                 Removes a package from the project dependencies.
  run                    Runs a command in the appropriate environment.
  search                 Searches for packages on remote repositories.
  self                   Interact with Poetry directly.
  shell                  Spawns a shell within the virtual environment.
  show                   Shows information about packages.
  update                 Update the dependencies as according to the pyproject.toml file.
  version                Shows the version of the project or bumps it when a valid bump rule is provided.

@crflynn
Copy link
Member

crflynn commented Nov 27, 2021

I think you are misunderstanding that issue #10 has nothing to do with the cleo bug. Rather it has to do with poetry not using the python version specified by asdf when creating venvs if it was installed with the new installer, essentially breaking the expected behavior of asdf with python in this particular case. I understand that this fixes the cleo bug with 3.10. However it makes #10 worse because now the issue applies to ^1.1.9 rather than just ^1.2.0.

@tony
Copy link
Contributor Author

tony commented Nov 27, 2021

My mistake, I misunderstand / don't understand the #10 issue.

@Kurt-von-Laven you were on the threads relating to the other bug (with virtualenv versions) at python-poetry/poetry#1888 (comment), python-poetry/poetry#4199 (comment).

Do you have any preferences or ideas? If you try this PR / approach how does it work for you in terms of the virtualenv + python interpreter issue?

@crflynn
Copy link
Member

crflynn commented Nov 27, 2021

To summarize briefly, when poetry is installed with the new installer: if you set your asdf python version to A, but poetry was installed using version B, poetry will disregard your asdf python version (A) and only use version B when creating virtualenvs. @Kurt-von-Laven 's workaround requires manually overriding poetry env config per project.

@Kurt-von-Laven
Copy link
Contributor

Kurt-von-Laven commented Nov 27, 2021

I think you are misunderstanding that issue #10 has nothing to do with the cleo bug. Rather it has to do with poetry not using the python version specified by asdf when creating venvs if it was installed with the new installer, essentially breaking the expected behavior of asdf with python in this particular case. I understand that this fixes the cleo bug with 3.10. However it makes #10 worse because now the issue applies to ^1.1.9 rather than just ^1.2.0.

In case this is helpful context, I encountered the cleo bug when attempting to upgrade to Poetry 3.10 and was advised to reinstall Poetry using the version of install-poetry.py on master (after uninstalling all versions of Poetry), which resolved the issue for me.

To summarize briefly, when poetry is installed with the new installer: if you set your asdf python version to A, but poetry was installed using version B, poetry will disregard your asdf python version (A) and only use version B when creating virtualenvs. @Kurt-von-Laven 's workaround requires manually overriding poetry env config per project.

This matches my understanding. I can't claim credit for the workaround, and I've lost track of who first posted it at this point, but I have used it enough times that I'm confident it addresses the issue of Poetry installed via install-poetry.py disregarding your asdf Python version when creating virtualenvs. Some nuances are that the version of Poetry presently on master does not exhibit this bug; you may have to uninstall all versions of Poetry installed on your computer by any means whatsoever in order to experience said fix according to a maintainer of Poetry; that install-poetry.py isn't tied to a Poetry release, although it first appeared during Poetry 1.2 development; and the version of install-poetry.py at the new install URL is functionally equivalent (although not byte-for-byte identical) to the version presently on master. As for what to do about all of this, I stand by my initial recommendation:

The maintainers of Poetry recently pointed out (c.f., #3345) that they have since resolved this issue on master, but the fix (#4433) has not yet been released. Look for it on poetry > 1.2.0a2 once that is released. At that time, I believe this plugin will be able to stop using the deprecated get-poetry.py even for poetry < 1.2. ~ #10

I would either simply keep using the poetry env use "$ASDF_DIR/shims/python or poetry env use "$(asdf which python)" workaround or remain on Poetry 1.1 (which, after all, is still the latest stable release of Poetry) in the meantime. The simplest, most helpful thing this plugin could do right now IMO is explain the needed workaround when installing the versions of Poetry that require it. If a more immediate patch is desired, the comment I linked above shows how to use the version of install-poetry.py on master (or, implicitly, how to pin a specific commit). The latest version of the installer could be copied directly into this project since they are both MIT licensed. At this time, I couldn't say whether these more aggressive solutions would solve more problems than they create, and I am not advocating for them. If anyone needs a fix out in a hurry, probably the most helpful thing they could do is offer their services towards getting the next Poetry release candidate cut.

@tony
Copy link
Contributor Author

tony commented Nov 27, 2021

@Kurt-von-Laven If this PR was scoped to python 3.10 (and beyond?), would that be sufficient? (since python 3.10 is DOA on installation with get-poetry.py method)

Assuming the detection of python version wasn't reliable for some reason (python shims for instance) where do you stand on this PR as is?

@Kurt-von-Laven
Copy link
Contributor

@tony, that is a good point; thanks for bringing that up. When I approved this PR I was under the mistaken impression that all of the needed patches were to install-poetry.py itself, but unfortunately that is not the case, so I would continue to use get-poetry.py on Poetry 1.1 for now. It would be nice to issue a warning that Poetry 1.2 is currently required for Python 3.10, however, to spare a lot of people some debugging/hassle with minimal effort. Please correct me if you are seeing something I am missing, but I don't see how this plugin could determine the pertinent Python version at Poetry install time in a meaningful way. The user might manage Python with asdf or pyenv or some other tool. If they use asdf for Python, we don't know which .tool-versions files they have on their machine at Poetry install time. Theoretically, they might have the entire cross-product of all Python and Poetry versions represented across a huge directory hierarchy, and the Python version that happens to be active when Poetry is installed doesn't really have anything to do with how the tools will be used in any deep sense.

Copy link
Contributor

@Kurt-von-Laven Kurt-von-Laven left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my comment immediately above.

@tony
Copy link
Contributor Author

tony commented Nov 27, 2021

@Kurt-von-Laven It would be helpful to have something more succinct, if possible.

It would be nice to issue a warning that Poetry 1.2 is currently required for Python 3.10

I'm using it right now on multiple systems with 10+ large projects with different python versions. poetry 1.1.11 (at least) is compatible with 3.10

the plugin is crashing outright and this PR fixes it. It adds 3.10 support for Python 1.1.9 - 1.1.11.

@tony
Copy link
Contributor Author

tony commented Nov 27, 2021

@Kurt-von-Laven The reason why is right now what we have threads that I'm having difficulty wrapping my head around. Would @crflynn's summarization at #14 (comment) describe what you're experiencing?

@Kurt-von-Laven
Copy link
Contributor

@Kurt-von-Laven It would be helpful to have something more succinct, if possible.

Until another version of Poetry 1.2 is released, I would keep the asdf Poetry plugin as is but add some warnings about needing Poetry 1.2 for Python 3.10 and using poetry env use "$(asdf which python)" if you manage Python with asdf to ensure the correct version of Python is used in your virtualenv.

I'm using it right now on multiple systems with 10+ large projects with different python versions. poetry 1.1.11 (at least) is compatible with 3.10.

Right, so my understanding is that it's really any version of Poetry (I think) installed via the install-poetry.py from the new install URL that works with Python 3.10. Hence, because you patched this plugin to use install-poetry.py, you are able to use Python 3.10 with at least some versions of Poetry 1.1.

@tony
Copy link
Contributor Author

tony commented Nov 27, 2021

@Kurt-von-Laven Thank you!

Until another version of Poetry 1.2 is released, I would keep the asdf Poetry plugin as is but add some warnings about needing Poetry 1.2 for Python 3.10 and using poetry env use "$(asdf which python)" if you manage Python with asdf to ensure the correct version of Python is used in your virtualenv.

So users with poetry 1.1 be able to use python 3.10?

As another option: perhaps a --classic flag, or environmental variable to point to the installer?

@Kurt-von-Laven
Copy link
Contributor

@Kurt-von-Laven Thank you!

Until another version of Poetry 1.2 is released, I would keep the asdf Poetry plugin as is but add some warnings about needing Poetry 1.2 for Python 3.10 and using poetry env use "$(asdf which python)" if you manage Python with asdf to ensure the correct version of Python is used in your virtualenv.

So users with poetry 1.1 be able to use python 3.10?

Probably only those that read this thread for now unfortunately? I think the warning should be clear enough about the fact that those users can upgrade to Poetry 1.2 or downgrade to Python 3.9 though. I agree that it's a regrettable situation. I suppose part of the beauty of asdf is that if someone really must use Python 3.10 with Poetry 1.1, they can use your (or their own) fork of the asdf Poetry plugin instead.

As another option: perhaps a --classic flag, or environmental variable to point to the installer?

I am quite new to the asdf ecosystem, but I'm not aware of other plugins that take special options like this, and to my mind one of the principal selling points of asdf is that it exposes the same interface to version management of so many different tools. It's probably better that someone with more experience with asdf weigh in though.

@tony
Copy link
Contributor Author

tony commented Nov 28, 2021

Thank you!

I am quite new to the asdf ecosystem, but I'm not aware of other plugins that take special options like this, and to my mind one of the principal selling points of asdf is that it exposes the same interface to version management of so many different tools. It's probably better that someone with more experience with asdf weigh in though.

I went with something asdf friendly: #16

It uses an optional, environmental variable: ASDF_POETRY_INSTALL_URL

For a common plugin example: asdf-python's environmental variable e.g. ASDF_PYTHON_PATCH_URL and asdf-nodejs's NODEJS_CHECK_SIGNATURES

@Kurt-von-Laven
Copy link
Contributor

That is pretty compelling to me at least.

@crflynn crflynn closed this Nov 28, 2021
@tony
Copy link
Contributor Author

tony commented Nov 28, 2021

This was superseded by #16, ASDF_POETRY_INSTALL_URL: Optionally override installer URL, merged in 29ca741

@tony tony deleted the tn-install-poetry-1.1.7 branch November 28, 2021 21:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants