Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

problem on Windows #1

Open
jrobbins-LiveData opened this issue Nov 7, 2021 · 8 comments
Open

problem on Windows #1

jrobbins-LiveData opened this issue Nov 7, 2021 · 8 comments

Comments

@jrobbins-LiveData
Copy link

Due to an apparent limitation in Windows Credential Manager, the maximum "password" length is 1280 characters. Since poetry uses keyring, and the default keyring backend on Windows is Credential Manager, the command executed by partifact

poetry config http-basic.ld aws <token longer than 1280 characters>

fails with the (initially confusing) error

  (1783, 'CredWrite', 'The stub received bad data.')

You can simply reproduce this Windows issue in the Python REPL on Windows:

import keyring
keyring.set_password('foo', 'bar', 'a' * 1280) # works
keyring.set_password('foo', 'bar', 'a' * 1281) # fails

I know it isn't partifact's job to fix Windows, but maybe someone with expertise in keyring could help by showing how to configure an alternative keyring on Windows that both

  1. Works under poetry
  2. Is no less "safe" than Windows Credential Manager
@davidsteiner
Copy link
Member

Thanks for raising this - we don't use Windows for development so we haven't encountered this issue. I'll make sure to mention it in the README.

Unfortunately I'm not an expert in keyring. There might be a suitable third-party backend for Windows.
https://keyring.readthedocs.io/en/latest/?badge=latest#third-party-backends

If you find a way to get things working on Windows and it requires changes to partifact, I'd be more than happy to implement the changes or take contributions!

@jrobbins-LiveData
Copy link
Author

Thanks very much; I'll keep you posted!

@jrobbins-LiveData
Copy link
Author

jrobbins-LiveData commented Nov 8, 2021

@davidsteiner just for now (to make progress), I hacked my local copy of keyring's Windows.py backend to "shard" the password across multiple keys. (Perhaps I could turn that hack into a more robust feature suitable for a PR to keyring.)

That got me almost all the way to a working test, except for a puzzle concerning the url format for poetry publish.

partifact's `config.py uses this template

URL_TEMPLATE = "https://{code_artifact_domain}-{aws_account}.d.codeartifact.{aws_region}.amazonaws.com/pypi/{code_artifact_repository}/simple/"

But that doesn't work for me.

I found that this template does work

URL_TEMPLATE = "https://{code_artifact_domain}-{aws_account}.d.codeartifact.{aws_region}.amazonaws.com/pypi/{code_artifact_repository}/"

This format corresponds to the endpoint returned by the twine instructions on the CodeArtifact console, so I'm wondering if URL_TEMPLATE is correct?

aws codeartifact get-repository-endpoint --domain <domain> --domain-owner <account> --repository <repo> --format pypi --query repositoryEndpoint --output text

This SO post also states that simple/ does not belong at the end of the pyproject.toml url.

@davidsteiner
Copy link
Member

@jrobbins-LiveData That's a fair observation. The /simple/ at the end is required for dependencies to be installed from a private repository, as per this documentation:
https://python-poetry.org/docs/repositories/#install-dependencies-from-a-private-repository

When you use poetry to publish, I believe it does not belong at the end, as you say. We have been using partifact to install private dependencies, rather than for publishing. In our case, publishing is done via automated pipelines where the convenience provided by partifact has not been a priority.

Having said that, I've made some quick changes to make the parsing of the repository URL more flexible in this commit:
13930ea

Let me know if this works for you. You can install it from pypi by upgrading to the latest version.

I've also added a note about the Windows issue.

@jrobbins-LiveData
Copy link
Author

jrobbins-LiveData commented Nov 10, 2021

Thank you! I will test with the upgraded version partifact-0.1.5.

I opened an issue with keyring, offering to try to fix the Windows issue.

[EDITED...deleted the poetry export digression]

Finally, I noticed a small nit in partifact's code I wanted to let you know about. There are at least two places that construct urls or arns that have aws: hardcoded in them. While this is generally what one wants, if you use AWS GovCloud, that prefix needs to be aws-us-gov: instead.

@jrobbins-LiveData
Copy link
Author

I had initially written this

I noticed a possible problem with poetry export when running "redirected" to a private PyPi repository -- poetry places a line like this at the top of the requirements.txt file

--index-url https://<DOMAIN REDACTED>-<ACCOUNT REDACTED>.d.codeartifact.us-east-2.amazonaws.com/pypi/private-repo/simple

The url is missing the trailing slash /. This seems to not work with CodeArtifact. I need to do more testing on that, and if it is an issue, I will open one with poetry.

But I got the same behavior from pip install -r requirements.txt with or without that trailing slask. It seems that placing that --index-url at the top of requirements.txt overrides partifact's configuration, and pip simply attempts to connect with CodeArtifact without the token/password, hence the prompt for username/password.

I still might open an issue with poetry about this, because even though poetry export does have a --with-credentials option, it would be nice to be able to tell it to not emit that --index-url.

@jrobbins-LiveData
Copy link
Author

Poetry fixed their export issue (so fast, really great!!) python-poetry/poetry#4741
And I've submitted my keyring PR, supporting large passwords on Windows. I'm running with my patched keyring on my Windows box and enjoying partifact -- thank you!!!

@davidsteiner
Copy link
Member

Thanks for raising the issue with GovCloud, that's something I've been blind to. I'll fix that sometime (unless you fancy submitting a PR?), leaving this issue open until it's fixed.

Thank you for taking the time to provide detailed descriptions and feedback, I'm glad it's now working for you. If you have any other issues or ideas for making the tool more useful, please let me know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants