-
Notifications
You must be signed in to change notification settings - Fork 9
Add documentation for external pypi #65
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
a0b3c7f
in progress
akaszynski 95a4331
merge main
akaszynski a0d7eb0
finalize docs
akaszynski 0aaa8da
remove duplicate label
akaszynski 11af44e
Update doc/source/guidelines/private_packaging.rst
akaszynski 3e9d810
add note regarding forked repos
akaszynski File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,79 @@ | ||
.. _private_dependencies: | ||
|
||
|
||
Hosting Private Dependencies | ||
============================ | ||
There will be cases in which it is necessary to host and pull packages that are | ||
not ready to be hosted to the public `PyPI`_. For example, if a PyAnsys library | ||
requires auto-generated gRPC interface files from an as-of-yet private feature | ||
or service, this package should be hosted on a private PyPI repository. | ||
|
||
Ansys has a private repository at `PyAnsys PyPI`_, and access is controlled via | ||
a secret PAT, specified in the GitHub secret ``PYANSYS_PYPI_PRIVATE_PAT`` which | ||
is only available to repositories within the `PyAnsys`_. | ||
|
||
.. note:: | ||
Forked GitHub repositories do not have access to GitHub secrets. This is | ||
designed to protect against PRs that could potentially scrape tokens from | ||
our CI/CD. | ||
|
||
|
||
Upload | ||
------ | ||
Packages can be uploaded to the private repository with the following short | ||
bash script. If you are operating out of a GitHub CI pipeline, email the | ||
PyAnsys Core team at pyansys.core@ansys.com for the PAT, | ||
``PYANSYS_PYPI_PRIVATE_PAT``. | ||
|
||
Assuming you are already in a Python repository containing your wheels and/or | ||
source distribution within the ``dist`` directory: | ||
|
||
.. code:: | ||
|
||
pip install build twine pip -U | ||
|
||
REPOSITORY_URL="https://pkgs.dev.azure.com/pyansys/_packaging/pyansys/pypi/upload" | ||
python -m twine upload dist/* \ | ||
-p $PYANSYS_PYPI_PRIVATE_PAT \ | ||
-u PAT \ | ||
--repository-url $REPOSITORY-URL | ||
|
||
Alternatively, you can use environment variables instead of CLI arguments for twine. | ||
|
||
.. code:: | ||
|
||
export TWINE_USERNAME=PAT | ||
export TWINE_PASSWORD=$PYANSYS_PYPI_PRIVATE_PAT | ||
export TWINE_REPOSITORY_URL="https://pkgs.dev.azure.com/pyansys/_packaging/pyansys/pypi/upload" | ||
|
||
python -m twine upload dist/* | ||
|
||
|
||
Download | ||
-------- | ||
To download the Python package from the `PyAnsys PyPI`_, use the following: | ||
|
||
.. code:: | ||
|
||
INDEX_URL=https://$PYANSYS_PYPI_PRIVATE_PAT@pkgs.dev.azure.com/pyansys/_packaging/pyansys/pypi/simple/ | ||
pip install ansys-<product/tool>-<library> --index-url $INDEX_URL --no-dependencies | ||
|
||
.. warning:: | ||
Take care to always use the ``--index-url`` switch rather than the | ||
``--extra-index-url`` switch. As noted in the `pip Documentation`_, the | ||
``--index-url`` switch changes the Python Package Index, and forces ``pip`` | ||
to only use packages from that package index. | ||
|
||
Our package index uses PyPI upstream, and therefore other users cannot | ||
inject packages from PyPI that would supersede our packages, even if they | ||
are of a higher version. | ||
|
||
akaszynski marked this conversation as resolved.
Show resolved
Hide resolved
|
||
This is not the case if you use ``--extra-index-url``, which adds rather | ||
than replaces the default package index. For security do not use | ||
``--extra-index-url``. | ||
|
||
|
||
.. _PyPI: https://pypi.org/ | ||
.. _PyAnsys PyPI: https://pkgs.dev.azure.com/pyansys/_packaging/pyansys/pypi | ||
.. _PyAnsys: https://github.com/pyansys | ||
.. _pip Documentation: https://pip.pypa.io/en/stable/cli/pip_install/ |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@akaszynski what are other companies doing which might have similar scenarios?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Desire
There is a huge push for packages to be hosted in GitHub packages, and I think the drive is to do that:
https://github.saobby.my.eu.orgmunity/t/pypi-compatible-github-package-registry/14615/103
Solutions I've seen in the wild
A common solution has been to use public ADO for hosting both universal artifacts and Python packages. Another alternative within GitHub is to use a repository for that:
https://github.com/astariul/github-hosted-pypi
While this solution works, it's non-optimal as you have to git push and we would still have to track a PAT as pulling would require access to a private repository.
Other alternatives include
In the end we chose ADO to: