From a0b3c7f3dc6f28863aa77af7947cc6b287438434 Mon Sep 17 00:00:00 2001 From: Alex Kaszynski Date: Thu, 24 Mar 2022 18:56:20 +0100 Subject: [PATCH 1/5] in progress --- doc/source/guidelines/index.rst | 1 + doc/source/guidelines/private_packaging.rst | 41 +++++++++++++++++++++ 2 files changed, 42 insertions(+) create mode 100644 doc/source/guidelines/private_packaging.rst diff --git a/doc/source/guidelines/index.rst b/doc/source/guidelines/index.rst index 8d053d41..19fdb626 100644 --- a/doc/source/guidelines/index.rst +++ b/doc/source/guidelines/index.rst @@ -18,3 +18,4 @@ functionalities such as logging, data transfer, and application APIs. logging service_abstraction test_practices + private_packaging diff --git a/doc/source/guidelines/private_packaging.rst b/doc/source/guidelines/private_packaging.rst new file mode 100644 index 00000000..3aecc7a3 --- /dev/null +++ b/doc/source/guidelines/private_packaging.rst @@ -0,0 +1,41 @@ +.. _testing: + + +Hosting Private Dependencies +============================ +There will be cases in which it is necessary to pull packages that are not +ready to be hosted to the public `PyPI`_. For example, if a PyAnsys library +requires autogenerated gRPC interface files from an as-of-yet private feature +or service, this package should be hosted on a private PyPI repository. + +Ansys has a private repository at `PyAnsys PyPI`_, and access is controlled via +a secret PAT, specified in the GitHub secret ``PYANSYS_PYPI_PRIVATE_PAT`` which +is only available to private and internal repositories within the `PyAnsys`_. + + +Upload +------ +Packages can be uploaded to the private repository with the following: + +.. code:: + + pip install pip -U + pip install build twine + + # + INDEX_URL=https://$PYANSYS_PYPI_PRIVATE_PAT@pkgs.dev.azure.com/pyansys/_packaging/pyansys/pypi/simple/ + pip install ansys--==0.2.0 --index-url $INDEX_URL --no-dependencies + + # Upload + export TWINE_USERNAME=PAT + export TWINE_PASSWORD=$PYANSYS_PYPI_PRIVATE_PAT + export TWINE_REPOSITORY_URL="https://pkgs.dev.azure.com/pyansys/_packaging/pyansys/pypi/upload" + + twine upload dist/* + + python -m twine upload dist/* -p $PYANSYS_PYPI_PAT -u PAT --repository-url "https://pkgs.dev.azure.com/pyansys/_packaging/pyansys/pypi/upload" + + +.. _PyPI: https://pypi.org/ +.. _PyAnsys PyPI: https://pkgs.dev.azure.com/pyansys/_packaging/pyansys/pypi +.. _PyAnsys: https://github.com/pyansys From a0d7eb0a97942b34a1a9afac8cabc20fa3c390c0 Mon Sep 17 00:00:00 2001 From: Alex Kaszynski Date: Fri, 25 Mar 2022 12:37:48 +0100 Subject: [PATCH 2/5] finalize docs --- doc/source/guidelines/private_packaging.rst | 63 ++++++++++++++++----- 1 file changed, 50 insertions(+), 13 deletions(-) diff --git a/doc/source/guidelines/private_packaging.rst b/doc/source/guidelines/private_packaging.rst index 3aecc7a3..316e7435 100644 --- a/doc/source/guidelines/private_packaging.rst +++ b/doc/source/guidelines/private_packaging.rst @@ -3,39 +3,76 @@ Hosting Private Dependencies ============================ -There will be cases in which it is necessary to pull packages that are not -ready to be hosted to the public `PyPI`_. For example, if a PyAnsys library -requires autogenerated gRPC interface files from an as-of-yet private feature +There will be cases in which it is necessary to host and pull packages that are +not ready to be hosted to the public `PyPI`_. For example, if a PyAnsys library +requires auto-generated gRPC interface files from an as-of-yet private feature or service, this package should be hosted on a private PyPI repository. Ansys has a private repository at `PyAnsys PyPI`_, and access is controlled via a secret PAT, specified in the GitHub secret ``PYANSYS_PYPI_PRIVATE_PAT`` which -is only available to private and internal repositories within the `PyAnsys`_. +is only available to repositories within the `PyAnsys`_. + +.. note:: + This includes forked repositories. This is designed to protect against PRs + that could potentially scrape tokens from our CI/CD. Upload ------ -Packages can be uploaded to the private repository with the following: +Packages can be uploaded to the private repository with the following short +bash script. If you are operating out of a GitHub CI pipeline, email the +PyAnsys Core team at pyansys.core@ansys.com for the PAT, +``PYANSYS_PYPI_PRIVATE_PAT``. + +Assuming you are already in a Python repository containing your wheels and/or +source distribution within the ``dist`` directory: .. code:: - pip install pip -U - pip install build twine + pip install build twine pip -U - # - INDEX_URL=https://$PYANSYS_PYPI_PRIVATE_PAT@pkgs.dev.azure.com/pyansys/_packaging/pyansys/pypi/simple/ - pip install ansys--==0.2.0 --index-url $INDEX_URL --no-dependencies + REPOSITORY_URL="https://pkgs.dev.azure.com/pyansys/_packaging/pyansys/pypi/upload" + python -m twine upload dist/* \ + -p $PYANSYS_PYPI_PRIVATE_PAT \ + -u PAT \ + --repository-url $REPOSITORY-URL + +Alternatively, you can use environment variables instead of CLI arguments for twine. + +.. code:: - # Upload export TWINE_USERNAME=PAT export TWINE_PASSWORD=$PYANSYS_PYPI_PRIVATE_PAT export TWINE_REPOSITORY_URL="https://pkgs.dev.azure.com/pyansys/_packaging/pyansys/pypi/upload" - twine upload dist/* + python -m twine upload dist/* + + +Download +-------- +To download the Python package from the `PyAnsys PyPI`_, use the following: + +.. code:: + + INDEX_URL=https://$PYANSYS_PYPI_PRIVATE_PAT@pkgs.dev.azure.com/pyansys/_packaging/pyansys/pypi/simple/ + pip install ansys-- --index-url $INDEX_URL --no-dependencies + +.. warning:: + Take care to always use the ``--index-url`` switch rather than the + ``--extra-index-url`` switch. As noted in the `pip Documentation`_, the + ``-index-url`` switch changes the Python Package Index, and forces ``pip`` + to only use packages from that package index. + + Our package index uses PyPI upstream, and therefore other users cannot + inject packages from PyPI that would supersede our packages, even if they + are of a higher version. - python -m twine upload dist/* -p $PYANSYS_PYPI_PAT -u PAT --repository-url "https://pkgs.dev.azure.com/pyansys/_packaging/pyansys/pypi/upload" + This is not the case if you use ``--extra-index-url``, which adds rather + than replaces the default package index. For security do not use + ``--extra-index-url``. .. _PyPI: https://pypi.org/ .. _PyAnsys PyPI: https://pkgs.dev.azure.com/pyansys/_packaging/pyansys/pypi .. _PyAnsys: https://github.com/pyansys +.. _pip Documentation: https://pip.pypa.io/en/stable/cli/pip_install/ From 0aaa8da5eed328ae5e53710a06d7405d1ff4d1fa Mon Sep 17 00:00:00 2001 From: Alex Kaszynski Date: Fri, 25 Mar 2022 12:44:29 +0100 Subject: [PATCH 3/5] remove duplicate label --- doc/source/guidelines/private_packaging.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/source/guidelines/private_packaging.rst b/doc/source/guidelines/private_packaging.rst index 316e7435..de6054ee 100644 --- a/doc/source/guidelines/private_packaging.rst +++ b/doc/source/guidelines/private_packaging.rst @@ -1,4 +1,4 @@ -.. _testing: +.. _private_dependencies: Hosting Private Dependencies From 11af44eda2761d55bce63cbfd56ce6654d6057c9 Mon Sep 17 00:00:00 2001 From: Alex Kaszynski Date: Fri, 25 Mar 2022 13:15:49 +0100 Subject: [PATCH 4/5] Update doc/source/guidelines/private_packaging.rst Co-authored-by: Dominik Gresch --- doc/source/guidelines/private_packaging.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/source/guidelines/private_packaging.rst b/doc/source/guidelines/private_packaging.rst index de6054ee..0edf888c 100644 --- a/doc/source/guidelines/private_packaging.rst +++ b/doc/source/guidelines/private_packaging.rst @@ -60,7 +60,7 @@ To download the Python package from the `PyAnsys PyPI`_, use the following: .. warning:: Take care to always use the ``--index-url`` switch rather than the ``--extra-index-url`` switch. As noted in the `pip Documentation`_, the - ``-index-url`` switch changes the Python Package Index, and forces ``pip`` + ``--index-url`` switch changes the Python Package Index, and forces ``pip`` to only use packages from that package index. Our package index uses PyPI upstream, and therefore other users cannot From 3e9d81042c8f8f4bbbefdcaa66b2e5f3e328afe6 Mon Sep 17 00:00:00 2001 From: Alex Kaszynski Date: Fri, 25 Mar 2022 13:27:29 +0100 Subject: [PATCH 5/5] add note regarding forked repos --- doc/source/guidelines/private_packaging.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/doc/source/guidelines/private_packaging.rst b/doc/source/guidelines/private_packaging.rst index 0edf888c..c4d14d5c 100644 --- a/doc/source/guidelines/private_packaging.rst +++ b/doc/source/guidelines/private_packaging.rst @@ -13,8 +13,9 @@ a secret PAT, specified in the GitHub secret ``PYANSYS_PYPI_PRIVATE_PAT`` which is only available to repositories within the `PyAnsys`_. .. note:: - This includes forked repositories. This is designed to protect against PRs - that could potentially scrape tokens from our CI/CD. + Forked GitHub repositories do not have access to GitHub secrets. This is + designed to protect against PRs that could potentially scrape tokens from + our CI/CD. Upload