[🚀 Feature]: Upload nightly python builds

[bollwyvl]

Feature and motivation

Expand the nightly CI to upload built python .tar.gz and .whl artifacts somewhere.

Downstreams would better be able to react to breaking changes before they are released.

The alternative, of every downstream properly configuring bazel and supporting to tools seems untenable for the general case.

Usage example

Given:

• pip-compatible python packages aren't coming to GitHub any time soon
• selenium's nightly CI job includes an actions/upload-artifact for the python distributions
• a python downstream such as SeleniumLibrary wants to test against the latest selenium

During downstream CI:

• a job would would find, download, extract, and cache the artifacts
  • using either the GitHub API or a third-party tool
• a job would install the cached artifacts and run the test suite
  • in the case of a fail, an issue in the downstream's issue tracker would be created or updated, with a pointer to the build logs
  • the downstream issue would be resolved by a downstream PR such as:
    • fixing a newly-introduced incompatibility to keep the current compatibility window
    • pinning to the current version of selenium
  • the downstream would release a compatible version before the next selenium release

End users:

• would pip install -U {downstream} and get a compatible version of selenium

[github-actions]

@bollwyvl, thank you for creating this issue. We will troubleshoot it as soon as we can.

---

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

[bollwyvl]

Not sure precisely how to proceed: perhaps workflows/bazel.yml could offer a general-purpose archive-path (and optional archive-name) input. This might look something like this:

diff --git a/.github/workflows/bazel.yml b/.github/workflows/bazel.yml
index 27a64df592..aa3b4b92c5 100644
--- a/.github/workflows/bazel.yml
+++ b/.github/workflows/bazel.yml
@@ -41,6 +41,16 @@ on:
         required: false
         type: string
         default: ''
+      archive-name:
+        description: Name of an artifact to upload
+        required: false
+        default: ''
+        type: string
+      archive-path:
+        description: Paths to upload as archive
+        required: false
+        default: ''
+        type: string
 
 jobs:
   bazel:
@@ -115,6 +125,12 @@ jobs:
           edge-version: ${{ inputs.browser-version || 'stable' }}
       - name: Run Bazel
         run: ${{ inputs.run }}
+      - name: Upload Artifacts
+        if: inputs.archive-path
+        uses: actions/upload-artifact@v3
+        with:
+          name: ${{ inputs.archive-name }}
+          path: ${{ inputs.archive-path }}
       - name: Start SSH session
         if: failure() && runner.debug == '1'
         uses: mxschmitt/action-tmate@v3
diff --git a/.github/workflows/nightly.yml b/.github/workflows/nightly.yml
index 6d1ef2ec62..97d090a436 100644
--- a/.github/workflows/nightly.yml
+++ b/.github/workflows/nightly.yml
@@ -10,6 +10,18 @@ on:
         type: string
 
 jobs:
+  python:
+    name: Python
+    uses: ./.github/workflows/bazel.yml
+    strategy:
+      fail-fast: false
+    with:
+      name: Archive
+      cache-key: py-nightly
+      run: bazel build //py:selenium-wheel //py:selenium-sdist
+      archive-name: python-nightly
+      archive-path: py/dist
+
   ruby:
     name: Ruby
     uses: ./.github/workflows/bazel.yml

[titusfortner]

This is a little hidden in the README:
https://github.com/SeleniumHQ/selenium/blob/trunk/README.md#python

To install locally run:

bazel build //py:selenium-wheel
pip install bazel-bin/py/selenium-*.whl

Just make sure you have the requirements, and it might take a minute for bazel to install all-the-things: https://github.com/SeleniumHQ/selenium/blob/trunk/README.md#requirements

[bollwyvl]

Yes, i am aware that every downstream could build all of its dependencies from source every time. The point of this issue is that with a relatively small change in this repo, all downstreams could be pulling "official" nightly releases from a predictable location.

[titusfortner]

Do we already have this, or are you saying we need to add it?

selenium's nightly CI job includes an actions/upload-artifact for the python distributions

Is that all we need to do and nightly.link will give you the info you need?

[diemol]

Is there no place where we can push a nightly release?

[bollwyvl]

Is that all we need to do and nightly.link will give you the info you need?

Yes, if those paths are all correct, then the GitHub API/CLI or a third-party service would be able to get the archives (just not something as easy as the ruby one, since python packages are not supported).

Nightly does suggest:

If you'll be publishing a link to your own repository's artifacts, please install the GitHub App anyway, so that downloads for your repositories don't share the global API rate limit. The throttling will likely become very bad over time.

Which doesn't seem unreasonable.

[bollwyvl]

Is there no place where we can push a nightly release?

Yes, PyPI does allow uploading releases with the --pre flag, and some projects do use this for nightlies. However, there is no way to filter the UI for end users on e.g. https://pypi.org/project/selenium/#history, so projects with nightlies become fairly hard to navigate.

There is also the test index, but I am not sure if it has any QoS guarantees, and would generally only be useful with an invocation like:

pip download selenium --no-cache-dir --no-deps --index-url=https://test.pypi.org/simple
pip install selenium-*.whl

Other third-party services (e.g. anaconda.org) will also allow uploading .whl files to a namespaced "channel", but sticking with distribution channels by familiar first-party providers like GitHub or PyPI would likely be most comfortable to use for downstreams.

The other path would be doing semi-official a/b/rc releases in advance of final releases, but this can add a whole other layer of friction.

[diemol]

It seems we have used test index in the past https://test.pypi.org/project/selenium/#history

And I believe it would be good enough for this purpose? We would need to get access to test index. @lmtierney are you around to help us have access?

[diemol]

Is there something like the Java SNAPSHOT but for Python? I guess it is not possible to overwrite a version?

[bollwyvl]

overwrite a version?

No, PyPI (even the test registry, i believe) does not allow overwriting packages, only "yanking" them.

good enough for this purpose?

Sure, the test index would be fine, but again, if documented, should carry the disclaimer that one shouldn't expect to be able to install a full environment from that index, only get the latest wheel/sdist.

access to test index

I guess that's one of the advantages of the "dumb" CI-based workflow: it could handle nightlies for every downstream language runtime, without managing potentially an entire duplicate stable of creds for test repos and separate instructions, instead, they would all work basically the same way:

export SE_LANG_NAME=python
export SE_RUN_ID=$(gh run list --repo seleniumhq/selenium  --workflow nightly --json workflowDatabaseId --limit 1)
gh run download ${SE_RUN_ID} -n ${SE_LANG_NAME}-nightly

But again: I am only specifically interested in Python, so anywhere is better than nowhere.

[lmtierney]

It seems we have used test index in the past https://test.pypi.org/project/selenium/#history

And I believe it would be good enough for this purpose? We would need to get access to test index. @lmtierney are you around to help us have access?

@diemol if you create an account in the test space I can transfer it to you

[diemol]

Thanks, @lmtierney! I just created an account -> diemol.

[diemol]

Thanks to @lmtierney we have now access. Now we need to figure out how to configure the version in the different places to generate the build and push it.

[diemol]

Useful link https://packaging.python.org/en/latest/guides/using-testpypi/

[isaulv]

Pypi (even test-pypi) doesn't really support nightlies. But test-pypi at least is ephemeral enough to delete old nightlies.

[diemol]

The versioning would be similar to https://github.com/SeleniumHQ/selenium/pkgs/rubygems/selenium-webdriver

The issue with Python is that the version number is spread in several files. We need first to put it in a single place so it is easy to do a Nightly release.

[bollwyvl]

Once again, if the .whl and .tar.gz were uploaded workflow artifacts (a few lines in one yml file), they could be downloaded, albeit not through normal means, without downstreams having to use bazel, or re-organizing all the versions (though this is a whacking good idea, especially pyproject.toml#/project/version and read with importlib(.|_)metadata.version('selenium')).

[diemol]

I know, but we want to do this automatically, and for that, we need to set a version in a simple way without going through 5 different files. Once that is sorted out, we can push to https://test.pypi.org/project/selenium.

[diemol]

We are now publishing nightly builds to TestPyPi.

The Selenium Downloads page has also been updated.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.