ci: use GHA instead of Azure 🔨 #1550

agoose77 · 2022-07-16T20:43:21Z

This PR moves the buildtest CI to GitHub Actions, and drops the doctest workflow.

My motivation for tackling this is that I am also moving the docs to GHA in a separate PR.

I've not used Azure before, and so this might not 100% match on the trigger logic. The exclusion logic looks as though it should never run, but clearly it does in main, so I must be missing something.

This PR sets the action to run on PR, like the tests.

codecov · 2022-07-16T20:53:43Z

Codecov Report

Merging #1550 (b0f834f) into main (9e17f29) will decrease coverage by 0.02%.
The diff coverage is 62.09%.

Impacted Files	Coverage Δ
src/awkward/_v2/_connect/cuda/__init__.py	`0.00% <0.00%> (ø)`
src/awkward/_v2/_connect/numexpr.py	`88.40% <0.00%> (ø)`
src/awkward/_v2/_connect/pyarrow.py	`88.46% <0.00%> (ø)`
src/awkward/_v2/numba.py	`93.47% <0.00%> (ø)`
src/awkward/_v2/operations/ak_from_avro_file.py	`66.66% <0.00%> (ø)`
src/awkward/_v2/operations/ak_local_index.py	`100.00% <ø> (ø)`
src/awkward/_v2/operations/ak_type.py	`53.84% <0.00%> (ø)`
src/awkward/_v2/operations/ak_transform.py	`8.62% <8.62%> (ø)`
src/awkward/_v2/contents/unionarray.py	`86.16% <50.00%> (-0.11%)`	⬇️
src/awkward/_v2/contents/bitmaskedarray.py	`71.53% <53.84%> (+5.11%)`	⬆️
... and 26 more

agoose77 · 2022-07-18T14:46:54Z

@jpivarski not sure who else to tag on this. Thought perhaps Henry, but I see he's marked himself as "busy".

jpivarski · 2022-07-18T15:24:32Z

Does this entirely switch from Azure to GitHub Actions? I had thought there were some blockers preventing us from doing that, but it looks like all of the tests are running here. I'll need to change the required tests in main's branch protection before we can merge it, and the other active PRs should adopt this before they merge, so this will take a little time.

As such, let's ask @henryiii to review this. He's busy this week with Snowmass, but maybe he'll get a chance to look at it next week. Meanwhile, I'll spread the word that this switchover is happening and to be ready for it. This would be a good reason for Awkward Array project meetings (or just "Awkward Meetings"?), for us all to keep in touch with what everyone else is doing. We might have a regular time for that by the end of August.

henryiii · 2022-07-18T15:28:23Z

I'll look at it later, but pretty sure there weren't blockers, it was just work that had never been done. Azure is just a earlier design of GHA.

.github/workflows/build-test.yml

agoose77 · 2022-07-18T17:13:01Z

Does this entirely switch from Azure to GitHub Actions?

Only buildtest, but the doctest will also be moved in a separate docs PR

agoose77 · 2022-07-19T09:35:45Z

I think I'll need to revisit this in the near future, once the docs are being built on GHA too. However, I think at that point we can look at whether an optimisation pass is necessary; right now, we will build one architecture 3X: once for docs, once for tests, and once for the wheel build test. Maybe that is OK, but regardless, a future problem!

agoose77 · 2022-07-19T13:11:09Z

I believe this is nearly ready to go. Just needs some input as to the triggers - do we want to run this on every PR push? Right now, that is what's happening on Azure, though I'm not sure if it is intentional

jpivarski · 2022-07-19T15:07:47Z

This is great, but let's actually merge it on an off-week. At the same time that it is merged, we'll have to switch the main branch protection to expect these tests, rather than the Azure ones, so everyone will need to merge this PR to get passing tests.

The first week of August (1‒5) is an off-week because several of us will be at CoDaS-HEP.

People with open PRs are you, me, @ianna, @ManasviGoyal, @aryan26roy (though he'll be done by August), and @martindurant.

martindurant · 2022-07-19T16:06:26Z

Mine can be merged, and I can iterate, right?

jpivarski · 2022-07-19T16:09:23Z

Mine can be merged, and I can iterate, right?

Okay, I didn't know it was declared ready. I'll try merging it with main to run the tests again, and I'll give it another scan, then merge it.

jpivarski · 2022-08-10T00:40:32Z

I just brought this up to date because it will be merged next week, probably right after our first group meeting. That would be a good way to warn everybody.

jpivarski · 2022-08-17T16:43:33Z

@agoose77 Actually, I should be able to remove the azure-doctest-awkward.yml from this also (i.e. no more .ci directory and no Azure tests at all). The documentation tests are minimal compare with what you're doing in the docs branch, and that's where all the new development is going.

Any reason why you kept it? If you're leaving it up to me, I'll delete it and we'll go entirely onto GitHub Actions (good for simplicity).

agoose77 · 2022-08-17T17:53:25Z

I removed the doctest CI in the docs branch, but I was leaving it here for now. I don't mind if you want to remove it here too!

jpivarski · 2022-08-19T21:28:51Z

I forgot about this; I'll have to do it on the weekend.

diff --git a/.github/workflows/build-test.yml b/.github/workflows/build-test.yml new file mode 100644 index 00000000..e59d081b --- /dev/null +++ b/.github/workflows/build-test.yml @@ -0,0 +1,154 @@ +name: Build Test + +on: + push: + branches-ignore: + - '*' + paths-ignore: + - README.md + - VERSION_INFO + - .ci/azure-doctest-awkward.yml + - .ci/azure-deploy-awkward.yml + - .readthedocs.yml + - docs-src/* + - docs-img/* + - docs-jupyter/* + - docs-doxygen/* + - docs-sphinx/* + - docs/* + - studies/* + - src/cuda-kernels + - cuda-kernels + pull-request: + branches-ignore: + - '*' + schedule: + - cron: '0 12 1 * *' + +concurrency: + group: build-test-${{ github.head_ref || github.run_id }} + cancel-in-progress: true + +jobs: + Windows: + strategy: + matrix: + python.version: + - 3.9 + - 3.10 + - 3.9 + - 3.8 + - 3.7 + - 3.6 + runs-on: windows-2019 + env: + PIP_ONLY_BINARY: cmake + steps: + - uses: actions/checkout@v2 + - name: Python ${{ matrix.python.version }} + uses: actions/setup-python@v1 + with: + python-version: ${{ matrix.python.version }} + - name: Install Pip + run: python -m pip install --upgrade pip + - name: Print versions + run: python -m pip list + - name: Build + run: python -m pip install -v .[test,dev] + - name: Check if kernel specification is sorted + run: python dev/kernel-diagnostics.py --check-spec-sorted + - name: Generate Kernel tests + run: python dev/generate-tests.py + - name: Test specification + run: python -m pytest -vv -rs tests-spec + - name: Test specification with explicitly defined values + run: python -m pytest -vv -rs tests-spec-explicit + - name: Test CPU kernels + run: python -m pytest -vv -rs tests-cpu-kernels + - name: Test + run: python -m pytest -vv -rs tests + + MacOS: + strategy: + matrix: + python.version: + - 3.10 + - 3.9 + - 3.8 + - 3.7 + - 3.6 + runs-on: macOS-10.15 + env: + PIP_ONLY_BINARY: cmake + steps: + - uses: actions/checkout@v2 + - name: Python ${{ matrix.python.version }} + uses: actions/setup-python@v1 + with: + python-version: ${{ matrix.python.version }} + - name: Install Pip + run: python -m pip install --upgrade pip + - name: Print versions + run: python -m pip list + - name: Build + run: python -m pip install -v .[test,dev] + - name: Check if kernel specification is sorted + run: python dev/kernel-diagnostics.py --check-spec-sorted + - name: Generate Kernel tests + run: python dev/generate-tests.py + - name: Test specification + run: python -m pytest -vv -rs tests-spec + - name: Test specification with explicitly defined values + run: python -m pytest -vv -rs tests-spec-explicit + - name: Test CPU kernels + run: python -m pytest -vv -rs tests-cpu-kernels + - name: Test + run: python -m pytest -vv -rs test + + Linux: + strategy: + matrix: + python.version: + - 3.10 + - 3.9 + - 3.8 + - 3.7 + - 3.6 + - 3.6 + runs-on: ubuntu-18.04 + env: + PIP_ONLY_BINARY: cmake + steps: + - uses: actions/checkout@v2 + - name: Python ${{ matrix.python.version }} + uses: actions/setup-python@v1 + with: + python-version: ${{ matrix.python.version }} + - name: Install Pip + run: python -m pip install --upgrade pip + - name: Install NumPy + run: | + if [ ${{ env.numpy.version }} = "latest" ]; then + python -m pip install numpy; + else + python -m pip install numpy==${{ env.numpy.version }}; + fi + - name: Build + run: python -m pip install -v .[test,dev] + - name: Print versions + run: python -m pip list + - name: Check if kernel specification is sorted + run: python dev/kernel-diagnostics.py --check-spec-sorted + - name: Generate Kernel tests + run: python dev/generate-tests.py + - name: Test specification + run: python -m pytest -vv -rs tests-spec + - name: Test specification with explicitly defined values + run: python -m pytest -vv -rs tests-spec-explicit + - name: Test CPU kernels + run: python -m pytest -vv -rs tests-cpu-kernels + - name: Test + run: python -m pytest -vv -rs tests --cov=awkward._v2 --cov-report=term --cov-report=xml + - name: Codecov + run: bash <(curl -s https://codecov.io/bash) + if: matrix.python.version == '3.9'

diff --git a/.github/workflows/build-test.yml b/.github/workflows/build-test.yml index e59d081b..fc30c44d 100644 --- a/.github/workflows/build-test.yml +++ b/.github/workflows/build-test.yml @@ -19,7 +19,7 @@ on: - studies/* - src/cuda-kernels - cuda-kernels - pull-request: + pull_request: branches-ignore: - '*' schedule:

diff --git a/.github/workflows/build-test.yml b/.github/workflows/build-test.yml index fc30c44d..e2f90174 100644 --- a/.github/workflows/build-test.yml +++ b/.github/workflows/build-test.yml @@ -25,6 +25,8 @@ on: schedule: - cron: '0 12 1 * *' + workflow_dispatch: + concurrency: group: build-test-${{ github.head_ref || github.run_id }} cancel-in-progress: true

diff --git a/.github/workflows/build-test.yml b/.github/workflows/build-test.yml index e2f90174..84c67545 100644 --- a/.github/workflows/build-test.yml +++ b/.github/workflows/build-test.yml @@ -2,13 +2,10 @@ name: Build Test on: push: - branches-ignore: - - '*' paths-ignore: - README.md - VERSION_INFO - - .ci/azure-doctest-awkward.yml - - .ci/azure-deploy-awkward.yml + - .ci/* - .readthedocs.yml - docs-src/* - docs-img/* @@ -19,9 +16,8 @@ on: - studies/* - src/cuda-kernels - cuda-kernels + # Don't run for any PR pull_request: - branches-ignore: - - '*' schedule: - cron: '0 12 1 * *'

diff --git a/.github/workflows/build-test.yml b/.github/workflows/build-test.yml index 84c67545..a22224ac 100644 --- a/.github/workflows/build-test.yml +++ b/.github/workflows/build-test.yml @@ -107,12 +107,14 @@ jobs: strategy: matrix: python.version: - - 3.10 - - 3.9 - - 3.8 - - 3.7 - - 3.6 - - 3.6 + - 3.10 + - 3.9 + - 3.8 + - 3.7 + - 3.6 + include: + - python.version: 3.6 + numpy.version: 1.13.1 runs-on: ubuntu-18.04 env: PIP_ONLY_BINARY: cmake @@ -126,10 +128,10 @@ jobs: run: python -m pip install --upgrade pip - name: Install NumPy run: | - if [ ${{ env.numpy.version }} = "latest" ]; then + if [ ${{ matrix.numpy.version }} = "latest" ]; then python -m pip install numpy; else - python -m pip install numpy==${{ env.numpy.version }}; + python -m pip install numpy==${{ matrix.numpy.version }}; fi - name: Build run: python -m pip install -v .[test,dev]

diff --git a/.github/workflows/build-test.yml b/.github/workflows/build-test.yml index a22224ac..da3023ef 100644 --- a/.github/workflows/build-test.yml +++ b/.github/workflows/build-test.yml @@ -32,12 +32,12 @@ jobs: strategy: matrix: python.version: - - 3.9 - - 3.10 - - 3.9 - - 3.8 - - 3.7 - - 3.6 + - '3.9' + - '3.10' + - '3.9' + - '3.8' + - '3.7' + - '3.6' runs-on: windows-2019 env: PIP_ONLY_BINARY: cmake @@ -70,11 +70,11 @@ jobs: strategy: matrix: python.version: - - 3.10 - - 3.9 - - 3.8 - - 3.7 - - 3.6 + - '3.10' + - '3.9' + - '3.8' + - '3.7' + - '3.6' runs-on: macOS-10.15 env: PIP_ONLY_BINARY: cmake @@ -107,14 +107,14 @@ jobs: strategy: matrix: python.version: - - 3.10 - - 3.9 - - 3.8 - - 3.7 - - 3.6 + - '3.10' + - '3.9' + - '3.8' + - '3.7' + - '3.6' include: - - python.version: 3.6 - numpy.version: 1.13.1 + - python.version: '3.6' + numpy.version: '1.13.1' runs-on: ubuntu-18.04 env: PIP_ONLY_BINARY: cmake

diff --git a/.github/workflows/build-test.yml b/.github/workflows/build-test.yml index da3023ef..a23432ff 100644 --- a/.github/workflows/build-test.yml +++ b/.github/workflows/build-test.yml @@ -69,7 +69,7 @@ jobs: MacOS: strategy: matrix: - python.version: + python-version: - '3.10' - '3.9' - '3.8' @@ -114,7 +114,7 @@ jobs: - '3.6' include: - python.version: '3.6' - numpy.version: '1.13.1' + numpy-version: '1.13.1' runs-on: ubuntu-18.04 env: PIP_ONLY_BINARY: cmake @@ -128,10 +128,10 @@ jobs: run: python -m pip install --upgrade pip - name: Install NumPy run: | - if [ ${{ matrix.numpy.version }} = "latest" ]; then + if [ ${{ matrix.numpy-version }} = "latest" ]; then python -m pip install numpy; else - python -m pip install numpy==${{ matrix.numpy.version }}; + python -m pip install numpy==${{ matrix.numpy-version }}; fi - name: Build run: python -m pip install -v .[test,dev]

diff --git a/.github/workflows/build-test.yml b/.github/workflows/build-test.yml index a23432ff..7aa9cbdf 100644 --- a/.github/workflows/build-test.yml +++ b/.github/workflows/build-test.yml @@ -31,7 +31,7 @@ jobs: Windows: strategy: matrix: - python.version: + python-version: - '3.9' - '3.10' - '3.9' @@ -43,10 +43,10 @@ jobs: PIP_ONLY_BINARY: cmake steps: - uses: actions/checkout@v2 - - name: Python ${{ matrix.python.version }} + - name: Python ${{ matrix.python-version }} uses: actions/setup-python@v1 with: - python-version: ${{ matrix.python.version }} + python-version: ${{ matrix.python-version }} - name: Install Pip run: python -m pip install --upgrade pip - name: Print versions @@ -80,10 +80,10 @@ jobs: PIP_ONLY_BINARY: cmake steps: - uses: actions/checkout@v2 - - name: Python ${{ matrix.python.version }} + - name: Python ${{ matrix.python-version }} uses: actions/setup-python@v1 with: - python-version: ${{ matrix.python.version }} + python-version: ${{ matrix.python-version }} - name: Install Pip run: python -m pip install --upgrade pip - name: Print versions @@ -106,32 +106,33 @@ jobs: Linux: strategy: matrix: - python.version: + python-version: - '3.10' - '3.9' - '3.8' - '3.7' - '3.6' include: - - python.version: '3.6' + - python-version: '3.6' numpy-version: '1.13.1' runs-on: ubuntu-18.04 env: PIP_ONLY_BINARY: cmake + NUMPY_VERSION: ${{ matrix.numpy-version }} steps: - uses: actions/checkout@v2 - - name: Python ${{ matrix.python.version }} + - name: Python ${{ matrix.python-version }} uses: actions/setup-python@v1 with: - python-version: ${{ matrix.python.version }} + python-version: ${{ matrix.python-version }} - name: Install Pip run: python -m pip install --upgrade pip - name: Install NumPy run: | - if [ ${{ matrix.numpy-version }} = "latest" ]; then + if [[ "${NUMPY_VERSION}" == "latest" ]]; then python -m pip install numpy; else - python -m pip install numpy==${{ matrix.numpy-version }}; + python -m pip install "numpy==${NUMPY_VERSION}"; fi - name: Build run: python -m pip install -v .[test,dev] @@ -151,4 +152,4 @@ jobs: run: python -m pytest -vv -rs tests --cov=awkward._v2 --cov-report=term --cov-report=xml - name: Codecov run: bash <(curl -s https://codecov.io/bash) - if: matrix.python.version == '3.9' + if: matrix.python-version == '3.9'

Oops! diff --git a/.github/workflows/build-test.yml b/.github/workflows/build-test.yml index 7aa9cbdf..7c5c0b48 100644 --- a/.github/workflows/build-test.yml +++ b/.github/workflows/build-test.yml @@ -1,7 +1,7 @@ name: Build Test on: - push: + pull_request: paths-ignore: - README.md - VERSION_INFO @@ -17,7 +17,6 @@ on: - src/cuda-kernels - cuda-kernels # Don't run for any PR - pull_request: schedule: - cron: '0 12 1 * *' @@ -129,7 +128,7 @@ jobs: run: python -m pip install --upgrade pip - name: Install NumPy run: | - if [[ "${NUMPY_VERSION}" == "latest" ]]; then + if [[ -z "${NUMPY_VERSION}" ]]; then python -m pip install numpy; else python -m pip install "numpy==${NUMPY_VERSION}";

diff --git a/.github/workflows/build-test.yml b/.github/workflows/build-test.yml index 7c5c0b48..7c421c29 100644 --- a/.github/workflows/build-test.yml +++ b/.github/workflows/build-test.yml @@ -41,9 +41,11 @@ jobs: env: PIP_ONLY_BINARY: cmake steps: - - uses: actions/checkout@v2 + - uses: actions/checkout@v3 + with: + submodules: true - name: Python ${{ matrix.python-version }} - uses: actions/setup-python@v1 + uses: actions/setup-python@v4 with: python-version: ${{ matrix.python-version }} - name: Install Pip @@ -78,9 +80,11 @@ jobs: env: PIP_ONLY_BINARY: cmake steps: - - uses: actions/checkout@v2 + - uses: actions/checkout@v3 + with: + submodules: true - name: Python ${{ matrix.python-version }} - uses: actions/setup-python@v1 + uses: actions/setup-python@v4 with: python-version: ${{ matrix.python-version }} - name: Install Pip @@ -119,9 +123,11 @@ jobs: PIP_ONLY_BINARY: cmake NUMPY_VERSION: ${{ matrix.numpy-version }} steps: - - uses: actions/checkout@v2 + - uses: actions/checkout@v3 + with: + submodules: true - name: Python ${{ matrix.python-version }} - uses: actions/setup-python@v1 + uses: actions/setup-python@v4 with: python-version: ${{ matrix.python-version }} - name: Install Pip

diff --git a/.github/workflows/build-test.yml b/.github/workflows/build-test.yml index 7c421c29..cfbf599f 100644 --- a/.github/workflows/build-test.yml +++ b/.github/workflows/build-test.yml @@ -126,6 +126,13 @@ jobs: - uses: actions/checkout@v3 with: submodules: true + - name: Setup ccache + uses: hendrikmuhs/ccache-action@v1.2 + with: + key: ${{ github.job }}-${{ matrix.os }}-${{matrix.python-version}}-${{matrix.numpy-version}} + - name: Use ccache + run: | + export PATH="/usr/lib/ccache:/usr/local/opt/ccache/libexec:$PATH" - name: Python ${{ matrix.python-version }} uses: actions/setup-python@v4 with:

diff --git a/.github/workflows/build-test.yml b/.github/workflows/build-test.yml index cfbf599f..14c94e4f 100644 --- a/.github/workflows/build-test.yml +++ b/.github/workflows/build-test.yml @@ -132,7 +132,8 @@ jobs: key: ${{ github.job }}-${{ matrix.os }}-${{matrix.python-version}}-${{matrix.numpy-version}} - name: Use ccache run: | - export PATH="/usr/lib/ccache:/usr/local/opt/ccache/libexec:$PATH" + echo "/usr/lib/ccache" >> $GITHUB_PATH + echo "/usr/local/opt/ccache/libexec" >> $GITHUB_PATH - name: Python ${{ matrix.python-version }} uses: actions/setup-python@v4 with:

diff --git a/.github/workflows/build-test.yml b/.github/workflows/build-test.yml index 14c94e4f..1c13737a 100644 --- a/.github/workflows/build-test.yml +++ b/.github/workflows/build-test.yml @@ -104,7 +104,7 @@ jobs: - name: Test CPU kernels run: python -m pytest -vv -rs tests-cpu-kernels - name: Test - run: python -m pytest -vv -rs test + run: python -m pytest -vv -rs tests Linux: strategy:

diff --git a/.github/workflows/build-test.yml b/.github/workflows/build-test.yml index 1c13737a..13741512 100644 --- a/.github/workflows/build-test.yml +++ b/.github/workflows/build-test.yml @@ -83,6 +83,14 @@ jobs: - uses: actions/checkout@v3 with: submodules: true + - name: Setup ccache + uses: hendrikmuhs/ccache-action@v1.2 + with: + key: ${{ github.job }}-${{ matrix.os }}-${{matrix.python-version}}-${{matrix.numpy-version}} + - name: Use ccache + run: | + echo "/usr/lib/ccache" >> $GITHUB_PATH + echo "/usr/local/opt/ccache/libexec" >> $GITHUB_PATH - name: Python ${{ matrix.python-version }} uses: actions/setup-python@v4 with:

diff --git a/.github/workflows/build-test.yml b/.github/workflows/build-test.yml index 13741512..5a32175c 100644 --- a/.github/workflows/build-test.yml +++ b/.github/workflows/build-test.yml @@ -86,7 +86,7 @@ jobs: - name: Setup ccache uses: hendrikmuhs/ccache-action@v1.2 with: - key: ${{ github.job }}-${{ matrix.os }}-${{matrix.python-version}}-${{matrix.numpy-version}} + key: ${{ github.job }}-${{matrix.python-version}}-${{matrix.numpy-version}} - name: Use ccache run: | echo "/usr/lib/ccache" >> $GITHUB_PATH @@ -137,7 +137,7 @@ jobs: - name: Setup ccache uses: hendrikmuhs/ccache-action@v1.2 with: - key: ${{ github.job }}-${{ matrix.os }}-${{matrix.python-version}}-${{matrix.numpy-version}} + key: ${{ github.job }}-${{matrix.python-version}}-${{matrix.numpy-version}} - name: Use ccache run: | echo "/usr/lib/ccache" >> $GITHUB_PATH

Co-authored-by: Henry Schreiner <henry.fredrick.schreiner@cern.ch>

diff --git a/.github/workflows/build-test.yml b/.github/workflows/build-test.yml index 5a32175c..df4df784 100644 --- a/.github/workflows/build-test.yml +++ b/.github/workflows/build-test.yml @@ -48,10 +48,6 @@ jobs: uses: actions/setup-python@v4 with: python-version: ${{ matrix.python-version }} - - name: Install Pip - run: python -m pip install --upgrade pip - - name: Print versions - run: python -m pip list - name: Build run: python -m pip install -v .[test,dev] - name: Check if kernel specification is sorted @@ -86,7 +82,7 @@ jobs: - name: Setup ccache uses: hendrikmuhs/ccache-action@v1.2 with: - key: ${{ github.job }}-${{matrix.python-version}}-${{matrix.numpy-version}} + key: ${{ github.job }}-${{matrix.python-version}} - name: Use ccache run: | echo "/usr/lib/ccache" >> $GITHUB_PATH @@ -95,10 +91,6 @@ jobs: uses: actions/setup-python@v4 with: python-version: ${{ matrix.python-version }} - - name: Install Pip - run: python -m pip install --upgrade pip - - name: Print versions - run: python -m pip list - name: Build run: python -m pip install -v .[test,dev] - name: Check if kernel specification is sorted @@ -146,8 +138,6 @@ jobs: uses: actions/setup-python@v4 with: python-version: ${{ matrix.python-version }} - - name: Install Pip - run: python -m pip install --upgrade pip - name: Install NumPy run: | if [[ -z "${NUMPY_VERSION}" ]]; then

diff --git a/.github/workflows/build-test.yml b/.github/workflows/build-test.yml index a97eae84..7dbef00a 100644 --- a/.github/workflows/build-test.yml +++ b/.github/workflows/build-test.yml @@ -3,27 +3,27 @@ name: Build Test on: pull_request: paths-ignore: - - README.md - - VERSION_INFO - - .ci/* - - .readthedocs.yml - - docs-src/* - - docs-img/* - - docs-jupyter/* - - docs-doxygen/* - - docs-sphinx/* - - docs/* - - studies/* - - src/cuda-kernels - - cuda-kernels - # Don't run for any PR + - README.md + - VERSION_INFO + - .ci/* + - .readthedocs.yml + - docs-src/* + - docs-img/* + - docs-jupyter/* + - docs-doxygen/* + - docs-sphinx/* + - docs/* + - studies/* + - src/cuda-kernels + - cuda-kernels + schedule: - - cron: '0 12 1 * *' + - cron: 0 12 1 * * - workflow_dispatch: + workflow_dispatch: null concurrency: - group: build-test-${{ github.head_ref || github.run_id }} + group: 'build-test-${{ github.head_ref || github.run_id }}' cancel-in-progress: true jobs: @@ -31,136 +31,187 @@ jobs: strategy: matrix: python-version: - - '3.9' - - '3.10' - - '3.9' - - '3.8' - - '3.7' - - '3.6' + - '3.10' + - '3.9' + - '3.8' + - '3.7' + - '3.6' + + python-architecture: x64 + + include: + - python-version: '3.6' + python-architecture: x86 + runs-on: windows-2019 + env: PIP_ONLY_BINARY: cmake + steps: - - uses: actions/checkout@v3 - with: - submodules: true - - name: Python ${{ matrix.python-version }} - uses: actions/setup-python@v4 - with: - python-version: ${{ matrix.python-version }} - - name: Build - run: python -m pip install -v .[test,dev] - - name: Check if kernel specification is sorted - run: python dev/kernel-diagnostics.py --check-spec-sorted - - name: Generate Kernel tests - run: python dev/generate-tests.py - - name: Test specification - run: python -m pytest -vv -rs tests-spec - - name: Test specification with explicitly defined values - run: python -m pytest -vv -rs tests-spec-explicit - - name: Test CPU kernels - run: python -m pytest -vv -rs tests-cpu-kernels - - name: Test - run: python -m pytest -vv -rs tests + - uses: actions/checkout@v3 + with: + submodules: true + + - name: 'Python ${{ matrix.python-version }}' + uses: actions/setup-python@v4 + with: + python-version: '${{ matrix.python-version }}' + architecture: '${{ matrix.python-architecture || }}' + + - name: Build + run: 'python -m pip install -v .[test,dev]' + + - name: Check if kernel specification is sorted + run: python dev/kernel-diagnostics.py --check-spec-sorted + + - name: Generate Kernel tests + run: python dev/generate-tests.py + + - name: Test specification + run: python -m pytest -vv -rs tests-spec + + - name: Test specification with explicitly defined values + run: python -m pytest -vv -rs tests-spec-explicit + + - name: Test CPU kernels + run: python -m pytest -vv -rs tests-cpu-kernels + + - name: Test + run: python -m pytest -vv -rs tests MacOS: strategy: matrix: python-version: - - '3.10' - - '3.9' - - '3.8' - - '3.7' - - '3.6' + - '3.10' + - '3.9' + - '3.8' + - '3.7' + - '3.6' + runs-on: macOS-11 + env: PIP_ONLY_BINARY: cmake + steps: - - uses: actions/checkout@v3 - with: - submodules: true - - name: Setup ccache - uses: hendrikmuhs/ccache-action@v1.2 - with: - key: ${{ github.job }}-${{matrix.python-version}} - - name: Use ccache - run: | - echo "/usr/lib/ccache" >> $GITHUB_PATH - echo "/usr/local/opt/ccache/libexec" >> $GITHUB_PATH - - name: Python ${{ matrix.python-version }} - uses: actions/setup-python@v4 - with: - python-version: ${{ matrix.python-version }} - - name: Build - run: python -m pip install -v .[test,dev] - - name: Check if kernel specification is sorted - run: python dev/kernel-diagnostics.py --check-spec-sorted - - name: Generate Kernel tests - run: python dev/generate-tests.py - - name: Test specification - run: python -m pytest -vv -rs tests-spec - - name: Test specification with explicitly defined values - run: python -m pytest -vv -rs tests-spec-explicit - - name: Test CPU kernels - run: python -m pytest -vv -rs tests-cpu-kernels - - name: Test - run: python -m pytest -vv -rs tests + - uses: actions/checkout@v3 + with: + submodules: true + + - name: Setup ccache + uses: hendrikmuhs/ccache-action@v1.2 + with: + key: '${{ github.job }}-${{matrix.python-version}}' + + - name: Use ccache + run: | + echo "/usr/lib/ccache" >> $GITHUB_PATH + echo "/usr/local/opt/ccache/libexec" >> $GITHUB_PATH + + - name: 'Python ${{ matrix.python-version }}' + uses: actions/setup-python@v4 + with: + python-version: '${{ matrix.python-version }}' + + - name: Build + run: 'python -m pip install -v .[test,dev]' + + - name: Check if kernel specification is sorted + run: python dev/kernel-diagnostics.py --check-spec-sorted + + - name: Generate Kernel tests + run: python dev/generate-tests.py + + - name: Test specification + run: python -m pytest -vv -rs tests-spec + + - name: Test specification with explicitly defined values + run: python -m pytest -vv -rs tests-spec-explicit + + - name: Test CPU kernels + run: python -m pytest -vv -rs tests-cpu-kernels + + - name: Test + run: python -m pytest -vv -rs tests Linux: strategy: matrix: python-version: - - '3.10' - - '3.9' - - '3.8' - - '3.7' - - '3.6' + - '3.10' + - '3.9' + - '3.8' + - '3.7' + - '3.6' + include: - - python-version: '3.6' - numpy-version: '1.13.1' + - python-version: '3.6' + numpy-version: 1.13.1 + runs-on: ubuntu-18.04 + env: PIP_ONLY_BINARY: cmake - NUMPY_VERSION: ${{ matrix.numpy-version }} + NUMPY_VERSION: '${{ matrix.numpy-version }}' + steps: - - uses: actions/checkout@v3 - with: - submodules: true - - name: Setup ccache - uses: hendrikmuhs/ccache-action@v1.2 - with: - key: ${{ github.job }}-${{matrix.python-version}}-${{matrix.numpy-version}} - - name: Use ccache - run: | - echo "/usr/lib/ccache" >> $GITHUB_PATH - echo "/usr/local/opt/ccache/libexec" >> $GITHUB_PATH - - name: Python ${{ matrix.python-version }} - uses: actions/setup-python@v4 - with: - python-version: ${{ matrix.python-version }} - - name: Install NumPy - run: | - if [[ -z "${NUMPY_VERSION}" ]]; then - python -m pip install numpy; - else - python -m pip install "numpy==${NUMPY_VERSION}"; - fi - - name: Build - run: python -m pip install -v .[test,dev] - - name: Print versions - run: python -m pip list - - name: Check if kernel specification is sorted - run: python dev/kernel-diagnostics.py --check-spec-sorted - - name: Generate Kernel tests - run: python dev/generate-tests.py - - name: Test specification - run: python -m pytest -vv -rs tests-spec - - name: Test specification with explicitly defined values - run: python -m pytest -vv -rs tests-spec-explicit - - name: Test CPU kernels - run: python -m pytest -vv -rs tests-cpu-kernels - - name: Test - run: python -m pytest -vv -rs tests --cov=awkward._v2 --cov-report=term --cov-report=xml - - name: Codecov - run: bash <(curl -s https://codecov.io/bash) - if: matrix.python-version == '3.9' + - uses: actions/checkout@v3 + with: + submodules: true + + - name: Setup ccache + uses: hendrikmuhs/ccache-action@v1.2 + with: + key: >- + ${{ github.job + }}-${{matrix.python-version}}-${{matrix.numpy-version}} + + - name: Use ccache + run: | + echo "/usr/lib/ccache" >> $GITHUB_PATH + echo "/usr/local/opt/ccache/libexec" >> $GITHUB_PATH + + - name: 'Python ${{ matrix.python-version }}' + uses: actions/setup-python@v4 + with: + python-version: '${{ matrix.python-version }}' + + - name: Install NumPy + run: | + if [[ -z "${NUMPY_VERSION}" ]]; then + python -m pip install numpy; + else + python -m pip install "numpy==${NUMPY_VERSION}"; + fi + + - name: Build + run: 'python -m pip install -v .[test,dev]' + + - name: Print versions + run: python -m pip list + + - name: Check if kernel specification is sorted + run: python dev/kernel-diagnostics.py --check-spec-sorted + + - name: Generate Kernel tests + run: python dev/generate-tests.py + + - name: Test specification + run: python -m pytest -vv -rs tests-spec + + - name: Test specification with explicitly defined values + run: python -m pytest -vv -rs tests-spec-explicit + + - name: Test CPU kernels + run: python -m pytest -vv -rs tests-cpu-kernels + + - name: Test + run: >- + python -m pytest -vv -rs tests --cov=awkward._v2 --cov-report=term + --cov-report=xml + + - name: Codecov + run: 'bash <(curl -s https://codecov.io/bash)' + if: matrix.python-version == '3.9'

diff --git a/.github/workflows/build-test.yml b/.github/workflows/build-test.yml index 7dbef00a..d285fb92 100644 --- a/.github/workflows/build-test.yml +++ b/.github/workflows/build-test.yml @@ -40,7 +40,7 @@ jobs: python-architecture: x64 include: - - python-version: '3.6' + - python-version: '3.9' python-architecture: x86 runs-on: windows-2019

diff --git a/.github/workflows/build-test.yml b/.github/workflows/build-test.yml index d285fb92..4ff700b6 100644 --- a/.github/workflows/build-test.yml +++ b/.github/workflows/build-test.yml @@ -37,7 +37,8 @@ jobs: - '3.7' - '3.6' - python-architecture: x64 + python-architecture: + - x64 include: - python-version: '3.9'

diff --git a/.github/workflows/wheels.yml b/.github/workflows/deploy.yml similarity index 99% rename from .github/workflows/wheels.yml rename to .github/workflows/deploy.yml index 0117d490..cf74bc2f 100644 --- a/.github/workflows/wheels.yml +++ b/.github/workflows/deploy.yml @@ -1,4 +1,4 @@ -name: Wheels +name: "[CD] Wheels" on: workflow_dispatch: diff --git a/.github/workflows/tests.yml b/.github/workflows/test-build-wheels.yml similarity index 98% rename from .github/workflows/tests.yml rename to .github/workflows/test-build-wheels.yml index 55d79458..d2cbbdfb 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/test-build-wheels.yml @@ -1,4 +1,4 @@ -name: Tests +name: "[CI] Wheels" on: pull_request: diff --git a/.github/workflows/build-test.yml b/.github/workflows/test.yml similarity index 95% rename from .github/workflows/build-test.yml rename to .github/workflows/test.yml index 4ff700b6..3dc209f7 100644 --- a/.github/workflows/build-test.yml +++ b/.github/workflows/test.yml @@ -1,4 +1,4 @@ -name: Build Test +name: "[CI] Tests" on: pull_request:

diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 3dc209f7..58767e3a 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -20,7 +20,7 @@ on: schedule: - cron: 0 12 1 * * - workflow_dispatch: null + workflow_dispatch: concurrency: group: 'build-test-${{ github.head_ref || github.run_id }}' @@ -58,7 +58,7 @@ jobs: uses: actions/setup-python@v4 with: python-version: '${{ matrix.python-version }}' - architecture: '${{ matrix.python-architecture || }}' + architecture: '${{ matrix.python-architecture }}' - name: Build run: 'python -m pip install -v .[test,dev]'

This reverts commit a74985b.

agoose77 changed the title ~~WIP: trial CI for GHA~~ Chore: use GHA for buildtest Jul 16, 2022

agoose77 changed the title ~~Chore: use GHA for buildtest~~ Chore: use GHA for buildtest 🔨 Jul 18, 2022

agoose77 marked this pull request as ready for review July 18, 2022 10:24

henryiii reviewed Jul 18, 2022

View reviewed changes

.github/workflows/build-test.yml Outdated Show resolved Hide resolved

henryiii reviewed Jul 18, 2022

View reviewed changes

.github/workflows/build-test.yml Outdated Show resolved Hide resolved

henryiii reviewed Jul 18, 2022

View reviewed changes

.github/workflows/build-test.yml Outdated Show resolved Hide resolved

agoose77 changed the title ~~Chore: use GHA for buildtest 🔨~~ CI: use GHA for buildtest 🔨 Jul 26, 2022

agoose77 changed the title ~~CI: use GHA for buildtest 🔨~~ CI: use GHA instead of Azure 🔨 Aug 17, 2022

henryiii mentioned this pull request Aug 18, 2022

chore: include Python 3.11 #1602

Merged

agoose77 changed the title ~~CI: use GHA instead of Azure 🔨~~ ci: use GHA instead of Azure 🔨 Aug 19, 2022

agoose77 added 6 commits August 19, 2022 22:30

Chore: drop buildtest CI for Azure

c06b25c

agoose77 and others added 20 commits August 19, 2022 22:30

Update .github/workflows/build-test.yml

86acc89

Co-authored-by: Henry Schreiner <henry.fredrick.schreiner@cern.ch>

Docs: update badge in README

cb80e77

Revert "Chore: rename tests for clarity"

917acee

This reverts commit a74985b.

Removed doctest, so there are no more Azure tests.

29d7420

agoose77 force-pushed the chore-use-gha branch from 9df9683 to 29d7420 Compare August 19, 2022 21:30

Merge branch 'main' into chore-use-gha

b0f834f

jpivarski merged commit b1f7792 into main Aug 20, 2022

jpivarski deleted the chore-use-gha branch August 20, 2022 16:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci: use GHA instead of Azure 🔨 #1550

ci: use GHA instead of Azure 🔨 #1550

agoose77 commented Jul 16, 2022 •

edited

Loading

codecov bot commented Jul 16, 2022 •

edited

Loading

agoose77 commented Jul 18, 2022

jpivarski commented Jul 18, 2022

henryiii commented Jul 18, 2022

agoose77 commented Jul 18, 2022

agoose77 commented Jul 19, 2022 •

edited

Loading

agoose77 commented Jul 19, 2022

jpivarski commented Jul 19, 2022

martindurant commented Jul 19, 2022

jpivarski commented Jul 19, 2022

jpivarski commented Aug 10, 2022

jpivarski commented Aug 17, 2022

agoose77 commented Aug 17, 2022 •

edited

Loading

jpivarski commented Aug 19, 2022

ci: use GHA instead of Azure 🔨 #1550

ci: use GHA instead of Azure 🔨 #1550

Conversation

agoose77 commented Jul 16, 2022 • edited Loading

codecov bot commented Jul 16, 2022 • edited Loading

Codecov Report

agoose77 commented Jul 18, 2022

jpivarski commented Jul 18, 2022

henryiii commented Jul 18, 2022

agoose77 commented Jul 18, 2022

agoose77 commented Jul 19, 2022 • edited Loading

agoose77 commented Jul 19, 2022

jpivarski commented Jul 19, 2022

martindurant commented Jul 19, 2022

jpivarski commented Jul 19, 2022

jpivarski commented Aug 10, 2022

jpivarski commented Aug 17, 2022

agoose77 commented Aug 17, 2022 • edited Loading

jpivarski commented Aug 19, 2022

agoose77 commented Jul 16, 2022 •

edited

Loading

codecov bot commented Jul 16, 2022 •

edited

Loading

agoose77 commented Jul 19, 2022 •

edited

Loading

agoose77 commented Aug 17, 2022 •

edited

Loading