Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Hendrix for CI. #1862

Merged
merged 52 commits into from
Feb 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
29f9fd6
First experiment at using Hendrix for CI.
athas Jan 31, 2023
915f155
Maybe like this.
athas Jan 31, 2023
5c79394
Import everything locally.
athas Jan 31, 2023
1b3b5e5
Merge branch 'master' into hendrixci
athas Jan 31, 2023
4d23093
Try this.
athas Jan 31, 2023
a754173
More systems.
athas Jan 31, 2023
f61ecd0
Also OpenCL.
athas Jan 31, 2023
caf766d
Also depend on these.
athas Jan 31, 2023
c301eb9
More readable when split.
athas Feb 1, 2023
a9a79fd
Import new CI actions.
athas Feb 7, 2023
11f4d84
Merge branch 'master' into hendrixci
athas Feb 7, 2023
48b9adc
Testing with slurm.
WilliamDue Feb 12, 2023
8feaa13
Forgot to specify hendrix and the partition flag might also be needed.
WilliamDue Feb 12, 2023
cd149c6
The wrong composite actions was included
WilliamDue Feb 12, 2023
b888269
Trying cuda and opencl on hendrix
WilliamDue Feb 12, 2023
326f4ef
Trying to use the composite test action for benchmarks.
WilliamDue Feb 12, 2023
951767b
Wrong amount of indentation
WilliamDue Feb 12, 2023
9ea962f
Forgot to add a |.
WilliamDue Feb 12, 2023
0920e49
Some small changes that will most likely not change things.
WilliamDue Feb 12, 2023
3c47643
trying to use sbatch
WilliamDue Feb 13, 2023
83ed05e
Merge branch 'master' into hendrixci
WilliamDue Feb 13, 2023
c30a9b2
switching to titanrtx and used the p flag wrong.
WilliamDue Feb 13, 2023
27b5b25
Merge branch 'master' into hendrixci
athas Feb 13, 2023
78b99a4
Trailing whitespace purge.
athas Feb 13, 2023
5e9ce8e
Skip these on TITAN X.
athas Feb 13, 2023
9dea305
Any GPU will work for these.
athas Feb 13, 2023
1bcd47f
Trying to run benchmarks without slurmbench.py
WilliamDue Feb 13, 2023
9bbfb13
Syntax errors
WilliamDue Feb 13, 2023
3b06369
Accidentally used old keyword test.
WilliamDue Feb 13, 2023
bd80dd0
found another syntax error i think
WilliamDue Feb 13, 2023
d20e700
I think the equality sign broke it
WilliamDue Feb 13, 2023
51d12fd
maybe this will work
WilliamDue Feb 13, 2023
0dac0fc
Used gres wrongly.
WilliamDue Feb 13, 2023
a3c6276
Merge branch 'master' into hendrixci
athas Feb 14, 2023
1392be3
Merge remote-tracking branch 'origin/hendrixci' into hendrixci
athas Feb 14, 2023
81ca1ab
Do not use old futhark-benchmarks.
athas Feb 14, 2023
6c4111f
Trying to use srun and cleaned up composite actions.
WilliamDue Feb 18, 2023
4d9a1bc
Add some comments.
athas Feb 14, 2023
5a0bf31
More explicit.
athas Feb 14, 2023
f8c6249
Fix #1878.
athas Feb 15, 2023
f76475b
Forbid access to interpreter.
athas Feb 15, 2023
7bb6e66
Ensure no apply-of-apply.
athas Feb 16, 2023
c9899b8
Revert "Trying to use srun and cleaned up composite actions."
WilliamDue Feb 19, 2023
6bfb384
Merge remote-tracking branch 'origin/master' into hendrixci
WilliamDue Feb 19, 2023
cd631ec
using srun and fixing commit history hopefully?
WilliamDue Feb 19, 2023
6eb2593
Adding an 8 hour time limit.
WilliamDue Feb 19, 2023
9f7008e
Missing -.
WilliamDue Feb 19, 2023
965e788
Newer version og futhark-benchmarks
WilliamDue Feb 19, 2023
77f3e45
Trying to use `${{ always() }}`.
WilliamDue Feb 19, 2023
f6a9e28
Revert "Newer version og futhark-benchmarks" because of `${{ always()…
WilliamDue Feb 19, 2023
82a1e39
Hopefully this is the correct version of the futhark-benchmarks
WilliamDue Feb 19, 2023
e253ac8
Remove always()
WilliamDue Feb 19, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 20 additions & 20 deletions .github/actions/benchmark/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,34 +7,34 @@ inputs:
system:
description: 'Name of system (e.g. GPU name)'
required: true
options:
futhark-options:
description: 'Options to pass to futhark bench'
required: false
default: ''
slurm-options:
description: 'Options to pass to srun'
required: false
default: ''
runs:
using: "composite"
steps:
- uses: actions/download-artifact@v3
with:
name: futhark-nightly-linux-x86_64.tar.xz
- name: Setup compiler
shell: bash
run: |
tar xvf futhark-nightly-linux-x86_64.tar.xz
echo "$PWD/futhark-nightly-linux-x86_64/bin" >> $GITHUB_PATH
- name: Get data
- name: Download Benchmarks.
shell: bash
run: |
cd futhark-benchmarks && pwd && ./get-data.sh external-data.txt
- name: Debug
shell: bash
run: |
env
- name: Run benchmarks
shell: bash
run: |
futhark bench futhark-benchmarks --ignore-files /lib/ --backend=${{inputs.backend}} --json futhark-${{inputs.backend}}-${{inputs.system}}-$GITHUB_SHA.json --exclude no_${{inputs.system}} ${{inputs.options}}
cd futhark-benchmarks
./get-data.sh external-data.txt

- uses: ./.github/actions/futhark-slurm
with:
script: |
futhark bench futhark-benchmarks \
--backend ${{inputs.backend}} \
--exclude no_${{inputs.system}} \
--json futhark-${{inputs.backend}}-${{inputs.system}}-$GITHUB_SHA.json \
--ignore-files /lib/ ${{inputs.futhark-options}}
slurm-options: ${{inputs.slurm-options}}

- uses: actions/upload-artifact@v1
with:
name: futhark-${{inputs.backend}}-${{inputs.system}}-${{ github.sha }}.json
path: futhark-${{inputs.backend}}-${{inputs.system}}-${{ github.sha }}.json
path: futhark-${{inputs.backend}}-${{inputs.system}}-${{ github.sha }}.json
40 changes: 40 additions & 0 deletions .github/actions/futhark-slurm/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: 'Futhark Script'
description: 'Run script on slurm if available where Futhark will be available.'
inputs:
script:
description: 'Script to run'
required: true
slurm-options:
description: 'Options to pass to srun'
required: false
default: ''
runs:
using: "composite"
steps:
- uses: actions/download-artifact@v3
with:
name: futhark-nightly-linux-x86_64.tar.xz

- name: Install from nightly tarball
shell: bash
run: |
tar xvf futhark-nightly-linux-x86_64.tar.xz
make -C futhark-nightly-linux-x86_64/ install PREFIX=$HOME/.local
echo "$HOME/.local/bin" >> $GITHUB_PATH

- uses: ./.github/actions/is-slurm
id: slurm

- if: steps.slurm.outputs.is-slurm == 'false'
shell: bash
run: |
${{inputs.script}}

- if: steps.slurm.outputs.is-slurm == 'true'
shell: bash
run: |
printf '#!/bin/bash
${{inputs.script}}' > temp.sh
chmod +x temp.sh
srun --time=0-08:00:00 ${{inputs.slurm-options}} temp.sh
rm temp.sh
21 changes: 21 additions & 0 deletions .github/actions/is-slurm/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
name: 'Is Slurm Installed'
description: 'Checks if slurm is installed by checking if srun can be used.'

outputs:
is-slurm:
description: "If slurm is used."
value: ${{ steps.slurm.outputs.is-slurm }}

runs:
using: "composite"
steps:
- name: Check if slurm can be run
id: slurm
shell: bash
run: |
printf '#!/bin/bash
if ! srun --version &> /dev/null; then
echo "is-slurm=false"
else
echo "is-slurm=true"
fi' | bash >> $GITHUB_OUTPUT
Loading