Skip to content

[CI] Add GPU reset for BMG #17381

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sarnex merged 2 commits into from
Mar 13, 2025
Merged

[CI] Add GPU reset for BMG #17381

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
00fa6e6
[CI] Add GPU reset for BMG
sarnex Mar 10, 2025
1052d94
disable tests
sarnex Mar 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions .github/workflows/sycl-linux-precommit.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,13 +112,12 @@ jobs:
runner: '["Linux", "bmg"]'
image_options: -u 1001 --device=/dev/dri -v /dev/dri/by-path:/dev/dri/by-path --privileged --cap-add SYS_ADMIN
target_devices: level_zero:gpu
# The new Xe kernel driver used by BMG doesn't support resetting.
reset_intel_gpu: false
reset_intel_gpu: true
- name: SPIR-V Backend / Intel Battlemage Graphics
runner: '["Linux", "bmg"]'
image_options: -u 1001 --device=/dev/dri -v /dev/dri/by-path:/dev/dri/by-path --privileged --cap-add SYS_ADMIN
target_devices: level_zero:gpu;opencl:gpu;opencl:cpu
reset_intel_gpu: false
reset_intel_gpu: true
extra_lit_opts: --param spirv-backend=True
e2e_binaries_artifact: sycl_e2e_bin_default_spirv_backend
uses: ./.github/workflows/sycl-linux-run-tests.yml
Expand Down
21 changes: 14 additions & 7 deletions .github/workflows/sycl-linux-run-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -203,15 +203,22 @@ jobs:
steps:
- name: Reset Intel GPU
if: inputs.reset_intel_gpu == 'true'
shell: bash
run: |
sudo mount -t debugfs none /sys/kernel/debug
base_dir="/sys/kernel/debug/dri"
if [[ '${{ inputs.runner }}' == '["Linux", "bmg"]' ]]; then
sudo bash -c 'echo 0000:05:00.0 > /sys/bus/pci/drivers/xe/unbind'
sudo bash -c 'echo 1 > /sys/bus/pci/devices/0000:05:00.0/reset'
sudo bash -c 'echo 0000:05:00.0 > /sys/bus/pci/drivers/xe/bind'
else
sudo mount -t debugfs none /sys/kernel/debug
base_dir="/sys/kernel/debug/dri"

for dir in "$base_dir"/*; do
if [ -f "$dir/i915_wedged" ]; then
sudo bash -c 'echo 1 > $0/i915_wedged' $dir
fi
done
for dir in "$base_dir"/*; do
if [ -f "$dir/i915_wedged" ]; then
sudo bash -c 'echo 1 > $0/i915_wedged' $dir
fi
done
fi
- uses: actions/checkout@v4
with:
ref: ${{ inputs.devops_ref || inputs.repo_ref }}
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/sycl-nightly.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@ jobs:
runner: '["Linux", "bmg"]'
image_options: -u 1001 --device=/dev/dri -v /dev/dri/by-path:/dev/dri/by-path --privileged --cap-add SYS_ADMIN
target_devices: level_zero:gpu
reset_intel_gpu: true

- name: Intel L0 Arc A-Series GPU
runner: '["Linux", "arc"]'
Expand Down
3 changes: 3 additions & 0 deletions sycl/test-e2e/KernelAndProgram/persistent-cache-multi-device.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@
// XFAIL: spirv-backend && run-mode
// XFAIL-TRACKER: CMPLRLLVM-64705

// XFAIL: linux && arch-intel_gpu_bmg_g21 && !igc-dev && run-mode
Copy link
Contributor

@uditagarwal97 uditagarwal97 Mar 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Weird that these tests are failing due to GPU reset.

Copy link
Contributor Author

@sarnex sarnex Mar 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah its consistent too, i have absolutely no idea whats going on

Copy link
Contributor

@uditagarwal97 uditagarwal97 Mar 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me look into this.

// XFAIL-TRACKER: https://github.com/intel/llvm/issues/17453

// Test checks that persistent cache works correctly with multiple devices.

#include <sycl/detail/core.hpp>
Expand Down
3 changes: 3 additions & 0 deletions sycl/test-e2e/ProgramManager/multi_device_bundle/build_twice.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@
// XFAIL: spirv-backend && run-mode
// XFAIL-TRACKER: CMPLRLLVM-64705

// XFAIL: linux && arch-intel_gpu_bmg_g21 && !igc-dev && run-mode
// XFAIL-TRACKER: https://github.com/intel/llvm/issues/17453

#include <sycl/detail/core.hpp>
#include <sycl/kernel_bundle.hpp>

Expand Down
3 changes: 3 additions & 0 deletions sycl/test-e2e/ProgramManager/multi_device_bundle/device_libs_and_caching.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,9 @@
// XFAIL: spirv-backend && run-mode
// XFAIL-TRACKER: CMPLRLLVM-64705

// XFAIL: linux && arch-intel_gpu_bmg_g21 && !igc-dev && run-mode
// XFAIL-TRACKER: https://github.com/intel/llvm/issues/17453

#include <cmath>
#include <complex>
#include <sycl/detail/core.hpp>
Expand Down
Loading