Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enable Windows GPU CI on GHA #7498

Merged
merged 19 commits into from
May 2, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 0 additions & 72 deletions .circleci/config.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

51 changes: 0 additions & 51 deletions .circleci/config.yml.in
Original file line number Diff line number Diff line change
Expand Up @@ -433,53 +433,6 @@ jobs:
docker push ${image_name}:${CIRCLE_WORKFLOW_ID}
docker push ${image_name}:latest

unittest_windows_gpu:
<<: *binary_common
executor:
name: windows-gpu
environment:
CUDA_VERSION: "11.7"
PYTHON_VERSION: << parameters.python_version >>
steps:
- checkout
- designate_upload_channel
- run:
name: Generate cache key
# This will refresh cache on Sundays, nightly build should generate new cache.
command: echo "$(date +"%Y-%U")" > .circleci-weekly
- restore_cache:
{% raw %}
keys:
- env-v1-windows-{{ arch }}-py<< parameters.python_version >>-{{ checksum ".circleci/unittest/windows/scripts/environment.yml" }}-{{ checksum ".circleci-weekly" }}
{% endraw %}
- run:
name: Setup
command: .circleci/unittest/windows/scripts/setup_env.sh
- save_cache:
{% raw %}
key: env-v1-windows-{{ arch }}-py<< parameters.python_version >>-{{ checksum ".circleci/unittest/windows/scripts/environment.yml" }}-{{ checksum ".circleci-weekly" }}
{% endraw %}
paths:
- conda
- env
- run:
name: Install CUDA
command: packaging/windows/internal/cuda_install.bat
- run:
name: Update CUDA driver
command: packaging/windows/internal/driver_update.bat
- run:
name: Install torchvision
command: .circleci/unittest/windows/scripts/install.sh
- run:
name: Run tests
command: .circleci/unittest/windows/scripts/run_test.sh
- run:
name: Post process
command: .circleci/unittest/windows/scripts/post_process.sh
- store_test_results:
path: test-results

cmake_linux_cpu:
<<: *binary_common
docker:
Expand Down Expand Up @@ -575,10 +528,6 @@ workflows:
{{ ios_workflows() }}
{{ android_workflows() }}

unittest:
jobs:
{{ unittest_workflows() }}

cmake:
jobs:
{{ cmake_workflows() }}
Expand Down
31 changes: 0 additions & 31 deletions .circleci/regenerate.py
Original file line number Diff line number Diff line change
Expand Up @@ -217,36 +217,6 @@ def indent(indentation, data_list):
return ("\n" + " " * indentation).join(yaml.dump(data_list, default_flow_style=False).splitlines())


def unittest_workflows(indentation=6):
jobs = []
for os_type in ["windows"]:
for device_type in ["gpu"]:
if os_type == "macos" and device_type == "gpu":
continue

for i, python_version in enumerate(PYTHON_VERSIONS):

# Turn off unit tests for 3.11, unit test are not setup properly
if python_version == "3.11":
continue

job = {
"name": f"unittest_{os_type}_{device_type}_py{python_version}",
"python_version": python_version,
}

if device_type == "gpu":
if python_version != "3.8":
job["filters"] = gen_filter_branch_tree("main", "nightly")
job["cu_version"] = "cu117"
else:
job["cu_version"] = "cpu"

jobs.append({f"unittest_{os_type}_{device_type}": job})

return indent(indentation, jobs)


def cmake_workflows(indentation=6):
jobs = []
python_version = "3.8"
Expand Down Expand Up @@ -331,7 +301,6 @@ def android_workflows(indentation=6, nightly=False):
f.write(
env.get_template("config.yml.in").render(
build_workflows=build_workflows,
unittest_workflows=unittest_workflows,
cmake_workflows=cmake_workflows,
ios_workflows=ios_workflows,
android_workflows=android_workflows,
Expand Down
12 changes: 6 additions & 6 deletions .github/workflows/test-windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,11 @@ jobs:
- "3.11"
runner: ["windows.4xlarge"]
gpu-arch-type: ["cpu"]
# FIXME: enable this as soon as nvjpeg is available on the Windows runner
# include:
# - python-version: "3.8"
# runner: windows.8xlarge.nvidia.gpu
# gpu-arch-type: cuda
# gpu-arch-version: "11.7"
include:
- python-version: "3.8"
runner: windows.g5.4xlarge.nvidia.gpu
gpu-arch-type: cuda
gpu-arch-version: "11.7"
fail-fast: false
uses: pytorch/test-infra/.github/workflows/windows_job.yml@main
with:
Expand All @@ -46,6 +45,7 @@ jobs:
# TODO: This should be handled by the generic Windows job the same as its done by the generic Linux job
export CUDA_HOME="/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v${{ matrix.gpu-arch-version }}"
export CUDA_PATH="${CUDA_HOME}"
export PATH="${CUDA_PATH}/bin:${PATH}"
fi

./.github/scripts/unittest.sh