Skip to content

add macOS unittest to GHA #7376

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 24 commits into from
Mar 8, 2023
Merged

add macOS unittest to GHA #7376

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
beed393
add macOS unittest to GHA
pmeier Mar 1, 2023
cdb2697
trigger CI
pmeier Mar 1, 2023
3b89814
fix version
pmeier Mar 1, 2023
773412d
increase timeout
pmeier Mar 2, 2023
5da3f42
add debug workflow
pmeier Mar 2, 2023
19402ef
fix install command
pmeier Mar 2, 2023
5e2253d
add tmate session
pmeier Mar 2, 2023
5fb92ff
always setup tmate
pmeier Mar 2, 2023
03ba33b
add jpeg debug and delocate
pmeier Mar 2, 2023
6a10cc6
try regular wheel build with libjpeg and libpng from conda
pmeier Mar 2, 2023
8925969
trigger CI
pmeier Mar 2, 2023
94db606
fix test infra ref
pmeier Mar 2, 2023
8b8bc87
trigger CI
pmeier Mar 2, 2023
90210c1
try uninstalling system jpeg libs on macos
pmeier Mar 2, 2023
893df7b
fix conditional
pmeier Mar 3, 2023
f76c795
try without delocate
pmeier Mar 3, 2023
da828de
revert changes on build workflow
pmeier Mar 3, 2023
914a1e0
Merge branch 'main' into macos-gha
pmeier Mar 3, 2023
c46fb06
add debug for M1
pmeier Mar 3, 2023
140f383
only uninstall jpeg libs on x86 macos runners
pmeier Mar 3, 2023
b265117
remove debug workflow
pmeier Mar 3, 2023
4145d70
wrap uninstalling into a group
pmeier Mar 6, 2023
68049f0
Merge branch 'main' into macos-gha
pmeier Mar 6, 2023
5b32e26
Merge branch 'main' into macos-gha
pmeier Mar 8, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions .github/unittest.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,32 @@ CONDA_PATH=$(which conda)
eval "$(${CONDA_PATH} shell.bash hook)"
conda config --set channel_priority strict

# Setup the OS_TYPE environment variable that should be used for conditions involving the OS below.
case $(uname) in
Linux)
OS_TYPE=linux
;;
Darwin)
OS_TYPE=macos
;;
*)
echo "Unknown OS type:" $(uname)
exit 1
;;
esac

echo '::group::Uninstall system JPEG libraries on macOS'
# The x86 macOS runners, e.g. the GitHub Actions native "macos-12" runner, has some JPEG libraries installed by default
# that interfere with our build. We uninstall them here and use the one from conda below.
if [[ "${OS_TYPE}" == "macos" && $(uname -m) == x86_64 ]]; then
JPEG_LIBS=$(brew list | grep jpeg)
echo $JPEG_LIBS
for lib in $JPEG_LIBS; do
brew uninstall --ignore-dependencies --force $lib || true
done
fi
Comment on lines +25 to +33
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are doing the same for the macOS wheel workflows:

vision/packaging/pre_build_script.sh

Lines 3 to 9 in 0e62c34

# Uninstall Conflicting jpeg brew formulae
jpeg_packages=$(brew list | grep jpeg)
echo "Existing Jpeg-related Brew libraries"
echo $jpeg_packages
for pkg in $jpeg_packages; do
brew uninstall --ignore-dependencies --force $pkg || true
done

echo '::endgroup::'

echo '::group::Set PyTorch conda channel and wheel index'
# TODO: Can we maybe have this as environment variable in the job template? For example, `IS_RELEASE`.
if [[ (${GITHUB_EVENT_NAME} = 'pull_request' && (${GITHUB_BASE_REF} = 'release'*)) || (${GITHUB_REF} = 'refs/heads/release'*) ]]; then
Expand Down
50 changes: 0 additions & 50 deletions .github/workflows/test-m1.yml
View file
Open in desktop

This file was deleted.

37 changes: 37 additions & 0 deletions .github/workflows/test-macos.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name: Unit-tests on macOS

on:
pull_request:
push:
branches:
- nightly
- main
- release/*
workflow_dispatch:

jobs:
tests:
strategy:
matrix:
python-version:
- "3.8"
- "3.9"
- "3.10"
- "3.11"
runner: ["macos-12"]
include:
- python-version: "3.8"
runner: macos-m1-12
fail-fast: false
uses: pytorch/test-infra/.github/workflows/macos_job.yml@main
with:
repository: pytorch/vision
# We need an increased timeout here, since the macos-12 runner is the free one from GH
# and needs roughly 2 hours to just run the test suite
timeout: 240
runner: ${{ matrix.runner }}
script: |
export PYTHON_VERSION=${{ matrix.python-version }}
export GPU_ARCH_TYPE=cpu

./.github/unittest.sh
2 changes: 2 additions & 0 deletions setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -298,6 +298,8 @@ def get_extensions():
use_jpeg = use_jpeg and jpeg_found
if use_jpeg:
print("Building torchvision with JPEG image support")
print(f" libpng include path: {jpeg_include}")
print(f" libpng lib path: {jpeg_lib}")
Comment on lines +301 to +302
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small drive-by that helped me debug. We do the same for PNG already:

vision/setup.py

Lines 278 to 279 in 0e62c34

print(f" libpng version: {png_version}")
print(f" libpng include path: {png_include}")

I wasn't able to get the version of the library in a way that would hold for all operating systems. I guess knowing the location is good enough.

image_link_flags.append("jpeg")
if jpeg_conda:
image_library += [jpeg_lib]
Expand Down