Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adding tagged builds for M1 #6140

Merged
merged 7 commits into from
Jun 9, 2022
Merged

Adding tagged builds for M1 #6140

merged 7 commits into from
Jun 9, 2022

Conversation

atalman
Copy link
Contributor

@atalman atalman commented Jun 8, 2022

Adding tagged builds for M1

@atalman atalman marked this pull request as draft June 8, 2022 21:41
@atalman atalman force-pushed the tagged_build_m1 branch from 994baf3 to 5eda8fd Compare June 8, 2022 23:17
malfet
malfet reviewed Jun 9, 2022
View reviewed changes
.github/workflows/build-m1-binaries.yml
shell: arch -arch arm64 bash {0}
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_PYTORCH_UPLOADER_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_PYTORCH_UPLOADER_SECRET_ACCESS_KEY }}
CHANNEL: nightly
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Err, this will fail for nightly builds, isn't it (I.e. Set CHANNEL (only for tagged pushes) would only set it to test, but what will set it to nightly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please see comment below we do set CHANNEL nightly by default

.github/workflows/build-m1-binaries.yml
@@ -113,9 +143,8 @@ jobs:
shell: arch -arch arm64 bash {0}
env:
CONDA_PYTORCHBOT_TOKEN: ${{ secrets.CONDA_PYTORCHBOT_TOKEN }}
UPLOAD_CHANNEL: nightly
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above, what will set a channel environment variable to nightly for regular builds?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please see comment below we do set CHANNEL nightly by default

packaging/pkg_helpers.bash
fi
export BUILD_VERSION="$_VERSION_BASE.dev$(date "+%Y%m%d")$VERSION_SUFFIX"
BUILD_VERSION="$BUILD_VERSION.dev$(date "+%Y%m%d")$VERSION_SUFFIX"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not a bash expert, doesn't one need to re-export it here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes totally its re-exported at line 119 below

atalman
atalman commented Jun 9, 2022
View reviewed changes
.github/workflows/build-m1-binaries.yml
workflow_dispatch:
env:
CHANNEL: "nightly"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is where we set default CHANNEL to nightly

This was referenced Jun 9, 2022
Closed
Closed
@datumbox
Copy link
Contributor

datumbox commented Jun 9, 2022

@atalman @malfet FYI, it seems that the unit-tests are failing on M1. We haven't been able to pinpoint the Core commit that broke it but it looks less likely we will be able to release M1 binaries for TorchVision. We've added all the details that we know so far at pytorch/pytorch#79421. Please let us know what you think.

@atalman atalman marked this pull request as ready for review June 9, 2022 12:44
@atalman atalman merged commit efc67ea into pytorch:main Jun 9, 2022
@github-actions
Copy link

github-actions bot commented Jun 9, 2022

Hey @atalman!

You merged this PR, but no labels were added. The list of valid labels is available at https://github.com/pytorch/vision/blob/main/.github/process_commit.py

This was referenced Jun 10, 2022
Closed
Merged
facebook-github-bot pushed a commit to pytorch/audio that referenced this pull request Jun 10, 2022
@atalman @facebook-github-bot
Adding tagged builds to torchaudio (#2471)
19d9328 
Summary:
Adding tagged builds for torchaudio
see: pytorch/vision#6140

Pull Request resolved: #2471

Reviewed By: hwangjeff

Differential Revision: D37080828

Pulled By: atalman

fbshipit-source-id: 13d754f522510514f0148ba465ce12a320058722
NicolasHug pushed a commit to YosuaMichael/vision that referenced this pull request Jun 13, 2022
@atalman @NicolasHug
Adding tagged builds for M1 (pytorch#6140)
434e02a 
* Adding tagged builds

* Testing

* Testing

* Testing

* Testing

* Adding conda builds
atalman added a commit to atalman/audio that referenced this pull request Jun 13, 2022
@atalman
Adding tagged builds to torchaudio (pytorch#2471)
2e17fd0 
Summary:
Adding tagged builds for torchaudio
see: pytorch/vision#6140

Pull Request resolved: pytorch#2471

Reviewed By: hwangjeff

Differential Revision: D37080828

Pulled By: atalman

fbshipit-source-id: 13d754f522510514f0148ba465ce12a320058722
@atalman atalman mentioned this pull request Jun 13, 2022
Merged
atalman added a commit to pytorch/audio that referenced this pull request Jun 13, 2022
@atalman
Adding tagged builds to torchaudio (#2471) (#2475)
ff034cb 
Summary:
Adding tagged builds for torchaudio
see: pytorch/vision#6140

Pull Request resolved: #2471

Reviewed By: hwangjeff

Differential Revision: D37080828

Pulled By: atalman

fbshipit-source-id: 13d754f522510514f0148ba465ce12a320058722
atalman added a commit to atalman/vision that referenced this pull request Jun 14, 2022
@atalman
Adding tagged builds for M1 (pytorch#6140)
b1c33ed 
* Adding tagged builds

* Testing

* Testing

* Testing

* Testing

* Adding conda builds
NicolasHug added a commit that referenced this pull request Jun 15, 2022
@atalman @malfet @NicolasHug
M1 fixes cherry (#6167)
94c81c2 
* Add M1 wheels binary builds (#5948)

* [M1] Set build version and delocate wheels (#6110)

This would package libpng and libjpeg.dylib into wheel files
Add a very simple test step, copied from https://github.com/pytorch/pytorch.github.io/blob/1eaa33a3d3f1b83b64c5031c6dd04dbb238d6105/scripts/test_install.py#L78
Cherry-picked from https://github.com/pytorch/builder/blob/d0bc74cc363a9da5a8b6a40e883d40d25d050036/build_m1_domains.sh#L22

* [BE] Unify version computation (#6117)

* [BE] Unify version computation

Instead of hardcoding dev version in various script, use one from
`version.txt` if `setup_build_version` is called without arguments

Also, pass `--pre` option to M1 build/test pip install commands to build
TorchVision against nightly pytorch

* Pin torchvision dependency to a specific pytorch version

* [M1] Install "jpeg<=9b" rather than OpenJpeg (#6122)

Explicitly set PATH to point to `conda` binary, otherwise libjpeg detection logic does not work
Pin libjpeg to the same version on x86 and m1
Add simple tests that jpeg can be decoded by a generated wheel

* Add unit-tests for M1 (#6132)

* Add M1 testing job

* libjpeg -> jpeg<=9b in test-m1.yml

* Added export PATH=~/miniconda3/bin... from 6122

* Tests were OK, let's see if we can remove the pinning

* GH: Add M1 conda build workflows (#6135)

Clean up Conda build folder before every run
Enable artifact upload to GitHub for every workflow run, but upload to Conda/S3 only on nightly pushes

Test plan: `conda install -c pytorch-nightly torchvision; python -c "import torchvision;print(torchvision.io.read_image('hummingbird.jpg').shape)"`

* Fix `Test M1` workflow

By passing `--pre` option to `pip install`, otherwise torchvision were always tested against last PyTorch release

* Adding tagged builds for M1 (#6140)

* Adding tagged builds

* Testing

* Testing

* Testing

* Testing

* Adding conda builds

* Fix `if` condition for s3/conda uploads (#6146)

Replace `steps.extract_branch.outputs.branch` (which were probably taken from
https://stackoverflow.com/questions/58033366/how-to-get-the-current-branch-within-github-actions ) with straightforward `github.event.ref`

Tested in
https://github.com/malfet/deleteme/runs/6822647720?check_suite_focus=true and https://github.com/malfet/deleteme/runs/6822691158?check_suite_focus=true

* Fix typo in GHA nightly build condition (#6158)

s#ref/heads/#refs/heads/#

I should have noticed it while copy-n-pasting the condition. Unfortunately there are no way to test is other than in prod, but nightly builds are still not getting pushed, see https://github.com/pytorch/vision/runs/6860407007?check_suite_focus=true for example

* Making sure we are building against release

* Testing

Testing

Testing

Testing

testing

Testing

Testing

Testing

Testing

Testing

Testing

Testing

* Testing

* Testing

* Cleanup

* Refactoring logic

Co-authored-by: Nikita Shulga <nshulga@fb.com>
Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>
NicolasHug added a commit to NicolasHug/vision that referenced this pull request Jun 16, 2022
@NicolasHug @facebook-github-bot
[fbsync] Adding tagged builds for M1 (pytorch#6140)
6915159 
Summary:
* Adding tagged builds

* Testing

* Testing

* Testing

* Testing

* Adding conda builds

Differential Revision: D37212653

fbshipit-source-id: b5575a1a6ea4264e0254db66c9ae42f5b3507a18
facebook-github-bot pushed a commit that referenced this pull request Jun 17, 2022
@NicolasHug @facebook-github-bot
[fbsync] Adding tagged builds for M1 (#6140)
962f983 
Summary:
* Adding tagged builds

* Testing

* Testing

* Testing

* Testing

* Adding conda builds

Reviewed By: datumbox

Differential Revision: D37212653

fbshipit-source-id: 1623f4d510d55a50419921174e9c9b43fffaf7c2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@malfet malfet malfet approved these changes

Assignees
No one assigned
Labels
cla signed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@atalman @datumbox @malfet @facebook-github-bot