Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update MacOS runner to Ventura, add MacOS Sonoma (M1) runner #4393

Merged
merged 5 commits into from
Feb 29, 2024
Merged

Update MacOS runner to Ventura, add MacOS Sonoma (M1) runner #4393

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
93db2ce
Use M1 Github runner.
fruffy Feb 5, 2024
8e8d361
Update ci-test and remove elements which have no effect.
fruffy Feb 27, 2024
df541a3
Add a cache.
fruffy Feb 27, 2024
173ce55
Fun with GPTs.
fruffy Feb 28, 2024
4d3755e
Cleanup.
fruffy Feb 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 53 additions & 8 deletions .github/workflows/ci-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,6 @@ concurrency:
jobs:
# Build with gcc and test p4c on Ubuntu 22.04.
test-ubuntu22:
strategy:
fail-fast: false
runs-on: ubuntu-22.04
env:
CTEST_PARALLEL_LEVEL: 4
Expand Down Expand Up @@ -79,8 +77,6 @@ jobs:

# Build and test p4c on Fedora.
test-fedora-linux:
strategy:
fail-fast: false
# This job runs in Fedora container that runs in Ubuntu VM.
runs-on: ubuntu-latest
container:
Expand Down Expand Up @@ -116,11 +112,51 @@ jobs:
run: sudo -E ctest --output-on-failure --schedule-random
working-directory: ./build

# Build and test p4c on MacOS 12
# Build and test p4c on MacOS for M1 Macs.
test-mac-os-m1:
runs-on: macos-14
env:
CTEST_PARALLEL_LEVEL: 4
steps:
- uses: actions/checkout@v4
with:
submodules: recursive

- name: ccache
uses: hendrikmuhs/ccache-action@v1
with:
key: test-${{ runner.os }}
max-size: 1000M

- name: Cache Homebrew Packages
id: cache-homebrew-packages
uses: actions/cache@v4
env:
cache-name: homebrew-packages
with:
path: $(brew --prefix)
key: ${{ runner.os }}-${{ hashFiles('tools/install_mac_deps.sh') }}

- name: Install dependencies (MacOS)
fruffy marked this conversation as resolved.
Show resolved Hide resolved
run: |
tools/install_mac_deps.sh

- name: Build (MacOS)
run: |
source ~/.bash_profile
./bootstrap.sh -DENABLE_GC=ON -DCMAKE_BUILD_TYPE=RELEASE \
-DCMAKE_UNITY_BUILD=ON
make -Cbuild -j$((`nproc`+1))

- name: Run tests (MacOS)
run: |
source ~/.bash_profile
ctest --output-on-failure --schedule-random -LE "bpf|ubpf"
working-directory: ./build

# Build and test p4c on MacOS 13 on x86.
test-mac-os:
strategy:
fail-fast: false
runs-on: macos-12
runs-on: macos-13
env:
CTEST_PARALLEL_LEVEL: 4
steps:
Expand All @@ -134,6 +170,15 @@ jobs:
key: test-${{ runner.os }}
max-size: 1000M

- name: Cache Homebrew Packages
id: cache-homebrew-packages
uses: actions/cache@v4
env:
cache-name: homebrew-packages
with:
path: $(brew --prefix)
key: ${{ runner.os }}-${{ hashFiles('tools/install_mac_deps.sh') }}

- name: Install dependencies (MacOS)
run: |
tools/install_mac_deps.sh
Expand Down
2 changes: 1 addition & 1 deletion requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ clang-format==15.0.6
black==22.3.0
isort==5.10.0
protobuf==3.20.2; python_version > '3.6'
grpcio==1.51.1; python_version > '3.6'
grpcio==1.59.3; python_version > '3.6'
googleapis-common-protos==1.53.0; python_version > '3.6'
# Ubuntu 18.04 uses Python 3.6, which is not supported by recent versions of Protobuf.
protobuf==3.19.2; python_version <= '3.6'
Expand Down
50 changes: 29 additions & 21 deletions tools/install_mac_deps.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,30 +5,38 @@
set -e # Exit on error.
set -x # Make command execution verbose

# Install some custom requirements on OS X using brew
BREW=/usr/local/bin/brew
if [[ ! -x $BREW ]]; then
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
fi

BOOST_LIB="boost@1.76"
# Set up Homebrew differently for arm64.
if [[ $(uname -m) == 'arm64' ]]; then
fruffy marked this conversation as resolved.
Show resolved Hide resolved
(echo; echo 'eval "$(/opt/homebrew/bin/brew shellenv)"') >> ~/.zprofile
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it a clean environment each time CI runs? I assume it is, but asking in case we need to check for the eval line before adding it to .zprofile.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have a suggestion for a cleaner command to set up brew? I do not have a Mac so I am just pasting what people suggest.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally, one should be able to run this command locally, too.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems reasonable to me.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Used ChatGPT to "enhance" the script with some guards, please take another look.

eval "$(/opt/homebrew/bin/brew shellenv)"
else
(echo; echo 'eval "$(/usr/local/bin/brew shellenv)"') >> ~/.zprofile
eval "$(/usr/local/bin/brew shellenv)"
fi

$BREW install autoconf automake bdw-gc ccache cmake \
libtool openssl pkg-config python coreutils
if [[ ! -x brew ]]; then
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
fi
HOMEBREW_PREFIX=$(brew --prefix)

# We need to link boost.
$BREW install ${BOOST_LIB}
$BREW link ${BOOST_LIB}
# Prefer Homebrew's bison and grep over the macOS-provided version
$BREW install bison
echo 'export PATH="/usr/local/opt/bison/bin:$PATH"' >> ~/.bash_profile
$BREW install grep
echo 'export PATH="/usr/local/opt/grep/libexec/gnubin:$PATH"' >> ~/.bash_profile
# Install some custom requirements on OS X using brew
BOOST_LIB="boost@1.84"
brew install autoconf automake bdw-gc ccache cmake \
libtool openssl pkg-config coreutils bison grep \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's about flex? I remember macos shipped some ancient version by default, I do not recall the present status (esp. on builders)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Ventura laptop I'm having:

/usr/bin/flex --version
flex 2.6.4 Apple(flex-34)

This seems to be enough, ok.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh, I have an older version on Sonoma (14.3.1):

flex 2.5.35 Apple(flex-32)

Copy link
Contributor

@asl asl Feb 27, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably depends on installed xcode / command line tools as well

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, should I add a custom version? I believe in any case flex should be new enough.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the default version is fine for recent versions of MacOS. I've built it locally on my system in the past with 2.5.35.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, default is fine now.

${BOOST_LIB}

# We need to link boost and openssl.
brew link ${BOOST_LIB} openssl
# Prefer Homebrew's bison and grep over the macOS-provided version.
# For Bison only `$(brew --prefix bison)/bin` seems to work...
echo 'export PATH="$(brew --prefix bison)/bin:$PATH"' >> ~/.bash_profile
echo 'export PATH="$HOMEBREW_PREFIX/opt/grep/libexec/gnubin:$PATH"' >> ~/.bash_profile
source ~/.bash_profile


# install pip and required pip packages
# curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
# python get-pip.py --user
# use scapy 2.4.5, which is the version on which ptf depends
pip3 install --user scapy==2.4.5 ply==3.8
# Fixes for stuck grpcio installation.
export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1
export GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1
# Install required pip packages
pip3 install --user -r requirements.txt
Loading