Skip to content

refactor(ci): update graal release infrastructure #1966

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ckipp01 merged 2 commits into from
Dec 27, 2022
Merged

refactor(ci): update graal release infrastructure #1966

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a087b68
refactor(ci): update graal release infrastructure
ckipp01 Dec 27, 2022
d696bed
fix: ensure we don't run githubRelease on merge
ckipp01 Dec 27, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
200 changes: 92 additions & 108 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,54 +14,60 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: olafurpg/setup-scala@v13
- uses: graalvm/setup-graalvm@v1
with:
java-version: graalvm-ce-java11@21.1.0
version: '22.3.0'
java-version: '17'
github-token: ${{ secrets.GITHUB_TOKEN }}

- name: Check formatting
run: ./bin/scalafmt --test

integrations:
name: Build integrations tests
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macOS-latest]
jdk: [graalvm-ce-java11@21.1.0]
steps:
- uses: actions/checkout@v3
with:
submodules: true
- uses: olafurpg/setup-scala@v13
- uses: graalvm/setup-graalvm@v1
with:
java-version: ${{ matrix.jdk }}
version: '22.3.0'
java-version: '11'
github-token: ${{ secrets.GITHUB_TOKEN }}

- name: Tests
run: |
./bin/sbt-ci.sh \
"sbtBloop/publishLocal" \
"sbtBloop/scripted"
run: sbt "sbtBloop/publishLocal; sbtBloop/scripted"
shell: bash

bridges:
name: Test platform bridges
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macOS-latest]
jdk: [graalvm-ce-java11@21.1.0]
steps:
- uses: actions/checkout@v2
with:
submodules: true
- uses: olafurpg/setup-scala@v13
- uses: graalvm/setup-graalvm@v1
with:
java-version: ${{ matrix.jdk }}
version: '22.3.0'
java-version: '11'
github-token: ${{ secrets.GITHUB_TOKEN }}

- name: Tests
run: |
./bin/sbt-ci.sh \
"jsBridge06/publishLocal" \
"jsBridge1/publishLocal" \
"jsBridge06/test" \
"jsBridge1/test" \
sbt \
"jsBridge06/publishLocal; \
jsBridge1/publishLocal; \
jsBridge06/test; \
jsBridge1/test"
shell: bash

launcher:
Expand All @@ -71,19 +77,21 @@ jobs:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macOS-latest]
jdk: [graalvm-ce-java11@21.1.0]
steps:
- uses: actions/checkout@v2
with:
submodules: true
- uses: olafurpg/setup-scala@v13
- uses: graalvm/setup-graalvm@v1
with:
java-version: ${{ matrix.jdk }}
version: '22.3.0'
java-version: '11'
github-token: ${{ secrets.GITHUB_TOKEN }}
components: 'native-image'

- name: Tests
run: |
echo $JAVA_HOME
which gu && gu install native-image
./bin/sbt-ci.sh "install" "launcherTest/test"
sbt "install; launcherTest/test"
shell: bash

test:
Expand All @@ -92,18 +100,31 @@ jobs:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macOS-latest]
jdk: [graalvm-ce-java11@21.1.0, adopt@1.11]
jdk: ["22.3.0", "11"]

name: Test ${{ matrix.os }} -- ${{ matrix.jdk }}
env:
HYDRA_LICENSE: ${{ secrets.HYDRA_LICENSE }}

steps:
- uses: actions/checkout@v3
with:
submodules: true
fetch-depth: 0
- uses: olafurpg/setup-scala@v13

- uses: actions/setup-java@v3
with:
distribution: 'temurin'
java-version: ${{ matrix.jdk }}
cache: 'sbt'
if: matrix.jdk == '11'

- uses: graalvm/setup-graalvm@v1
with:
version: ${{ matrix.jdk }}
java-version: '11'
if: matrix.jdk == '22.3.0'

- uses: actions/setup-node@v3
with:
node-version: "10.x"
Expand All @@ -114,17 +135,19 @@ jobs:
java -version
[[ $HYDRA_LICENSE == floating-key=* ]] && mkdir -p $HOME/.triplequote && echo "$HYDRA_LICENSE" > "$HOME/.triplequote/hydra.license" || echo "Hydra license file was not created"
Copy link
Contributor

@tgodzik tgodzik Dec 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A random comment, I think we are no longer running any hydra tests and potentially we could remove it. We should probably talk with the triple quote people.

Copy link
Member Author

@ckipp01 ckipp01 Dec 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh, yea I just looked. Honestly I don't know hardly anything about Hydra. However seeing that it's a paid project I have no issues ripping all the hydra stuff out. Is it something that is needed? Something that people have requested? Who uses hydra? I'll create a follow-up issue on this.

Copy link
Contributor

@tgodzik tgodzik Dec 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly I have no idea if there are people that both use hydra and Bloop or how much it was requested 🤔

shell: bash

- name: Compile and test main projects
run: |
./bin/sbt-ci.sh \
"frontend/test:compile" \
"backend/test" \
"docs/compile" \
"frontend/testOnly bloop.ScalaVersionsSpec" \
"frontend/testOnly -bloop.ScalaVersionsSpec" \
"frontend/runMain bloop.util.CommandsDocGenerator --test" \
"frontend/runMain bloop.util.CommandsDocGenerator --out ../docs/cli/reference.md"
sbt \
"frontend/test:compile; \
backend/test; \
docs/compile; \
frontend/testOnly bloop.ScalaVersionsSpec; \
frontend/testOnly -bloop.ScalaVersionsSpec; \
frontend/runMain bloop.util.CommandsDocGenerator --test; \
frontend/runMain bloop.util.CommandsDocGenerator --out ../docs/cli/reference.md"
shell: bash

- name: Check docs are up-to-date
run: |
./bin/check-good-practices.sh
Expand All @@ -137,51 +160,12 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ubuntu-20.04, macOS-latest]
jdk: [graalvm-ce-java11@21.1.0]
os: [ubuntu-20.04, macOS-latest, windows-latest]
include:
- os: ubuntu-20.04
artifact: bloop-linux
- os: macos-latest
artifact: bloop-macos
env:
CI: true
SKIP_TEST_RESOURCES_GENERATION: true
GITHUB_DEPLOY_KEY: ${{ secrets.GITHUB_DEPLOY_KEY }}
BLOOPOID_GITHUB_TOKEN: ${{ secrets.BLOOPOID_GITHUB_TOKEN }}
ARTIFACT_NAME: ${{ matrix.artifact }}
steps:
- uses: actions/checkout@v3
with:
submodules: true
fetch-depth: 0
- uses: olafurpg/setup-scala@v13
with:
java-version: ${{ matrix.jdk }}
- name: Publish GraalVM Native artifacts
run: |
echo $JAVA_HOME
which gu
gu install native-image
./bin/sbt-ci.sh "bloopgun/graalvm-native-image:packageBin"
mkdir bloop-artifacts
cp bloopgun/target/bloopgun-2.12/graalvm-native-image/bloopgun-core bloop-artifacts/$ARTIFACT_NAME
shell: bash
- uses: actions/upload-artifact@v3
with:
name: bloop-artifacts
path: bloop-artifacts/${{ matrix.artifact }}

publish-binaries-windows:
name: Publish binaries for ${{ matrix.os }}
runs-on: ${{ matrix.os }}
if: github.event_name == 'push'
strategy:
fail-fast: false
matrix:
os: [windows-latest]
jdk: [graalvm-ce-java11@21.1.0]
include:
- os: windows-latest
artifact: bloop-windows
env:
Expand All @@ -191,43 +175,38 @@ jobs:
BLOOPOID_GITHUB_TOKEN: ${{ secrets.BLOOPOID_GITHUB_TOKEN }}
ARTIFACT_NAME: ${{ matrix.artifact }}
steps:
- name: Configure Pagefile
uses: al-cheb/configure-pagefile-action@v1.3
- name: Refresh Pagefile
run: |
(Get-CimInstance Win32_PageFileUsage).AllocatedBaseSize
- uses: actions/checkout@v3
with:
submodules: true
fetch-depth: 0
- uses: olafurpg/setup-scala@v13

- uses: graalvm/setup-graalvm@v1
with:
java-version: ${{ matrix.jdk }}
- name: Install GraalVM Native Image
run: |
echo $JAVA_HOME
"$JAVA_HOME\bin\gu.cmd" install native-image
shell: bash
- name: Pre-load sbt with bash
# FIXME: This step shouldn't be necessary, but sbt fails to find
# org.scala-lang.modules#scala-xml_2.12;1.2.0!scala-xml_2.12.jar when launched with
# cmd.
# Keep the sbt version in sync with `sbt-ci-release.bat`.
run: |
sbt -sbt-version 1.8.0 version
shell: bash
version: '22.3.0'
java-version: '17'
components: 'native-image'
native-image-job-reports: 'true'
github-token: ${{ secrets.GITHUB_TOKEN }}

- name: Publish GraalVM Native artifacts
run: >-
"C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvars64.bat" &&
"%CD%"\bin\sbt-ci-release.bat
shell: cmd
- name: Prepare generated Windows GraalVM binary
run: sbt "bloopgun213/graalvm-native-image:packageBin"

- name: Copy artifacts (windows)
if: matrix.os == 'windows-latest'
run: |
echo $ARTIFACT_NAME
ls bloopgun/target/bloopgun-2.12/graalvm-native-image/bloopgun-core.exe
mkdir bloop-artifacts
cp bloopgun/target/bloopgun-2.12/graalvm-native-image/bloopgun-core.exe bloop-artifacts/$ARTIFACT_NAME
ls bloopgun/target/bloopgun-2.13/graalvm-native-image/bloopgun-core.exe
mkdir -p bloop-artifacts
cp bloopgun/target/bloopgun-2.13/graalvm-native-image/bloopgun-core.exe bloop-artifacts/$ARTIFACT_NAME
shell: bash

- name: Copy artifacts (not windows)
if: matrix.os != 'windows-latest'
run: |
echo $ARTIFACT_NAME
mkdir -p bloop-artifacts
cp bloopgun/target/bloopgun-2.13/graalvm-native-image/bloopgun-core bloop-artifacts/$ARTIFACT_NAME

- uses: actions/upload-artifact@v3
with:
name: bloop-artifacts
Expand All @@ -237,14 +216,13 @@ jobs:
name: Release version on ${{ matrix.os }}
needs:
- publish-binaries
- publish-binaries-windows
runs-on: ${{ matrix.os }}
if: github.event_name == 'push'
strategy:
fail-fast: true
matrix:
os: [ubuntu-20.04]
jdk: [adopt@1.8]

env:
CI: true
SKIP_TEST_RESOURCES_GENERATION: true
Expand All @@ -261,9 +239,10 @@ jobs:
with:
submodules: true
fetch-depth: 0
- uses: olafurpg/setup-scala@v13
- uses: actions/setup-java@v3
with:
java-version: ${{ matrix.jdk }}
distribution: 'temurin'
java-version: '11'
- uses: actions/setup-node@v3
with:
node-version: "10.x"
Expand All @@ -287,10 +266,15 @@ jobs:
ls -l bloop-artifacts
mkdir -p frontend/target
mv bloop-artifacts frontend/target/graalvm-binaries
./bin/sbt-ci-publish.sh "ci-release" "docs/docusaurusPublishGhpages"
sbt "ci-release; docs/docusaurusPublishGhpages"
shell: bash

- name: Cut GitHub release and update installers on ${{ matrix.os }}
run: |
. bin/ci-setup-bloopoid-keys.sh
export GITHUB_TOKEN="$BLOOPOID_GITHUB_TOKEN"
./bin/sbt-ci-release.sh "frontend/updateHomebrewFormula" "frontend/updateScoopFormula" "frontend/githubRelease"
if [[ $GITHUB_REF == "refs/tags"* ]]; then
. bin/ci-setup-bloopoid-keys.sh
export GITHUB_TOKEN="$BLOOPOID_GITHUB_TOKEN"
sbt "frontend/updateHomebrewFormula; frontend/updateScoopFormula; frontend/githubRelease"
else
echo "skipping release since this is a merge and no new tag is detected"
fi
13 changes: 0 additions & 13 deletions bin/sbt-ci-publish.sh
View file
Open in desktop

This file was deleted.

2 changes: 0 additions & 2 deletions bin/sbt-ci-release.bat
View file
Open in desktop

This file was deleted.

8 changes: 0 additions & 8 deletions bin/sbt-ci-release.sh
View file
Open in desktop

This file was deleted.

13 changes: 0 additions & 13 deletions bin/sbt-ci.sh
View file
Open in desktop

This file was deleted.

Loading