Skip to content

Commit a087b68

Browse files
ckipp01ckipp01
committed
refactor(ci): update graal release infrastructure
This was a bit of a rabbit hole, but I just went with it. With the changes to sbt-ci-release, _somehow_ the graal native image generation was failing on windows. In order to combat that this pr makes some changes to CI and to the args we're using with Graal. I'll outline the changes below. - Migrate to `graalvm/setup-graalvm` for the graal jobs. This is necessary if we want to use the newer graal versions. When I tried with `setup-scala` I got jabba erros that what I was looking for doesn't exist. Plus `setup-scala` isn't _really_ worked on anymore, so migrating away is preferable. Plus, when using `setup-graalvm` it's easy to get things like the `native-image` command on all platforms. - Bump to 22.3.0. I tried to just bump slightly, but there is a bug in 22.1.0 on windows that I hit on oracle/graal#4502. It's fixed in the newer ones, so I just bumped up. - Use `actions/setup-java` instead of `setup-scala` for jvm tests. Same reason as above with maintenance, but also built-in sbt cache. For now we're still doing a lot on 11 (where the graal stuff is on 17) but I'm trying to not change this too much for now until I get everything green and releasing. Then I'll address that. - Move `publish-binaries-windows` into `publish-binaries`. Now that we are using `setup-graalvm` it's trivial to just keep these together, using `sbt` and just call it a day. No need for two separate ones. - Remove the windows-specific `graalVMNativeImageCommand`. Again, now that we're using `setup-graalvm` we easily have the `native-image` command on the PATH so we don't need to worry about all this extra stuff. - Change a few flags: - `--no-server` wasn't valid it seemed as we were getting a message on every run that it wasn't recognized and ignored. - `-H:EnableUrlProtocols` no just uses `--enable-url-protocols`. Apparently it's frowned upon to use the `-H` stuff since it's internal, so this was just a bit of cleaning up. We do have a couple others, but there doesn't seem to be alternatives so I left them as is. - `--allow-incomlete-classpath` This is the default now, so no need to have it. You can see a green build for all 3 os's [here in my fork](https://github.com/ckipp01/bloop/actions/runs/3787821939/jobs/6439973910). But let's see how all the other jobs go.
1 parent cfc902d commit a087b68

File tree

6 files changed

+87
-152
lines changed

6 files changed

+87
-152
lines changed

‎.github/workflows/ci.yml‎

Lines changed: 85 additions & 106 deletions
Original file line numberDiff line numberDiff line change
@@ -14,54 +14,60 @@ jobs:
1414
runs-on: ubuntu-latest
1515
steps:
1616
- uses: actions/checkout@v3
17-
- uses: olafurpg/setup-scala@v13
17+
- uses: graalvm/setup-graalvm@v1
1818
with:
19-
java-version: graalvm-ce-java11@21.1.0
19+
version: '22.3.0'
20+
java-version: '17'
21+
github-token: ${{ secrets.GITHUB_TOKEN }}
22+
2023
- name: Check formatting
2124
run: ./bin/scalafmt --test
25+
2226
integrations:
2327
name: Build integrations tests
2428
runs-on: ${{ matrix.os }}
2529
strategy:
2630
fail-fast: false
2731
matrix:
2832
os: [ubuntu-latest, macOS-latest]
29-
jdk: [graalvm-ce-java11@21.1.0]
3033
steps:
3134
- uses: actions/checkout@v3
3235
with:
3336
submodules: true
34-
- uses: olafurpg/setup-scala@v13
37+
- uses: graalvm/setup-graalvm@v1
3538
with:
36-
java-version: ${{ matrix.jdk }}
39+
version: '22.3.0'
40+
java-version: '11'
41+
github-token: ${{ secrets.GITHUB_TOKEN }}
42+
3743
- name: Tests
38-
run: |
39-
./bin/sbt-ci.sh \
40-
"sbtBloop/publishLocal" \
41-
"sbtBloop/scripted"
44+
run: sbt "sbtBloop/publishLocal; sbtBloop/scripted"
4245
shell: bash
46+
4347
bridges:
4448
name: Test platform bridges
4549
runs-on: ${{ matrix.os }}
4650
strategy:
4751
fail-fast: false
4852
matrix:
4953
os: [ubuntu-latest, windows-latest, macOS-latest]
50-
jdk: [graalvm-ce-java11@21.1.0]
5154
steps:
5255
- uses: actions/checkout@v2
5356
with:
5457
submodules: true
55-
- uses: olafurpg/setup-scala@v13
58+
- uses: graalvm/setup-graalvm@v1
5659
with:
57-
java-version: ${{ matrix.jdk }}
60+
version: '22.3.0'
61+
java-version: '11'
62+
github-token: ${{ secrets.GITHUB_TOKEN }}
63+
5864
- name: Tests
5965
run: |
60-
./bin/sbt-ci.sh \
61-
"jsBridge06/publishLocal" \
62-
"jsBridge1/publishLocal" \
63-
"jsBridge06/test" \
64-
"jsBridge1/test" \
66+
sbt \
67+
"jsBridge06/publishLocal; \
68+
jsBridge1/publishLocal; \
69+
jsBridge06/test; \
70+
jsBridge1/test"
6571
shell: bash
6672

6773
launcher:
@@ -71,19 +77,21 @@ jobs:
7177
fail-fast: false
7278
matrix:
7379
os: [ubuntu-latest, windows-latest, macOS-latest]
74-
jdk: [graalvm-ce-java11@21.1.0]
7580
steps:
7681
- uses: actions/checkout@v2
7782
with:
7883
submodules: true
79-
- uses: olafurpg/setup-scala@v13
84+
- uses: graalvm/setup-graalvm@v1
8085
with:
81-
java-version: ${{ matrix.jdk }}
86+
version: '22.3.0'
87+
java-version: '11'
88+
github-token: ${{ secrets.GITHUB_TOKEN }}
89+
components: 'native-image'
90+
8291
- name: Tests
8392
run: |
8493
echo $JAVA_HOME
85-
which gu && gu install native-image
86-
./bin/sbt-ci.sh "install" "launcherTest/test"
94+
sbt "install; launcherTest/test"
8795
shell: bash
8896

8997
test:
@@ -92,18 +100,31 @@ jobs:
92100
fail-fast: false
93101
matrix:
94102
os: [ubuntu-latest, windows-latest, macOS-latest]
95-
jdk: [graalvm-ce-java11@21.1.0, adopt@1.11]
103+
jdk: ["22.3.0", "11"]
104+
96105
name: Test ${{ matrix.os }} -- ${{ matrix.jdk }}
97106
env:
98107
HYDRA_LICENSE: ${{ secrets.HYDRA_LICENSE }}
108+
99109
steps:
100110
- uses: actions/checkout@v3
101111
with:
102112
submodules: true
103113
fetch-depth: 0
104-
- uses: olafurpg/setup-scala@v13
114+
115+
- uses: actions/setup-java@v3
105116
with:
117+
distribution: 'temurin'
106118
java-version: ${{ matrix.jdk }}
119+
cache: 'sbt'
120+
if: matrix.jdk == '11'
121+
122+
- uses: graalvm/setup-graalvm@v1
123+
with:
124+
version: ${{ matrix.jdk }}
125+
java-version: '11'
126+
if: matrix.jdk == '22.3.0'
127+
107128
- uses: actions/setup-node@v3
108129
with:
109130
node-version: "10.x"
@@ -114,17 +135,19 @@ jobs:
114135
java -version
115136
[[ $HYDRA_LICENSE == floating-key=* ]] && mkdir -p $HOME/.triplequote && echo "$HYDRA_LICENSE" > "$HOME/.triplequote/hydra.license" || echo "Hydra license file was not created"
116137
shell: bash
138+
117139
- name: Compile and test main projects
118140
run: |
119-
./bin/sbt-ci.sh \
120-
"frontend/test:compile" \
121-
"backend/test" \
122-
"docs/compile" \
123-
"frontend/testOnly bloop.ScalaVersionsSpec" \
124-
"frontend/testOnly -bloop.ScalaVersionsSpec" \
125-
"frontend/runMain bloop.util.CommandsDocGenerator --test" \
126-
"frontend/runMain bloop.util.CommandsDocGenerator --out ../docs/cli/reference.md"
141+
sbt \
142+
"frontend/test:compile; \
143+
backend/test; \
144+
docs/compile; \
145+
frontend/testOnly bloop.ScalaVersionsSpec; \
146+
frontend/testOnly -bloop.ScalaVersionsSpec; \
147+
frontend/runMain bloop.util.CommandsDocGenerator --test; \
148+
frontend/runMain bloop.util.CommandsDocGenerator --out ../docs/cli/reference.md"
127149
shell: bash
150+
128151
- name: Check docs are up-to-date
129152
run: |
130153
./bin/check-good-practices.sh
@@ -137,51 +160,12 @@ jobs:
137160
strategy:
138161
fail-fast: false
139162
matrix:
140-
os: [ubuntu-20.04, macOS-latest]
141-
jdk: [graalvm-ce-java11@21.1.0]
163+
os: [ubuntu-20.04, macOS-latest, windows-latest]
142164
include:
143165
- os: ubuntu-20.04
144166
artifact: bloop-linux
145167
- os: macos-latest
146168
artifact: bloop-macos
147-
env:
148-
CI: true
149-
SKIP_TEST_RESOURCES_GENERATION: true
150-
GITHUB_DEPLOY_KEY: ${{ secrets.GITHUB_DEPLOY_KEY }}
151-
BLOOPOID_GITHUB_TOKEN: ${{ secrets.BLOOPOID_GITHUB_TOKEN }}
152-
ARTIFACT_NAME: ${{ matrix.artifact }}
153-
steps:
154-
- uses: actions/checkout@v3
155-
with:
156-
submodules: true
157-
fetch-depth: 0
158-
- uses: olafurpg/setup-scala@v13
159-
with:
160-
java-version: ${{ matrix.jdk }}
161-
- name: Publish GraalVM Native artifacts
162-
run: |
163-
echo $JAVA_HOME
164-
which gu
165-
gu install native-image
166-
./bin/sbt-ci.sh "bloopgun/graalvm-native-image:packageBin"
167-
mkdir bloop-artifacts
168-
cp bloopgun/target/bloopgun-2.12/graalvm-native-image/bloopgun-core bloop-artifacts/$ARTIFACT_NAME
169-
shell: bash
170-
- uses: actions/upload-artifact@v3
171-
with:
172-
name: bloop-artifacts
173-
path: bloop-artifacts/${{ matrix.artifact }}
174-
175-
publish-binaries-windows:
176-
name: Publish binaries for ${{ matrix.os }}
177-
runs-on: ${{ matrix.os }}
178-
if: github.event_name == 'push'
179-
strategy:
180-
fail-fast: false
181-
matrix:
182-
os: [windows-latest]
183-
jdk: [graalvm-ce-java11@21.1.0]
184-
include:
185169
- os: windows-latest
186170
artifact: bloop-windows
187171
env:
@@ -191,43 +175,38 @@ jobs:
191175
BLOOPOID_GITHUB_TOKEN: ${{ secrets.BLOOPOID_GITHUB_TOKEN }}
192176
ARTIFACT_NAME: ${{ matrix.artifact }}
193177
steps:
194-
- name: Configure Pagefile
195-
uses: al-cheb/configure-pagefile-action@v1.3
196-
- name: Refresh Pagefile
197-
run: |
198-
(Get-CimInstance Win32_PageFileUsage).AllocatedBaseSize
199178
- uses: actions/checkout@v3
200179
with:
201180
submodules: true
202181
fetch-depth: 0
203-
- uses: olafurpg/setup-scala@v13
182+
183+
- uses: graalvm/setup-graalvm@v1
204184
with:
205-
java-version: ${{ matrix.jdk }}
206-
- name: Install GraalVM Native Image
207-
run: |
208-
echo $JAVA_HOME
209-
"$JAVA_HOME\bin\gu.cmd" install native-image
210-
shell: bash
211-
- name: Pre-load sbt with bash
212-
# FIXME: This step shouldn't be necessary, but sbt fails to find
213-
# org.scala-lang.modules#scala-xml_2.12;1.2.0!scala-xml_2.12.jar when launched with
214-
# cmd.
215-
# Keep the sbt version in sync with `sbt-ci-release.bat`.
216-
run: |
217-
sbt -sbt-version 1.8.0 version
218-
shell: bash
185+
version: '22.3.0'
186+
java-version: '17'
187+
components: 'native-image'
188+
native-image-job-reports: 'true'
189+
github-token: ${{ secrets.GITHUB_TOKEN }}
190+
219191
- name: Publish GraalVM Native artifacts
220-
run: >-
221-
"C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvars64.bat" &&
222-
"%CD%"\bin\sbt-ci-release.bat
223-
shell: cmd
224-
- name: Prepare generated Windows GraalVM binary
192+
run: sbt "bloopgun213/graalvm-native-image:packageBin"
193+
194+
- name: Copy artifacts (windows)
195+
if: matrix.os == 'windows-latest'
225196
run: |
226197
echo $ARTIFACT_NAME
227-
ls bloopgun/target/bloopgun-2.12/graalvm-native-image/bloopgun-core.exe
228-
mkdir bloop-artifacts
229-
cp bloopgun/target/bloopgun-2.12/graalvm-native-image/bloopgun-core.exe bloop-artifacts/$ARTIFACT_NAME
198+
ls bloopgun/target/bloopgun-2.13/graalvm-native-image/bloopgun-core.exe
199+
mkdir -p bloop-artifacts
200+
cp bloopgun/target/bloopgun-2.13/graalvm-native-image/bloopgun-core.exe bloop-artifacts/$ARTIFACT_NAME
230201
shell: bash
202+
203+
- name: Copy artifacts (not windows)
204+
if: matrix.os != 'windows-latest'
205+
run: |
206+
echo $ARTIFACT_NAME
207+
mkdir -p bloop-artifacts
208+
cp bloopgun/target/bloopgun-2.13/graalvm-native-image/bloopgun-core bloop-artifacts/$ARTIFACT_NAME
209+
231210
- uses: actions/upload-artifact@v3
232211
with:
233212
name: bloop-artifacts
@@ -237,14 +216,13 @@ jobs:
237216
name: Release version on ${{ matrix.os }}
238217
needs:
239218
- publish-binaries
240-
- publish-binaries-windows
241219
runs-on: ${{ matrix.os }}
242220
if: github.event_name == 'push'
243221
strategy:
244222
fail-fast: true
245223
matrix:
246224
os: [ubuntu-20.04]
247-
jdk: [adopt@1.8]
225+
248226
env:
249227
CI: true
250228
SKIP_TEST_RESOURCES_GENERATION: true
@@ -261,9 +239,10 @@ jobs:
261239
with:
262240
submodules: true
263241
fetch-depth: 0
264-
- uses: olafurpg/setup-scala@v13
242+
- uses: actions/setup-java@v3
265243
with:
266-
java-version: ${{ matrix.jdk }}
244+
distribution: 'temurin'
245+
java-version: '11'
267246
- uses: actions/setup-node@v3
268247
with:
269248
node-version: "10.x"
@@ -287,10 +266,10 @@ jobs:
287266
ls -l bloop-artifacts
288267
mkdir -p frontend/target
289268
mv bloop-artifacts frontend/target/graalvm-binaries
290-
./bin/sbt-ci-publish.sh "ci-release" "docs/docusaurusPublishGhpages"
269+
sbt "ci-release; docs/docusaurusPublishGhpages"
291270
shell: bash
292271
- name: Cut GitHub release and update installers on ${{ matrix.os }}
293272
run: |
294273
. bin/ci-setup-bloopoid-keys.sh
295274
export GITHUB_TOKEN="$BLOOPOID_GITHUB_TOKEN"
296-
./bin/sbt-ci-release.sh "frontend/updateHomebrewFormula" "frontend/updateScoopFormula" "frontend/githubRelease"
275+
sbt "frontend/updateHomebrewFormula; frontend/updateScoopFormula; frontend/githubRelease"

‎bin/sbt-ci-publish.sh‎

Lines changed: 0 additions & 13 deletions
This file was deleted.

‎bin/sbt-ci-release.bat‎

Lines changed: 0 additions & 2 deletions
This file was deleted.

‎bin/sbt-ci-release.sh‎

Lines changed: 0 additions & 8 deletions
This file was deleted.

‎bin/sbt-ci.sh‎

Lines changed: 0 additions & 13 deletions
This file was deleted.

0 commit comments

Comments
 (0)