openjdk: add ARM version #68946

fxcoudert · 2021-01-13T18:36:45Z

Trying to rework #65670 into the direction decided by the TSC (see #65670 (comment) and below)

Formula/openjdk.rb

carlocab · 2021-01-13T19:25:45Z

Looks like you need

Lines 38 to 42 in 3eb4055

 # Calculate Xcode's dual-arch JavaNativeFoundation.framework path 

 def framework_path 

 File.expand_path("../SharedFrameworks/ContentDeliveryServices.framework/Versions/Current/itms/java/Frameworks", 

 MacOS::Xcode.prefix) 

 end

?

tresf · 2021-01-13T19:38:00Z

Formula/openjdk.rb

- url "https://download.java.net/java/GA/jdk14.0.2/205943a0976c4ed48cb16f1043c5c647/12/GPL/openjdk-14.0.2_osx-x64_bin.tar.gz"
- sha256 "386a96eeef63bf94b450809d69ceaa1c9e32a97230e0a120c1b41786b743ae84"
+ if Hardware::CPU.arm?
+  url "https://download.java.net/java/early_access/jdk16/29/GPL/openjdk-16-ea+29_osx-x64_bin.tar.gz"


Quoting an outstanding task from #65670:

Before merge, [we'll] need a reliable 16-ea URL or can just hard-link any 16 build if @VladimirKempik thinks it'll be OK.

@VladimirKempik replied with the following information:

jep391 branch will soon be merged with upstream and become 17-ea

I'm not sure how soon this would happen, but working from upstream (and not a sandbox) is obviously preferred, but we'll need a boot-jdk that won't be pulled down days or weeks after this PR is merged.

@VladimirKempik I'm sorry if I asked this before, but is it absolutely mandatory that the boot-jdk has JEP-391 code? If not, we can probably find a more reliable link.

If it is mandatory that we have a JEP-391 enabled boot-jdk, I'd need some guidance from Homebrew as to where to retrieve this from. From my experience with other formulae, I understand volatile links are not uncommon, but I don't know if it's OK here? Once the bottles are created, it would only affect the source builds, I'm just not sure if that's acceptable.

How often would the link break?

boot-jdk has to be either jdkX or jdkX-1 ( where X is jdk version you build)
build-jdk has to be the same version of jdk you build
both doesn't have to have jep-391 code in it.
so you can just have some fixed version of upstream(jdk/jdk) to be used as boot/build jdk. this will exist forever

How often would the link break?

It broke while the old PR was open (26-29, now 31), so they change frequently, I'm not sure how long they maintain the old URLs for though.

Although any JDK16 version will do, I'm unsure where to find a stable link. Of the commercial providers, Zulu has an EA link that we could potentially borrow -- just for boot-jdk purposes, but I assume this is frowned upon.

I've switched it to the latest (openjdk-16-ea+31) and I've mirror it on our side.

fxcoudert · 2021-01-13T20:42:12Z

I thought I hadn't changed anything on Intel, but we get:

configure: Potential Build JDK found at /private/tmp/openjdk-20210113-63129-k3mi50/jdk15u-jdk-15.0.1-ga/boot-jdk/Contents/Home is incorrect JDK version (openjdk version "14.0.2" 2020-07-14); ignoring
configure: (Your Build JDK must be version 15)

tresf · 2021-01-13T20:52:42Z

I thought I hadn't changed anything on Intel, but we get:

configure: Potential Build JDK found at /private/tmp/openjdk-20210113-63129-k3mi50/jdk15u-jdk-15.0.1-ga/boot-jdk/Contents/Home is incorrect JDK version (openjdk version "14.0.2" 2020-07-14); ignoring
configure: (Your Build JDK must be version 15)

Agreed, that doesn't appear to be newly introduced with this PR. I'll fire a vanilla source build on Intel and see if the same error occurs.

tresf · 2021-01-13T21:13:20Z

Hmm... oddly, it doesn't fail on the vanilla formula.

...
checking for Boot JDK... /private/tmp/openjdk-20210113-59114-1t9ow2k/jdk15u-jdk-15.0.1-ga/boot-jdk/Contents/Home
checking Boot JDK version... openjdk version "14.0.2" 2020-07-14 OpenJDK Runtime Environment (build 14.0.2+12-46) OpenJDK 64-Bit Server VM (build 14.0.2+12-46, mixed mode, sharing) 
checking for java in Boot JDK... ok
...

VladimirKempik · 2021-01-13T21:15:13Z

Hmm... oddly, it doesn't fail on the vanilla formula.

...
checking for Boot JDK... /private/tmp/openjdk-20210113-59114-1t9ow2k/jdk15u-jdk-15.0.1-ga/boot-jdk/Contents/Home
checking Boot JDK version... openjdk version "14.0.2" 2020-07-14 OpenJDK Runtime Environment (build 14.0.2+12-46) OpenJDK 64-Bit Server VM (build 14.0.2+12-46, mixed mode, sharing) 
checking for java in Boot JDK... ok
...

you build jdk15 using jdk14 as boot jdk - this is fine

when you try to build jdk16 using jdk14 as boot jdk - it will fail

carlocab · 2021-01-13T21:16:27Z

when you try to build jdk16 using jdk14 as boot jdk - it will fail

Where does the formula do this? I don't see it...

VladimirKempik · 2021-01-13T21:18:49Z

when you try to build jdk16 using jdk14 as boot jdk - it will fail

Where does the formula do this? I don't see it...

#68946 (comment)

jdk15u-jdk-15.0.1-ga/boot-jdk/Contents/Home is incorrect JDK version (openjdk version "14.0.2" 2020-07-14); ignoring

fxcoudert · 2021-01-13T21:20:02Z

@VladimirKempik this is not building jdk16, it's building jdk15 (and it definitely used to work)

carlocab · 2021-01-13T21:21:37Z

when you try to build jdk16 using jdk14 as boot jdk - it will fail

Where does the formula do this? I don't see it...

#68946 (comment)

jdk15u-jdk-15.0.1-ga/boot-jdk/Contents/Home is incorrect JDK version (openjdk version "14.0.2" 2020-07-14); ignoring

yes, but where in the formula code is this coming from?

VladimirKempik · 2021-01-13T21:22:09Z

excuse me, I missed one word there:

configure: Potential Build JDK found at /private/tmp/openjdk-20210113-63129-k3mi50/jdk15u-jdk-15.0.1-ga/boot-jdk/Contents/Home is incorrect JDK version (openjdk version "14.0.2" 2020-07-14); ignoring

build jdk has to match the version of jdk you building

but one doesnt need build-jdk if not cross-compiling

tresf · 2021-01-13T21:26:50Z

but one doesnt need build-jdk if not cross-compiling

@VladimirKempik's right, good catch. I added that for all builds, it's a mistake and should be moved to the arm-only block.

This is what master looks like, notice there's no build-jdk provided.

homebrew-core/Formula/openjdk.rb

Line 88 in fcef5e5

"--with-boot-jdk=#{boot_jdk}",

fxcoudert · 2021-01-13T21:31:21Z

@tresf I think you mean build, not boot

Updated, and pushed for a new round of testing.

carlocab · 2021-01-13T23:27:43Z

Are we not doing ARM caveats like with go and rust?

Formula/openjdk.rb

jonchang · 2021-01-14T06:08:52Z

on_macos blocks within resource blocks must contain only a url and sha256 or a url, version, and sha256 (in those orders).

Homebrew/brew#10320

tresf · 2021-01-14T06:12:49Z

on_macos blocks within resource blocks must contain only a url and sha256 or a url, version, and sha256 (in those orders).

Homebrew/brew#10320

You may find out more about this predicament here: #65670 (comment)

jonchang · 2021-01-14T07:00:00Z

I've restarted CI now that Homebrew/brew#10320 is merged.

fxcoudert · 2021-01-14T07:39:29Z

Thanks @jonchang you rock!

tresf · 2021-01-14T18:02:54Z

Formula/openjdk.rb

- .max
- raise "cannot find build number in .hg_archival.txt" if build.nil?
+ if Hardware::CPU.arm?
+ build = File.read(".hgtags")


I was testing this approach on 15.0.1 (to allow us to clean up the comment above) and it reports incorrectly. It will report 15.0.1+8 instead of 15.0.1+9. @VladimirKempik moving forward, should we add +1 to the last tag in .hgtags or is this handled automatically in newer releases? How should we calculate this value moving forward?

How much of a deal-breaker is that? Testing has been progressing nicely so far, so I planned to merge this tomorrow. Should I wait for a solution to this, or is it a minor issue?

How much of a deal-breaker is that? Testing has been progressing nicely so far, so I planned to merge this tomorrow. Should I wait for a solution to this, or is it a minor issue?

I was thinking the same thing when I saw the "ready to merge" tag. I suppose it can be tackled at some future time when this is consolidated, but I'd still like to know moving forward as the comment suggests the old versioning information will eventually go away.

fxcoudert · 2021-01-14T21:40:45Z

The ARM testing was stuck on mockserver; many formulas before that worked, and several failed (mostly for cross-dependencies to older openjdk). I am merging, will build a bottle, and then queue bottling of dependents.

fxcoudert · 2021-01-14T22:26:44Z

Oops. https://github.com/Homebrew/homebrew-core/runs/1705106711?check_suite_focus=true

Error: Bottles are for openjdk 15.99.99 but formula is version 15.0.1!

So we actually can't have arch-dependent version.

fxcoudert · 2021-01-15T10:28:19Z

First few builds of openjdk-dependent formulas on ARM are a mixed bag:

jmeter worked
zookeeper fails https://github.com/Homebrew/homebrew-core/runs/1707698425?check_suite_focus=true
elasticsearch fails https://github.com/Homebrew/homebrew-core/runs/1707700107?check_suite_focus=true

fxcoudert · 2021-01-15T10:52:25Z

OK, here are better stats

ARM bottling worked:

jmeter, abcl, anjuta, apache-forrest, apollo, cfr-decompiler, duck, fastbit, htmlcleaner, javacc, jdnssec-tools, jhipster, jollie, joshua, jsvc, lcm, leiningen, link-grammar, liquigraph, midgard2, nesc, onscripter, parquet-tools, sleuthkit, swagger-codegen, tomcat-native

ARM bottling failed:

zookeeper https://github.com/Homebrew/homebrew-core/runs/1707698425?check_suite_focus=true
elasticsearch https://github.com/Homebrew/homebrew-core/runs/1707700107?check_suite_focus=true
beagle tries to use x86 asm https://github.com/Homebrew/homebrew-core/runs/1707914973?check_suite_focus=true
cbmc has outdated catch header https://github.com/Homebrew/homebrew-core/runs/1707916375?check_suite_focus=true
micronaut https://github.com/Homebrew/homebrew-core/runs/1707929705?check_suite_focus=true
planck https://github.com/Homebrew/homebrew-core/runs/1707933280?check_suite_focus=true
smali https://github.com/Homebrew/homebrew-core/runs/1707935885?check_suite_focus=true
upscaledb tries to use x86 asm https://github.com/Homebrew/homebrew-core/runs/1707938939?check_suite_focus=true
mercury https://github.com/Homebrew/homebrew-core/runs/1707929452?check_suite_focus=true

Audit failures, probably fixable on Homebrew side:

fxcoudert · 2021-01-15T11:04:45Z

@tresf @VladimirKempik regarding openjdk@8 and openjdk@11, I understand there may not be a backport to the official openjdk (I've read the redhat position on that, which you posted on the previous PR). However, for each of those two versions: is there a set of patches, or a fork (or downstream distribution, I don't know how they're called) that we could switch to?

That would be on ARM only, we would keep the official source on Intel. The idea being that for our ARM users, it's better to distribute a patch or branch from a well-known vendor, than nothing from the official branch.

(Excuse my ignorance about the whole openjdk ecosystem…)

unitof · 2021-01-15T14:02:52Z

Bottle installs!

However I get the error

Error: Failed applying an ad-hoc signature to /opt/homebrew/Cellar/openjdk/15.0.1/libexec/openjdk.jdk/Contents/MacOS/libjli.dylib

during the Pouring step, and postinstall does not complete successfully, though I don't see an obvious error:

~ ▶ brew postinstall openjdk -v
==> Postinstalling openjdk
/usr/bin/sandbox-exec -f /private/tmp/homebrew20210115-96559-wci16w.sb nice ruby -W0 -I $LOAD_PATH -- /opt/homebrew/Library/Homebrew/postinstall.rb /opt/homebrew/Library/Taps/homebrew/homebrew-core/Formula/openjdk.rb
Warning: The post-install step did not complete successfully
You can try again using `brew postinstall openjdk`

VladimirKempik · 2021-01-15T14:04:57Z

@tresf @VladimirKempik regarding openjdk@8 and openjdk@11, I understand there may not be a backport to the official openjdk (I've read the redhat position on that, which you posted on the previous PR). However, for each of those two versions: is there a set of patches, or a fork (or downstream distribution, I don't know how they're called) that we could switch to?

That would be on ARM only, we would keep the official source on Intel. The idea being that for our ARM users, it's better to distribute a patch or branch from a well-known vendor, than nothing from the official branch.

(Excuse my ignorance about the whole openjdk ecosystem…)

You will need to wait for the answer here while we will think about it.

fxcoudert · 2021-01-15T15:02:40Z

@unitof weird, can you please file an issue about it, and provide all the requested information? I'll try to debug that

tresf · 2021-01-15T16:13:08Z

@tresf @VladimirKempik regarding openjdk@8 and openjdk@11 [...] is there a set of patches [...] that we could switch to?

@VladimirKempik kindly provided ARM64 patches for openjdk@11 here: #65670 (comment) (the patch is > 4K lines, it won't apply cleanly as he's warned).

I haven't seen any publicly available patches for openjdk@8 land yet.

fxcoudert added the CI-force-arm [DEPRECATED] Don't pass --skip-unbottled-arm to brew test-bot. label Jan 13, 2021

fxcoudert mentioned this pull request Jan 13, 2021

openjdk: Add support for Apple silicon #65670

Closed

7 tasks

BrewTestBot added the no ARM bottle Formula has no ARM bottle label Jan 13, 2021

unitof reviewed Jan 13, 2021

View reviewed changes

Formula/openjdk.rb Show resolved Hide resolved

fxcoudert force-pushed the openjdk branch 3 times, most recently from a05a528 to 80791f0 Compare January 13, 2021 19:25

tresf reviewed Jan 13, 2021

View reviewed changes

fxcoudert force-pushed the openjdk branch 2 times, most recently from 632d575 to d7b7565 Compare January 13, 2021 23:23

fxcoudert added the ready to merge PR can be merged once CI is green label Jan 13, 2021

fxcoudert force-pushed the openjdk branch from d7b7565 to a7dbbef Compare January 13, 2021 23:34

max-ae reviewed Jan 14, 2021

View reviewed changes

Formula/openjdk.rb Outdated Show resolved Hide resolved

dnsco mentioned this pull request Jan 14, 2021

Apple Silicon support in Homebrew Homebrew/brew#10152

Closed

jonchang mentioned this pull request Jan 14, 2021

rubocops: permit mirror in OS-specific resources Homebrew/brew#10320

Merged

8 tasks

openjdk: add ARM version

41659f1

fxcoudert force-pushed the openjdk branch from a7dbbef to 41659f1 Compare January 14, 2021 07:39

tresf reviewed Jan 14, 2021

View reviewed changes

tresf mentioned this pull request Jan 14, 2021

macOS 11 Big Sur compatibility on Apple Silicon Homebrew/brew#7857

Closed

9 tasks

fxcoudert merged commit da9e9e5 into Homebrew:master Jan 14, 2021

fxcoudert deleted the openjdk branch January 14, 2021 21:41

fxcoudert mentioned this pull request Jan 14, 2021

openjdk: fix version on ARM #69029

Merged

carlocab mentioned this pull request Jan 14, 2021

go: install go1.16beta1 in go1.16beta1 directory #69000

Closed

unitof mentioned this pull request Jan 15, 2021

Openjdk ARM bottle installs but with errors #69102

Closed

3 tasks

carlocab mentioned this pull request Jan 21, 2021

openjdk 15.0.2 #69460

Closed

dfreilich mentioned this pull request Jan 27, 2021

Build and Release for Mac Arm64 architecture buildpacks/pack#1030

Merged

1 task

BrewTestBot added the outdated PR was locked due to age label Feb 15, 2021

Homebrew locked as resolved and limited conversation to collaborators Feb 15, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

openjdk: add ARM version #68946

openjdk: add ARM version #68946

fxcoudert commented Jan 13, 2021

carlocab commented Jan 13, 2021

tresf Jan 13, 2021 •

edited

Loading

fxcoudert Jan 13, 2021

VladimirKempik Jan 13, 2021 •

edited

Loading

tresf Jan 13, 2021 •

edited

Loading

fxcoudert Jan 13, 2021

fxcoudert commented Jan 13, 2021

tresf commented Jan 13, 2021

tresf commented Jan 13, 2021

VladimirKempik commented Jan 13, 2021

carlocab commented Jan 13, 2021

VladimirKempik commented Jan 13, 2021

fxcoudert commented Jan 13, 2021

carlocab commented Jan 13, 2021

VladimirKempik commented Jan 13, 2021 •

edited

Loading

tresf commented Jan 13, 2021 •

edited

Loading

fxcoudert commented Jan 13, 2021 •

edited

Loading

carlocab commented Jan 13, 2021

jonchang commented Jan 14, 2021

tresf commented Jan 14, 2021 •

edited

Loading

jonchang commented Jan 14, 2021

fxcoudert commented Jan 14, 2021

tresf Jan 14, 2021 •

edited

Loading

fxcoudert Jan 14, 2021

tresf Jan 14, 2021

fxcoudert commented Jan 14, 2021

fxcoudert commented Jan 14, 2021

fxcoudert commented Jan 15, 2021

fxcoudert commented Jan 15, 2021 •

edited

Loading

fxcoudert commented Jan 15, 2021

unitof commented Jan 15, 2021 •

edited

Loading

VladimirKempik commented Jan 15, 2021

fxcoudert commented Jan 15, 2021

tresf commented Jan 15, 2021 •

edited

Loading

openjdk: add ARM version #68946

openjdk: add ARM version #68946

Conversation

fxcoudert commented Jan 13, 2021

carlocab commented Jan 13, 2021

tresf Jan 13, 2021 • edited Loading

Choose a reason for hiding this comment

fxcoudert Jan 13, 2021

Choose a reason for hiding this comment

VladimirKempik Jan 13, 2021 • edited Loading

Choose a reason for hiding this comment

tresf Jan 13, 2021 • edited Loading

Choose a reason for hiding this comment

fxcoudert Jan 13, 2021

Choose a reason for hiding this comment

fxcoudert commented Jan 13, 2021

tresf commented Jan 13, 2021

tresf commented Jan 13, 2021

VladimirKempik commented Jan 13, 2021

carlocab commented Jan 13, 2021

VladimirKempik commented Jan 13, 2021

fxcoudert commented Jan 13, 2021

carlocab commented Jan 13, 2021

VladimirKempik commented Jan 13, 2021 • edited Loading

tresf commented Jan 13, 2021 • edited Loading

fxcoudert commented Jan 13, 2021 • edited Loading

carlocab commented Jan 13, 2021

jonchang commented Jan 14, 2021

tresf commented Jan 14, 2021 • edited Loading

jonchang commented Jan 14, 2021

fxcoudert commented Jan 14, 2021

tresf Jan 14, 2021 • edited Loading

Choose a reason for hiding this comment

fxcoudert Jan 14, 2021

Choose a reason for hiding this comment

tresf Jan 14, 2021

Choose a reason for hiding this comment

fxcoudert commented Jan 14, 2021

fxcoudert commented Jan 14, 2021

fxcoudert commented Jan 15, 2021

fxcoudert commented Jan 15, 2021 • edited Loading

ARM bottling worked:

ARM bottling failed:

Audit failures, probably fixable on Homebrew side:

fxcoudert commented Jan 15, 2021

unitof commented Jan 15, 2021 • edited Loading

VladimirKempik commented Jan 15, 2021

fxcoudert commented Jan 15, 2021

tresf commented Jan 15, 2021 • edited Loading

tresf Jan 13, 2021 •

edited

Loading

VladimirKempik Jan 13, 2021 •

edited

Loading

tresf Jan 13, 2021 •

edited

Loading

VladimirKempik commented Jan 13, 2021 •

edited

Loading

tresf commented Jan 13, 2021 •

edited

Loading

fxcoudert commented Jan 13, 2021 •

edited

Loading

tresf commented Jan 14, 2021 •

edited

Loading

tresf Jan 14, 2021 •

edited

Loading

fxcoudert commented Jan 15, 2021 •

edited

Loading

unitof commented Jan 15, 2021 •

edited

Loading

tresf commented Jan 15, 2021 •

edited

Loading