JEP 393: Foreign-Memory Access API (Third Incubator) #11027

pshipton · 2020-10-28T16:00:38Z

See https://openjdk.java.net/jeps/393

pshipton · 2020-10-28T16:00:45Z

@tajila

tajila · 2020-10-28T16:44:53Z

This has been assigned to @babsingh

babsingh · 2020-11-20T04:52:22Z

#11182 introduces the following two native stubs:

Java_jdk_internal_misc_ScopedMemoryAccess_registerNatives: stays empty since our tooling handles the registration of Java native methods.
Java_jdk_internal_misc_ScopedMemoryAccess_closeScope0: currently, throws an assertion; needs an implementation.

New tests and failures are seen for JEP393 (Foreign Memory Access API's Third Incubator):

[SUMMARY] Test results: passed: 20; failed: 4; error: 1

Failed: java/foreign/TestSharedAccess.java
Failed: java/foreign/TestSpliterator.java
Failed: java/foreign/TestByteBuffer.java
Failed: java/foreign/TestHandshake.java
Failed: java/foreign/TestCleaner.java

More details about the failures can be found in the following log: jep393_tests_19Nov20.log.

Need to implement Java_jdk_internal_misc_ScopedMemoryAccess_closeScope0 in order to resolve some of the above JEP393 failures.

babsingh · 2020-11-20T15:19:48Z

More JEP393 failures: #11135 (comment).

JasonFengJ9 · 2020-12-04T16:51:54Z

Note: following tests are excluded via adoptium/aqa-tests#2085

Failed: java/foreign/TestSharedAccess.java
Failed: java/foreign/TestSpliterator.java
Failed: java/foreign/TestByteBuffer.java
Failed: java/foreign/TestHandshake.java
Failed: java/foreign/TestCleaner.java

JasonFengJ9 · 2020-12-04T17:44:22Z

More excludes for java/foreign tests via adoptium/aqa-tests#2088 before the implementation is ready.

java/foreign/stackwalk/TestStackWalk.java 
java/foreign/valist/VaListTest.java 
java/foreign/StdLibTest.java 
java/foreign/TestAdaptVarHandles.java 
java/foreign/TestAddressHandle.java 
java/foreign/TestDowncall.java 
java/foreign/TestIllegalLink.java 
java/foreign/TestIntrinsics.java 
java/foreign/TestMemoryAccess.java 
java/foreign/TestMemoryAccessStatics.java 
java/foreign/TestMemoryAlignment.java 
java/foreign/TestMemoryCopy.java 
java/foreign/TestMemoryHandleAsUnsigned.java 
java/foreign/TestMismatch.java 
java/foreign/TestNativeScope.java 
java/foreign/TestNulls.java 
java/foreign/TestRebase.java 
java/foreign/TestSegments.java 
java/foreign/TestSlices.java 
java/foreign/TestTypeAccess.java 
java/foreign/TestUpcall.java 
java/foreign/TestUpcallHighArity.java 
java/foreign/TestUpcallStubs.java 
java/foreign/TestVarArgs.java 
java/foreign/TestVarHandleCombinators.java

fyi @tajila @babsingh @pshipton

babsingh · 2020-12-04T17:59:49Z

More excludes for java/foreign tests via AdoptOpenJDK/openjdk-tests#2088

Do all the tests, which are being excluded, fail? As per the log file in #11027 (comment) (generated 2 weeks ago), few of these tests pass.

tajila · 2020-12-04T18:01:24Z

More excludes for java/foreign tests

Some of these tests are related to JEP389 rather than JEP393

babsingh · 2020-12-04T18:09:31Z

Some of these tests are related to JEP389 rather than JEP393

JEP389 tests also run as part of the jdk_foreign test suite. I do not think it's possible to run JEP389 and JEP393 tests separately. So, I have identified the JEP389 tests below:

java/foreign/stackwalk/TestStackWalk.java 
java/foreign/valist/VaListTest.java 
java/foreign/StdLibTest.java 
java/foreign/TestDowncall.java 
java/foreign/TestIllegalLink.java 
java/foreign/TestIntrinsics.java 
java/foreign/TestUpcall.java 
java/foreign/TestUpcallHighArity.java 
java/foreign/TestUpcallStubs.java 
java/foreign/TestVarArgs.java

JasonFengJ9 · 2020-12-04T18:26:36Z

Do all the tests, which are being excluded, fail? As per the log file in #11027 (comment) (generated 2 weeks ago), few of these tests pass.

Yes, all excluded tests still fail in internal tests.

JasonFengJ9 · 2020-12-04T18:28:08Z

Some of these tests are related to JEP389 rather than JEP393

Will check and refer JEP 389 instead.

tajila · 2020-12-04T18:37:39Z

We need to track them differently as there is a chance that JEP393 will be ready for 0.25 and JEP389 will not

JasonFengJ9 · 2020-12-04T19:12:23Z

From an internal build Test_openjdknext_j9_sanity.openjdk_x86-64_linux_Nightly/363:

22:36:40  --------------------------------------------------
22:36:43  TEST: java/foreign/TestAdaptVarHandles.java

22:36:43  Caused by: java.lang.ClassCastException: java.lang.invoke.DirectHandle incompatible with java.lang.invoke.BoundMethodHandle
22:36:43  	at java.base/java.lang.invoke.VarHandles.noCheckedExceptions(VarHandles.java:636)
22:36:43  	at java.base/java.lang.invoke.VarHandles.filterCoordinates(VarHandles.java:476)
22:36:43  	at java.base/java.lang.invoke.MethodHandleImpl$1.filterCoordinates(MethodHandleImpl.java:1790)
22:36:43  	at jdk.incubator.foreign/jdk.incubator.foreign.MemoryHandles.filterCoordinates(MemoryHandles.java:374)
22:36:43  	at jdk.incubator.foreign/jdk.internal.foreign.Utils.fixUpVarHandle(Utils.java:92)
22:36:43  	at jdk.incubator.foreign/jdk.internal.foreign.LayoutPath.dereferenceHandle(LayoutPath.java:159)
22:36:43  	at jdk.incubator.foreign/jdk.incubator.foreign.MemoryLayout.lambda$varHandle$3(MemoryLayout.java:411)
22:36:43  	at jdk.incubator.foreign/jdk.incubator.foreign.MemoryLayout$$Lambda$55/0x0000000000000000.apply(Unknown Source)
22:36:43  	at jdk.incubator.foreign/jdk.incubator.foreign.MemoryLayout.computePathOp(MemoryLayout.java:457)
22:36:43  	at jdk.incubator.foreign/jdk.incubator.foreign.MemoryLayout.varHandle(MemoryLayout.java:411)
22:36:43  	at TestAdaptVarHandles.<clinit>(TestAdaptVarHandles.java:90)
22:36:43  	... 30 more
22:36:43  

22:36:43  TEST RESULT: Failed. Execution failed: `main' threw exception: org.testng.TestNGException: An error occurred while instantiating class TestAdaptVarHandles: null

Similar exception as above
22:36:43  ===============================================
22:36:43  java/foreign/TestAddressHandle.java
22:36:43  Total tests run: 6, Failures: 4, Skips: 2
22:36:43  ===============================================

Similar exception as above
22:37:09  ===============================================
22:37:09  java/foreign/TestMemoryAccess.java
22:37:09  Total tests run: 256, Failures: 252, Skips: 0
22:37:09  ===============================================

Similar exception as above
22:37:09  ===============================================
22:37:09  java/foreign/TestMemoryAccessStatics.java
22:37:09  Total tests run: 50, Failures: 50, Skips: 0
22:37:09  ===============================================

Similar exception as above
22:37:13  ===============================================
22:37:13  java/foreign/TestMemoryAlignment.java
22:37:13  Total tests run: 117, Failures: 63, Skips: 0
22:37:13  ===============================================

Similar exception as above
22:37:13  --------------------------------------------------
22:37:13  TEST: java/foreign/TestMemoryCopy.java
22:37:13  TEST RESULT: Failed. Execution failed: `main' threw exception: org.testng.TestNGException: An error occurred while instantiating class TestMemoryCopy: null

Similar exception as above
22:37:13  ===============================================
22:37:13  java/foreign/TestMemoryHandleAsUnsigned.java
22:37:13  Total tests run: 2360, Failures: 2360, Skips: 0
22:37:13  ===============================================

22:37:16  --------------------------------------------------
22:37:16  TEST: java/foreign/TestNulls.java
Similar exception as above
22:37:16  TEST RESULT: Failed. Execution failed: `main' threw exception: org.testng.TestNGException: An error occurred while instantiating class TestNulls: null

22:37:16  --------------------------------------------------
22:37:20  TEST: java/foreign/TestRebase.java
Similar exception as above
22:37:20  TEST RESULT: Failed. Execution failed: `main' threw exception: java.lang.Exception: failures: 1

Similar exception as above
22:37:20  ===============================================
22:37:20  java/foreign/TestSegments.java
22:37:20  Total tests run: 120, Failures: 14, Skips: 1
22:37:20  ===============================================

22:37:20  --------------------------------------------------
22:37:23  TEST: java/foreign/TestSlices.java
Similar exception as above
22:37:23  TEST RESULT: Failed. Execution failed: `main' threw exception: org.testng.TestNGException: An error occurred while instantiating class TestSlices: null

22:37:23  --------------------------------------------------
22:37:23  TEST: java/foreign/TestTypeAccess.java
Similar exception as above
22:37:23  TEST RESULT: Failed. Execution failed: `main' threw exception: org.testng.TestNGException: An error occurred while instantiating class TestTypeAccess: null

Similar exception as above
22:37:26  ===============================================
22:37:26  java/foreign/TestVarHandleCombinators.java
22:37:26  Total tests run: 13, Failures: 6, Skips: 0
22:37:26  ===============================================

22:37:13  --------------------------------------------------
22:37:13  TEST: java/foreign/TestMismatch.java
Similar exception as above
22:37:13  TEST RESULT: Failed. Execution failed: `main' threw exception: org.testng.TestNGException: An error occurred while instantiating class TestMismatch: null

22:37:13  --------------------------------------------------
22:37:16  TEST: java/foreign/TestNativeScope.java

Similar exception as above and ASSERTION failure:

22:37:16  STDERR:
22:37:16  WARNING: Using incubator modules: jdk.incubator.foreign
22:37:16  03:37:13.015 0x1c6800   j9jcl.347    *   ** ASSERTION FAILED ** at common/jdk_internal_misc_ScopedMemoryAccess.cpp:40: ((0 ))
22:37:16  JVMDUMP039I Processing dump event "traceassert", detail "" at 2020/12/02 19:37:13 - please wait.

babsingh · 2020-12-04T21:03:15Z

Re: #11027 (comment) and #11027 (comment)

Locally, some of the jdk/foreign tests pass for me because I have the following fix applied to my JDK16 extensions repo: ibmruntimes/openj9-openjdk-jdk#248. This fix adds support for OpenJ9 VarHandles but at the same time, it breaks support for OpenJDK VarHandles. This fix will not be merged since we plan to use OpenJDK VarHandles in JDK16. Related: #11135 (comment). For the nightly openjdk sanity builds, @JasonFengJ9's failures are correctly documented.

babsingh · 2021-01-21T17:23:17Z

Below is the updated list of JEP393 test failures after ibmruntimes/openj9-openjdk-jdk16#9 is merged for the 0.25 release:

java/foreign/TestByteBuffer.java
java/foreign/TestSharedAccess.java
java/foreign/TestSpliterator.java
java/foreign/TestHandshake.java
java/foreign/TestCleaner.java
java/foreign/TestNativeScope.java

The above tests fail because of

** ASSERTION FAILED ** at common/jdk_internal_misc_ScopedMemoryAccess.cpp:40: ((0 ))

The above assertion failure is being tracked via this issue #11027 (comment) and #11360 (duplicate).

babsingh · 2021-04-16T00:07:55Z

JEP393 is targeted for JDK17 (0.28 release).

They were excluded under eclipse-openj9/openj9#11027 but are not excluded from jdk19+. Signed-off-by: Peter Shipton <Peter_Shipton@ca.ibm.com>

pshipton · 2022-12-20T20:35:48Z

Un-exclude some of the tests excluded under this issue via adoptium/aqa-tests#4211

The remaining jdk17 excludes done via this issue are still excluded on jdk19 under the following (open) issues:
#13994
java/foreign/valist/VaListTest.java
java/foreign/TestDowncall.java
java/foreign/TestIntrinsics.java
java/foreign/TestVarArgs.java

#13999
java/foreign/StdLibTest.java
java/foreign/TestUpcall.java
java/foreign/TestUpcallHighArity.java

#14002 - perm exclude
java/foreign/TestIllegalLink.java

#14134
java/foreign/TestMemoryAccess.java
java/foreign/TestNulls.java

plus #13993 which is a permanent exclude
java/foreign/stackwalk/TestStackWalk.java#default_gc
java/foreign/stackwalk/TestStackWalk.java#shenandoah
java/foreign/stackwalk/TestStackWalk.java#zgc

They were excluded under eclipse-openj9/openj9#11027 but are not excluded from jdk19+. Signed-off-by: Peter Shipton <Peter_Shipton@ca.ibm.com>

They were excluded under eclipse-openj9/openj9#11027 but are not excluded from jdk19+. Signed-off-by: Peter Shipton <Peter_Shipton@ca.ibm.com> Signed-off-by: Peter Shipton <Peter_Shipton@ca.ibm.com>

pshipton added JEP jdk16 labels Oct 28, 2020

pshipton added this to the Release 0.25 (Java 16) milestone Oct 28, 2020

babsingh mentioned this issue Nov 20, 2020

JDK16 VarHandle Support #11182

Merged

babsingh mentioned this issue Dec 4, 2020

JDK16 java/foreign/TestCleaner.java j9jcl.347 * ** ASSERTION FAILED ** at common/jdk_internal_misc_ScopedMemoryAccess.cpp:40: ((0 )) #11360

Closed

JasonFengJ9 mentioned this issue Dec 4, 2020

Exclude JEP 393: Foreign-Memory Access API tests adoptium/aqa-tests#2085

Merged

smlambert added the test excluded label Dec 4, 2020

tajila assigned babsingh Dec 4, 2020

JasonFengJ9 mentioned this issue Dec 4, 2020

JEP 389: Foreign Linker API (Incubator) #11195

Closed

pshipton modified the milestones: Release 0.25 (Java 16), Release 0.26 (Java 8, 11, 16) Apr refresh Jan 29, 2021

tajila modified the milestones: Release 0.26 (Java 8, 11, 16) Apr refresh, Release 0.27 (Java 8, 11, 16) July refresh Mar 4, 2021

babsingh mentioned this issue Apr 15, 2021

JEP 393 : ScopedMemoryAccess.closeScope0 Implementation #12446

Closed

babsingh removed this from the Release 0.27 (Java 8, 11, 16) July refresh milestone Apr 16, 2021

babsingh added this to the Release 0.28 (Java 17) milestone Apr 16, 2021

babsingh added jdk17 and removed jdk16 labels Apr 16, 2021

tajila modified the milestones: Release 0.28 (Java 17), Release 0.29 (Java 8, 11, 17) Oct refresh Aug 9, 2021

pshipton modified the milestones: Release 0.29 (Java 8, 11, 17) Oct refresh, Release 0.30 (Java 8, 11, 17) Jan refresh Oct 13, 2021

tajila modified the milestones: Release 0.30 (Java 8, 11, 17) Jan refresh, Release 0.31 (Java 18) Dec 2, 2021

tajila closed this as completed Feb 8, 2022

pshipton added a commit to pshipton/openjdk-tests that referenced this issue Dec 20, 2022

Un-exclude java/foreign tests that should pass

4e9671b

They were excluded under eclipse-openj9/openj9#11027 but are not excluded from jdk19+. Signed-off-by: Peter Shipton <Peter_Shipton@ca.ibm.com>

pshipton added a commit to pshipton/openjdk-tests that referenced this issue Dec 20, 2022

Un-exclude jdk11 java/foreign tests that should pass

16237c1

They were excluded under eclipse-openj9/openj9#11027 but are not excluded from jdk19+. Signed-off-by: Peter Shipton <Peter_Shipton@ca.ibm.com>

pshipton mentioned this issue Dec 20, 2022

Un-exclude jdk17 java/foreign tests that should pass adoptium/aqa-tests#4211

Merged

pshipton added a commit to pshipton/openjdk-tests that referenced this issue Dec 20, 2022

Un-exclude jdk17 java/foreign tests that should pass

d85f408

They were excluded under eclipse-openj9/openj9#11027 but are not excluded from jdk19+. Signed-off-by: Peter Shipton <Peter_Shipton@ca.ibm.com>

pshipton added perm excluded The test is unsuitable and permanently excluded. and removed test excluded labels Jan 24, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

JEP 393: Foreign-Memory Access API (Third Incubator) #11027

JEP 393: Foreign-Memory Access API (Third Incubator) #11027

pshipton commented Oct 28, 2020

pshipton commented Oct 28, 2020

tajila commented Oct 28, 2020

babsingh commented Nov 20, 2020 •

edited

Loading

babsingh commented Nov 20, 2020

JasonFengJ9 commented Dec 4, 2020

JasonFengJ9 commented Dec 4, 2020

babsingh commented Dec 4, 2020 •

edited

Loading

tajila commented Dec 4, 2020

babsingh commented Dec 4, 2020 •

edited

Loading

JasonFengJ9 commented Dec 4, 2020

JasonFengJ9 commented Dec 4, 2020

tajila commented Dec 4, 2020

JasonFengJ9 commented Dec 4, 2020 •

edited

Loading

babsingh commented Dec 4, 2020 •

edited

Loading

babsingh commented Jan 21, 2021

babsingh commented Apr 16, 2021 •

edited

Loading

pshipton commented Dec 20, 2022 •

edited

Loading

JEP 393: Foreign-Memory Access API (Third Incubator) #11027

JEP 393: Foreign-Memory Access API (Third Incubator) #11027

Comments

pshipton commented Oct 28, 2020

pshipton commented Oct 28, 2020

tajila commented Oct 28, 2020

babsingh commented Nov 20, 2020 • edited Loading

babsingh commented Nov 20, 2020

JasonFengJ9 commented Dec 4, 2020

JasonFengJ9 commented Dec 4, 2020

babsingh commented Dec 4, 2020 • edited Loading

tajila commented Dec 4, 2020

babsingh commented Dec 4, 2020 • edited Loading

JasonFengJ9 commented Dec 4, 2020

JasonFengJ9 commented Dec 4, 2020

tajila commented Dec 4, 2020

JasonFengJ9 commented Dec 4, 2020 • edited Loading

babsingh commented Dec 4, 2020 • edited Loading

babsingh commented Jan 21, 2021

babsingh commented Apr 16, 2021 • edited Loading

pshipton commented Dec 20, 2022 • edited Loading

babsingh commented Nov 20, 2020 •

edited

Loading

babsingh commented Dec 4, 2020 •

edited

Loading

babsingh commented Dec 4, 2020 •

edited

Loading

JasonFengJ9 commented Dec 4, 2020 •

edited

Loading

babsingh commented Dec 4, 2020 •

edited

Loading

babsingh commented Apr 16, 2021 •

edited

Loading

pshipton commented Dec 20, 2022 •

edited

Loading