openjdk zip or inflate/deflate failures on RH9 (z15+) or AIX 7.3 (p10) #14948
Comments
|
I tried Adoptium jdk-11.0.15+8 with java/lang/ClassLoader/securityManager/ClassLoaderTest.java and it has the same problem. |
|
I believe some of this is fixed by openjdk/jdk11u-dev#1163 |
|
Also discovered some failures on AIX 7.3 P10. jdk_security3 tools/pack200/PackTestZip64.java java/util/zip/ZipFile/CopyJar.java |
pshipton
changed the title
This was referenced Jul 21, 2022
|
An internal build( |
|
With the jdk11 rc1a build (containing the fix linked above) we get the following failures (for both compressed and non-compressed runs). None of the failures are for "invalid entry compressed size". zlinux on RHEL9 z15 java/util/zip/DeflateIn_InflateOut.java.DeflateIn_InflateOut sun/security/tools/jarsigner/OldSig.java passed AIX on AIX 7.3 P10 java/util/zip/CopyZipFile.java.CopyZipFile java/util/zip/DeInflate.java.DeInflate java/util/zip/FlaterTest.java.FlaterTest java/util/zip/InflaterInputStream/TestAvailable.java.TestAvailable |
|
Looking at the CopyZipFile.java.CopyZipFile failures on AIX, where we fail with the this exception, issue seems to be related to zlib on AIX 7.3 P10 (Or integration JDK with zlib) itself. Test simply while creating a Zip file puts different entry with different compression level and different compression method. We fail because of the third entry in the ZipFile [1], where the entry is added with Now when I modified the test to print out the compressed size of each file that was written when we first create zip File, I see that for third entry, which should be stored with For the above unit test for
while on my workstation (Mac), I got following,
See the difference in compressed data size, where on Mac with no_Compression, we ended up using more data (Probably the metadata/header of the deflated data) , while on AIX it still ends up compressing data. In the same unit test, if I construct the Deflater object with passing it the compression level, it works as expected. [1]. https://github.com/ibmruntimes/openj9-openjdk-jdk11/blob/20d13352ee5c9569416880e340d5c589161082f9/test/jdk/java/util/zip/CopyZipFile.java#L73-L76 |
|
For the The test stores the compressed data size in numReadable variable for the input data that was generated by reset() in the beginning of the test. Later it tests skipping different number of compressed bytes with different set of data[1][2]. It compares this total of [1]. https://github.com/ibmruntimes/openj9-openjdk-jdk11/blob/20d13352ee5c9569416880e340d5c589161082f9/test/jdk/java/util/zip/DeflateIn_InflateOut.java#L165-L177 |
|
@zl-wang fyi the zlib problems on AIX 7.3 P10. I'm assuming zlib is providing some hardware compression but seems to have additional problem behaviour to RHEL 9/z15 as well. |
|
@rmnattas this was the feature you enabled for AIX. please take a look ... |
r30shah
added a commit
to r30shah/openj9-openjdk-jdk11
that referenced
this issue
Jul 25, 2022
java/util/zip/DeflateIn_InflateOut.skipBytes verifies functionality of DeflaterInputStream.skipBytes. While testing skipping of bytes, test uses different input data with same data length, and compares the compressed sizes of these data to verify skipping bytes. As per zlib specification which is what is used by Deflater in java, length of compressed data is highly dependable on input data contents, so it will be incorrect to assume that compression ratio would be constant for same length input data. For more details please see explaination in eclipse-openj9/openj9#14948 (comment). Signed-off-by: Rahil Shah <rahil@ca.ibm.com>
|
I have opened up WIP PR in ibmruntimes/openj9-openjdk-jdk11#559 that fixes the |
|
Moving this forward while we continue to investigate and update tests. |
pshipton
modified the milestones:
Release 0.33 (Java 8, 11, 17, 18) July refresh,
Release 0.35 (Java 8, 11, 17, 19) Oct refresh
|
Further looked into the failures seen with I copied the golden.jar that is used as input and manually performed steps from the test on RHEL9/Z15 machine and I do see that the size of jar file produced using Looking into the code that is executed for [1]. https://github.com/ibmruntimes/openj9-openjdk-jdk11/blob/20d13352ee5c9569416880e340d5c589161082f9/test/jdk/tools/pack200/PackTestZip64.java#L89 |
|
For the AIX failures: on P9 and P10 AIX machines, OpenJ9 uses zlibNX (a hardware accelerated version of zlib) if it's installed in the system. The method |
|
The AIX zlibNX library have been fixed and a patched is available (Defect, libz.a) The hardware accelerated compression algorithm have only one compression level, hence changing the compression level (1-9) is meaningless and doesn't have an effect. The edge case which fails this test before the patch is changing from/to level 0 (no compression). |
|
Created infrastructure/issues/7144 to see about updating the machines. |
pshipton
changed the title
|
An internal build( |
|
@r30shah pls also take a look at the failure in the previous comment when you get a chance. |
|
An internal build( Some errors are similar w/ #14948 (comment) |
pshipton
modified the milestones:
Release 0.38 (Java 8, 11, 17) Apr refresh,
Release 0.40? (Java 8, 11, 17, 20) July refresh
|
JDK8 ppc64_aix( |
|
@bhavanisn could the above wrong crc32 be related to your recent check-in? please take a look |
|
Also seen in other Java level/platforms |
|
Created a separate issue It occurred across Java level/platforms abuilds, tentatively set it as a blocker. |
pshipton
modified the milestones:
Release 0.40 (Java 8, 11, 17, 20) July refresh,
Release 0.41 (Java 8, 11, 17) Oct refresh
|
JDK11 s390x_linux milestone 2( |
pshipton
modified the milestones:
Release 0.41 (Java 8, 11, 17) Oct refresh,
Release 0.43 (Java 8, 11, 17, 21) Jan refresh
pshipton
added a commit
to pshipton/openjdk-tests
that referenced
this issue
Sep 20, 2023
It doesn't pass when the system zlib is optimized to use hardware, such as with rhel8 on z15. Issue eclipse-openj9/openj9#14948 Signed-off-by: Peter Shipton <Peter_Shipton@ca.ibm.com>
|
Exclude java/util/zip/CloseInflaterDeflaterTest on zlinux. |
llxia
pushed a commit
to adoptium/aqa-tests
that referenced
this issue
Sep 21, 2023
It doesn't pass when the system zlib is optimized to use hardware, such as with rhel8 on z15. Issue eclipse-openj9/openj9#14948 Signed-off-by: Peter Shipton <Peter_Shipton@ca.ibm.com>
|
JDK11 s390x_linux milestone 2( |
|
https://openj9-jenkins.osuosl.org/job/Test_openjdk17_j9_sanity.openjdk_s390x_linux_OMR_testList_2/32 - rh8-390-2 |
pshipton
modified the milestones:
Release 0.43 (Java 8, 11, 17, 21) Jan refresh,
Release 0.44 (Java 8, 11, 17, 21) Apr refresh
|
JDK17 s390x_linux( |
pshipton
modified the milestones:
Release 0.44 (Java 8, 11, 17, 21) Apr refresh,
Release 0.45 (Java 8, 11, 17, 21, 22) July refresh
pshipton
removed this from the Release 0.46 (Java 8, 11, 17, 21, 22) July refresh milestone
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This is on zlinux.
/view/Test_grinder/job/Grinder/22939
jdk_lang_0
java/lang/ClassLoader/securityManager/ClassLoaderTest.java
jdk_lang_1
java/lang/ClassLoader/securityManager/ClassLoaderTest.java
jdk_util_0, jdk_util_1
java/util/zip/ZipFile/CopyJar.java
jdk_util_0, jdk_util_1
java/util/zip/DeflateIn_InflateOut.java
jdk11_tier1_pack200_0, jdk11_tier1_pack200_1
tools/pack200/PackTestZip64.java
The text was updated successfully, but these errors were encountered: