JDK8 DaaLoadTest_daa2_ConcurrentScavenge_0_FAILED : ASSERTION FAILED ** at OwnableSynchronizerObjectBuffer.cpp:100 ((false)) #10489

JasonFengJ9 · 2020-08-31T00:43:13Z

Failure link

https://ci.eclipse.org/openj9/job/Test_openjdk8_j9_sanity.system_ppc64le_linux_Nightly/472/tapResults/
For example, to rebuild the failed tests in =https://ci.eclipse.org/openj9/job/Grinder, use the following links:
23:30:47 https://ci.eclipse.org/openj9/job/Grinder/parambuild/?JDK_VERSION=8&JDK_IMPL=openj9&BUILD_LIST=system&PLATFORM=ppc64le_linux&TARGET=DaaLoadTest_daa2_ConcurrentScavenge_0

Optional info

Failure output (captured from console output)

DLT 23:21:09.234 -   26 JUnit[net.openj9.test.PD2Primitive.TestI2PD]  Weighting=1 
DLT 23:21:09.234 -   27 JUnit[net.openj9.test.PD2Primitive.TestPD2I]  Weighting=1 
DLT 23:21:09.273 - Starting thread. Suite=0 thread=0
DLT 23:21:09.275 - Starting thread. Suite=0 thread=1
DLT 23:21:09.275 - Starting thread. Suite=0 thread=2
DLT 23:21:09.276 - Starting thread. Suite=0 thread=3
DLT 23:21:09.276 - Starting thread. Suite=0 thread=4
DLT 23:21:09.276 - Starting thread. Suite=0 thread=5
DLT 23:21:09.277 - Starting thread. Suite=0 thread=6
DLT 23:21:09.277 - Starting thread. Suite=0 thread=7
DLT 23:21:09.277 - Starting thread. Suite=0 thread=8
DLT 23:21:09.277 - Starting thread. Suite=0 thread=9
DLT 23:21:09.278 - Starting thread. Suite=0 thread=10
DLT 23:21:09.278 - Starting thread. Suite=0 thread=11
DLT 23:21:09.278 - Starting thread. Suite=0 thread=12
DLT 23:21:09.279 - Starting thread. Suite=0 thread=13
DLT stderr Attempt to access ownable synchronizer object located outside of heap (stack allocated?) 0000000000740073
DLT stderr 03:21:10.200 0x118d00    j9mm.141    *   ** ASSERTION FAILED ** at OwnableSynchronizerObjectBuffer.cpp:100: ((false))
DLT stderr JVMDUMP039I Processing dump event "traceassert", detail "" at 2020/08/29 23:21:10 - please wait.
DLT stderr JVMDUMP032I JVM requested System dump using '/home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64le_linux_Nightly/openjdk-tests/TKG/test_output_15987574207831/DaaLoadTest_daa2_ConcurrentScavenge_0/20200829-232106-DaaLoadTest/results/core.20200829.232110.14174.0001.dmp' in response to an event
DLT stderr JVMPORT030W /proc/sys/kernel/core_pattern setting "|/usr/share/apport/apport %p %s %c %d %P" specifies that the core dump is to be piped to an external program.  Attempting to rename either core or core.14224.
DLT stderr 
DLT stderr JVMDUMP010I System dump written to /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64le_linux_Nightly/openjdk-tests/TKG/test_output_15987574207831/DaaLoadTest_daa2_ConcurrentScavenge_0/20200829-232106-DaaLoadTest/results/core.20200829.232110.14174.0001.dmp
DLT stderr JVMDUMP032I JVM requested Java dump using '/home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64le_linux_Nightly/openjdk-tests/TKG/test_output_15987574207831/DaaLoadTest_daa2_ConcurrentScavenge_0/20200829-232106-DaaLoadTest/results/javacore.20200829.232110.14174.0002.txt' in response to an event
STF 23:21:11.296 - Found dump at: /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64le_linux_Nightly/openjdk-tests/TKG/test_output_15987574207831/DaaLoadTest_daa2_ConcurrentScavenge_0/20200829-232106-DaaLoadTest/results/javacore.20200829.232110.14174.0002.txt
STF 23:21:11.296 - Found dump at: /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64le_linux_Nightly/openjdk-tests/TKG/test_output_15987574207831/DaaLoadTest_daa2_ConcurrentScavenge_0/20200829-232106-DaaLoadTest/results/core.20200829.232110.14174.0001.dmp
DLT stderr javacore file generated - /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64le_linux_Nightly/openjdk-tests/TKG/test_output_15987574207831/DaaLoadTest_daa2_ConcurrentScavenge_0/20200829-232106-DaaLoadTest/results/javacore.20200829.232110.14174.0002.txt
DLT stderr core file generated - /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64le_linux_Nightly/openjdk-tests/TKG/test_output_15987574207831/DaaLoadTest_daa2_ConcurrentScavenge_0/20200829-232106-DaaLoadTest/results/core.20200829.232110.14174.0001.dmp
DLT stderr JVMDUMP010I Java dump written to /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64le_linux_Nightly/openjdk-tests/TKG/test_output_15987574207831/DaaLoadTest_daa2_ConcurrentScavenge_0/20200829-232106-DaaLoadTest/results/javacore.20200829.232110.14174.0002.txt
DLT stderr JVMDUMP032I JVM requested Snap dump using '/home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64le_linux_Nightly/openjdk-tests/TKG/test_output_15987574207831/DaaLoadTest_daa2_ConcurrentScavenge_0/20200829-232106-DaaLoadTest/results/Snap.20200829.232110.14174.0003.trc' in response to an event
STF 23:21:12.796 - Found dump at: /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64le_linux_Nightly/openjdk-tests/TKG/test_output_15987574207831/DaaLoadTest_daa2_ConcurrentScavenge_0/20200829-232106-DaaLoadTest/results/Snap.20200829.232110.14174.0003.trc
DLT stderr Snap file generated - /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64le_linux_Nightly/openjdk-tests/TKG/test_output_15987574207831/DaaLoadTest_daa2_ConcurrentScavenge_0/20200829-232106-DaaLoadTest/results/Snap.20200829.232110.14174.0003.trc
DLT stderr JVMDUMP010I Snap dump written to /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64le_linux_Nightly/openjdk-tests/TKG/test_output_15987574207831/DaaLoadTest_daa2_ConcurrentScavenge_0/20200829-232106-DaaLoadTest/results/Snap.20200829.232110.14174.0003.trc
DLT stderr JVMDUMP013I Processed dump event "traceassert", detail "".
STF 23:21:12.919 - **FAILED** Process DLT ended with exit code (255) and not the expected exit code/s (0)
STF 23:21:12.919 - Monitoring Report Summary:
STF 23:21:12.919 -   o Process DLT has crashed unexpectedly
STF 23:21:12.920 - Killing processes: DLT
STF 23:21:12.920 -   o Process DLT is not running
**FAILED** at step 1 (Run daa load test). Expected return value=0 Actual=1 at /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64le_linux_Nightly/openjdk-tests/TKG/../TKG/test_output_15987574207831/DaaLoadTest_daa2_ConcurrentScavenge_0/20200829-232106-DaaLoadTest/execute.pl line 94.
STF 23:21:12.956 - **FAILED** execute script failed. Expected return value=0 Actual=1
STF 23:21:12.956 - 
STF 23:21:12.956 - ====================   T E A R D O W N   ====================
STF 23:21:12.956 - Running teardown: perl /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64le_linux_Nightly/openjdk-tests/TKG/../TKG/test_output_15987574207831/DaaLoadTest_daa2_ConcurrentScavenge_0/20200829-232106-DaaLoadTest/tearDown.pl
STF 23:21:13.017 - TEARDOWN stage completed
STF 23:21:13.019 - 
STF 23:21:13.019 - =====================   R E S U L T S   =====================
STF 23:21:13.019 - Stage results:
STF 23:21:13.019 -   setUp:     pass
STF 23:21:13.019 -   execute:  *fail*
STF 23:21:13.019 -   teardown:  pass
STF 23:21:13.019 - 
STF 23:21:13.020 - Overall result: **FAILED**

DaaLoadTest_daa2_ConcurrentScavenge_0_FAILED

The text was updated successfully, but these errors were encountered:

dmitripivkine · 2020-08-31T18:26:47Z

Corrupted pointer has been discovered in Ownable Synchronizer List (corrupted heap or stall pointer) and takes 0x00740073 ([ s.t. ]) as a next object pointer.

dmitripivkine · 2020-08-31T20:45:16Z

Bogus value has been added to the list by mutator thread (most likely jit code):

> !mm_ownablesynchronizerobjectlist 0x00003FFF940D1900
MM_OwnableSynchronizerObjectList at 0x3fff940d1900 {
  Fields for MM_Base:
  Fields for MM_BaseNonVirtual:
	0x0: const U8* _typeId = !j9x 0x00003FFF99FFF5F0
  Fields for MM_OwnableSynchronizerObjectList:
	0x8: volatile struct J9Object* _head = !j9object 0x0000000000000000
	0x10: struct J9Object* _priorHead = !j9object 0x0000000080C61878<FAULT> <-----
	0x18: class MM_OwnableSynchronizerObjectList* _nextList = !mm_ownablesynchronizerobjectlist 0x00003FFF940D18D0
	0x20: class MM_OwnableSynchronizerObjectList* _previousList = !mm_ownablesynchronizerobjectlist 0x00003FFF940D1930
	0x28: UDATA _objectCount = 0x0000000000000000 (0)
}

0x80C61860 :  001bee00 00000000 80c61870 80c61870 [ ........p...p... ]
0x80C61870 :  001bf400 00000000 00000000 00000000 [ ................ ] <-----
0x80C61880 :  00042100 80c61890 00000062 00000000 [ .!......b....... ]
0x80C61890 :  0003da00 000000b2 00650074 00740073 [ ........t.e.s.t. ] <-----
0x80C618A0 :  00680054 00650072 00420065 00740079 [ T.h.r.e.e.B.y.t. ]

The Ownable Synchronizer List is built using ownableSynchronizerLink field (offset = 8 + 4 header size)
So we tried 0x80C61878 (garbage not marked) and got link to next "element" 0x80c61890 (happen to be valid object). However it is not marked either so we advanced to next one 0x00740073 and asserted.

dmitripivkine · 2020-08-31T20:45:50Z

@andrewcraik FYI

dmitripivkine · 2020-09-14T19:27:41Z

@gita-omr FYI

dmitripivkine · 2020-09-14T19:30:31Z

Actual corruption is in Tenure so it might be not a CS specific (or just some side effect of CS)

mnalam-p · 2020-09-16T17:24:00Z

@dmitripivkine I could not find the DaaLoadTest in openj9/test. Can you kindly suggest me where can I find the test class? I want to run the test locally and see if I can reproduce.

pshipton · 2020-09-16T18:14:15Z

See https://github.com/eclipse/openj9-systemtest

mnalam-p · 2020-09-18T14:21:25Z

I ran the test in question on local system for 200 times and did not see any test failure. Any other suggestion to try to reproduce the failure?

JasonFengJ9 · 2020-09-21T21:54:25Z

Another occurrence at https://ci.eclipse.org/openj9/job/Test_openjdk8_j9_sanity.system_ppc64_aix_Nightly/499/tapResults/
For example, to rebuild the failed tests in =https://ci.eclipse.org/openj9/job/Grinder, use the following links:
00:55:44 https://ci.eclipse.org/openj9/job/Grinder/parambuild/?JDK_VERSION=8&JDK_IMPL=openj9&BUILD_LIST=system&PLATFORM=ppc64_aix&TARGET=DaaLoadTest_daa2_ConcurrentScavenge_0

DLT 23:12:35.735 - Starting thread. Suite=0 thread=13
DLT stderr Unhandled exception
DLT stderr Type=Segmentation error vmState=0x00020002
DLT stderr J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000033
DLT stderr Handler1=09001000A0DEFBB0 Handler2=09001000A0BE3328
DLT stderr R0=0000000000730074 R1=00000100226B5C10 R2=09001000A0DC8EE0 R3=00000100100A8F98
DLT stderr R4=0000000000730074 R5=0000000000000000 R6=0000000080220788 R7=000001001017E3F0
DLT stderr R8=00000000000001B0 R9=00000000000001B0 R10=0000000000000090 R11=0000000000000000
DLT stderr R12=0000000000000000 R13=00000100226BE800 R14=0000000000000000 R15=0000000000000000
DLT stderr R16=0000000000000000 R17=0000000000000000 R18=0000000000000000 R19=0000000000000000
DLT stderr R20=0000000000000001 R21=0000000000000001 R22=0000000090DF00EC R23=0000000000000160
DLT stderr R24=0000000000000150 R25=0000000000000007 R26=0000010010686D30 R27=0000000000000001
DLT stderr R28=0000010010686C50 R29=000000000000000C R30=0000000000730074 R31=0000010022670910
DLT stderr IAR=09000000091556FC LR=090000000917F1F0 MSR=A00000000000D032 CTR=0900000009181A00
DLT stderr CR=2400028B00000000 FPSCR=8202000000000000 XER=0000000082020000
DLT stderr FPR0 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR1 c3e0000000000000 (f: 0.000000, d: -9.223372e+18)
DLT stderr FPR2 41cdcd6500000000 (f: 0.000000, d: 1.000000e+09)
DLT stderr FPR3 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR4 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR5 c3e0000000000000 (f: 0.000000, d: -9.223372e+18)
DLT stderr FPR6 4070b28739071e79 (f: 956767872.000000, d: 2.671580e+02)
DLT stderr FPR7 408f400000000000 (f: 0.000000, d: 1.000000e+03)
DLT stderr FPR8 4103ec6800000000 (f: 0.000000, d: 1.632130e+05)
DLT stderr FPR9 4530000000000000 (f: 0.000000, d: 1.934281e+25)
DLT stderr FPR10 408f400000000000 (f: 0.000000, d: 1.000000e+03)
DLT stderr FPR11 43300000000003e8 (f: 1000.000000, d: 4.503600e+15)
DLT stderr FPR12 4530000000000000 (f: 0.000000, d: 1.934281e+25)
DLT stderr FPR13 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR16 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR17 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR18 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR19 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR20 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR21 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR22 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR23 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR24 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR25 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR26 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR27 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR28 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR29 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR30 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR31 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr Module=/home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix_Nightly/openjdkbinary/j2sdk-image/jre/lib/ppc64/compressedrefs/libj9gc29.so
DLT stderr Module_base_address=090000000912D000
DLT stderr Target=2_90_20200919_526 (AIX 7.1)
DLT stderr CPU=ppc64 (16 logical CPUs) (0x200000000 RAM)
DLT stderr ----------- Stack Backtrace -----------
DLT stderr add__34MM_OwnableSynchronizerObjectBufferFP18MM_EnvironmentBaseP8J9Object+0x44c (0x090000000917F1F0 [libj9gc29.so+0x521f0])
DLT stderr scanOwnableSynchronizerObjects__27MM_MarkingSchemeRootClearerFP18MM_EnvironmentBase+0x440 (0x09000000091FC064 [libj9gc29.so+0xcf064])
DLT stderr scanClearable__14MM_RootScannerFP18MM_EnvironmentBase+0x248 (0x09000000091EB84C [libj9gc29.so+0xbe84c])
DLT stderr workerCompleteGC__18MM_MarkingDelegateFP18MM_EnvironmentBase+0x1a0 (0x09000000091E3204 [libj9gc29.so+0xb6204])
DLT stderr markLiveObjectsComplete__16MM_MarkingSchemeFP18MM_EnvironmentBase+0x10 (0x09000000091DB414 [libj9gc29.so+0xae414])
DLT stderr run__19MM_ParallelMarkTaskFP18MM_EnvironmentBase+0x6c (0x090000000930CC50 [libj9gc29.so+0x1dfc50])
DLT stderr workerEntryPoint__21MM_ParallelDispatcherFP18MM_EnvironmentBase+0x22c (0x090000000929CC30 [libj9gc29.so+0x16fc30])
DLT stderr dispatcher_thread_proc2__FP14OMRPortLibraryPv+0x27c (0x090000000929BD80 [libj9gc29.so+0x16ed80])
DLT stderr omrsig_protect+0x488 (0x0900000007927ECC [libj9prt29.so+0x58ecc])
DLT stderr dispatcher_thread_proc+0x4c (0x090000000929C9B0 [libj9gc29.so+0x16f9b0])
DLT stderr thread_wrapper+0x524 (0x090000000728E808 [libj9thr29.so+0x4808])
DLT stderr _pthread_body+0xf0 (0x0900000000570E14 [libpthreads.a+0x3e14])
DLT stderr ---------------------------------------
DLT stderr JVMDUMP039I Processing dump event "gpf", detail "" at 2020/09/19 23:12:36 - please wait.
DLT stderr JVMDUMP032I JVM requested System dump using '/home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix_Nightly/openjdk-tests/TKG/test_output_16005714406736/DaaLoadTest_daa2_ConcurrentScavenge_0/20200919-231232-DaaLoadTest/results/core.20200919.231236.18153542.0001.dmp' in response to an event
STF 23:12:36.342 - Found dump at: /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix_Nightly/openjdk-tests/TKG/test_output_16005714406736/DaaLoadTest_daa2_ConcurrentScavenge_0/20200919-231232-DaaLoadTest/results/core.20200919.231236.18153542.0001.dmp
DLT stderr core file generated - /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix_Nightly/openjdk-tests/TKG/test_output_16005714406736/DaaLoadTest_daa2_ConcurrentScavenge_0/20200919-231232-DaaLoadTest/results/core.20200919.231236.18153542.0001.dmp
DLT stderr JVMDUMP010I System dump written to /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix_Nightly/openjdk-tests/TKG/test_output_16005714406736/DaaLoadTest_daa2_ConcurrentScavenge_0/20200919-231232-DaaLoadTest/results/core.20200919.231236.18153542.0001.dmp
DLT stderr JVMDUMP032I JVM requested Java dump using '/home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix_Nightly/openjdk-tests/TKG/test_output_16005714406736/DaaLoadTest_daa2_ConcurrentScavenge_0/20200919-231232-DaaLoadTest/results/javacore.20200919.231236.18153542.0002.txt' in response to an event
STF 23:12:45.342 - Found dump at: /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix_Nightly/openjdk-tests/TKG/test_output_16005714406736/DaaLoadTest_daa2_ConcurrentScavenge_0/20200919-231232-DaaLoadTest/results/javacore.20200919.231236.18153542.0002.txt
DLT stderr javacore file generated - /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix_Nightly/openjdk-tests/TKG/test_output_16005714406736/DaaLoadTest_daa2_ConcurrentScavenge_0/20200919-231232-DaaLoadTest/results/javacore.20200919.231236.18153542.0002.txt
DLT stderr JVMDUMP010I Java dump written to /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix_Nightly/openjdk-tests/TKG/test_output_16005714406736/DaaLoadTest_daa2_ConcurrentScavenge_0/20200919-231232-DaaLoadTest/results/javacore.20200919.231236.18153542.0002.txt
DLT stderr JVMDUMP032I JVM requested Snap dump using '/home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix_Nightly/openjdk-tests/TKG/test_output_16005714406736/DaaLoadTest_daa2_ConcurrentScavenge_0/20200919-231232-DaaLoadTest/results/Snap.20200919.231236.18153542.0003.trc' in response to an event
STF 23:12:45.567 - Found dump at: /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix_Nightly/openjdk-tests/TKG/test_output_16005714406736/DaaLoadTest_daa2_ConcurrentScavenge_0/20200919-231232-DaaLoadTest/results/Snap.20200919.231236.18153542.0003.trc
DLT stderr Snap file generated - /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix_Nightly/openjdk-tests/TKG/test_output_16005714406736/DaaLoadTest_daa2_ConcurrentScavenge_0/20200919-231232-DaaLoadTest/results/Snap.20200919.231236.18153542.0003.trc
DLT stderr JVMDUMP010I Snap dump written to /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix_Nightly/openjdk-tests/TKG/test_output_16005714406736/DaaLoadTest_daa2_ConcurrentScavenge_0/20200919-231232-DaaLoadTest/results/Snap.20200919.231236.18153542.0003.trc
DLT stderr JVMDUMP032I JVM requested JIT dump using '/home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix_Nightly/openjdk-tests/TKG/test_output_16005714406736/DaaLoadTest_daa2_ConcurrentScavenge_0/20200919-231232-DaaLoadTest/results/jitdump.20200919.231236.18153542.0004.dmp' in response to an event
DLT stderr JVMDUMP010I JIT dump written to /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix_Nightly/openjdk-tests/TKG/test_output_16005714406736/DaaLoadTest_daa2_ConcurrentScavenge_0/20200919-231232-DaaLoadTest/results/jitdump.20200919.231236.18153542.0004.dmp
DLT stderr JVMDUMP013I Processed dump event "gpf", detail "".
STF 23:12:45.903 - **FAILED** Process DLT ended with exit code (255) and not the expected exit code/s (0)
STF 23:12:45.903 - Monitoring Report Summary:
STF 23:12:45.903 -   o Process DLT has crashed unexpectedly
STF 23:12:45.903 - Killing processes: DLT
STF 23:12:45.903 -   o Process DLT is not running
**FAILED** at step 1 (Run daa load test). Expected return value=0 Actual=1 at /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix_Nightly/openjdk-tests/TKG/../TKG/test_output_16005714406736/DaaLoadTest_daa2_ConcurrentScavenge_0/20200919-231232-DaaLoadTest/execute.pl line 94.
STF 23:12:45.914 - **FAILED** execute script failed. Expected return value=0 Actual=1
STF 23:12:45.914 - 
STF 23:12:45.914 - ====================   T E A R D O W N   ====================
STF 23:12:45.914 - Running teardown: perl /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix_Nightly/openjdk-tests/TKG/../TKG/test_output_16005714406736/DaaLoadTest_daa2_ConcurrentScavenge_0/20200919-231232-DaaLoadTest/tearDown.pl
STF 23:12:46.013 - TEARDOWN stage completed
STF 23:12:46.021 - 
STF 23:12:46.021 - =====================   R E S U L T S   =====================
STF 23:12:46.021 - Stage results:
STF 23:12:46.021 -   setUp:     pass
STF 23:12:46.021 -   execute:  *fail*
STF 23:12:46.021 -   teardown:  pass
STF 23:12:46.021 - 
STF 23:12:46.021 - Overall result: **FAILED**

DaaLoadTest_daa2_ConcurrentScavenge_0_FAILED

This is the error as per #10587 which was closed as dup of this.

mnalam-p · 2020-09-24T15:18:34Z

Was not able to reproduce the crash on grinder (100 serial iterations). All 100 run passed.

mnalam-p · 2020-09-24T17:58:50Z

Was not able to reproduce the crash on grinder after 200 iterations. All 200 run passed

gita-omr · 2020-09-28T17:29:51Z

@JasonFengJ9 @pshipton considering the same build did not fail after 200 runs should we close this issue or at least declare a non-stop-ship?

JasonFengJ9 · 2020-09-28T17:49:30Z

@gita-omr I feel we should keep this issue open to track potential future occurrence for a while. Considering the low failure rate and not reproducible in a grinder, it probably is not qualify for a stop ship defect unless @pshipton disagrees.

pshipton · 2020-09-28T18:08:00Z

This isn't assigned to the 0.23 milestone, or any milestone, so it was never a ship issue.

pshipton · 2020-11-10T13:38:20Z

https://ci.eclipse.org/openj9/job/Test_openjdk8_j9_sanity.system_ppc64le_linux_xl_Nightly/325
DaaLoadTest_daa2_ConcurrentScavenge_0
cent7-ppcle-3

DLT 04:03:51.646 - Starting thread. Suite=0 thread=5
DLT stderr Attempt to access ownable synchronizer object located outside of heap (stack allocated?) 00003FFE68017500
DLT stderr 08:03:53.303 0x3ffe94002200    j9mm.141    *   ** ASSERTION FAILED ** at OwnableSynchronizerObjectBuffer.cpp:100: ((false))
DLT stderr JVMDUMP039I Processing dump event "traceassert", detail "" at 2020/11/10 04:03:53 - please wait.

pshipton · 2020-12-08T02:01:34Z

See additional failures in #11379, which is closed as a dup of this.

lumpfish · 2020-12-08T10:07:30Z

There was a failure on the adoptopenjdk 0.24.0-m1 build which looks like a dup of this: https://ci.adoptopenjdk.net/job/Test_openjdk8_j9_sanity.system_ppc64_aix/98/consoleFull.

13:46:24  openjdk version "1.8.0_282"
13:46:24  OpenJDK Runtime Environment (build 1.8.0_282-202012071159-b04)
13:46:24  Eclipse OpenJ9 VM (build openj9-0.24.0-m1, JRE 1.8.0 AIX ppc64-64-Bit Compressed References 20201207_913 (JIT enabled, AOT enabled)
13:46:24  OpenJ9   - c757b5115
13:46:24  OMR      - 5080dab1e
13:46:24  JCL      - 9910cb2350 based on jdk8u282-b04)

13:46:24  STF 08:46:23.960 - +------ Step 1 - Run daa load test
13:46:24  STF 08:46:23.960 - | Run foreground process
13:46:24  STF 08:46:23.960 - |   Program:     /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/openjdkbinary/j2sdk-image/bin/java
13:46:24  STF 08:46:23.960 - |   Mnemonic:    DLT
13:46:24  STF 08:46:23.960 - |   Echo:        ECHO_ON
13:46:24  STF 08:46:23.960 - |   Expectation: CLEAN_RUN within 3h
13:46:24  STF 08:46:23.960 - |
13:46:24  STF 08:46:23.961 - Running command: /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/openjdkbinary/j2sdk-image/bin/java -Xgcpolicy:gencon -Xgc:concurrentScavenge -classpath /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/jvmtest/system/stf/stf.load/bin:/home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/jvmtest/system/stf/stf.core/bin:/home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/jvmtest/system/systemtest_prereqs/log4j-2.13.3/log4j-api-2.13.3.jar:/home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/jvmtest/system/systemtest_prereqs/log4j-2.13.3/log4j-core-2.13.3.jar:/home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/jvmtest/system/systemtest_prereqs/junit-4.12/junit-4.12.jar:/home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/jvmtest/system/systemtest_prereqs/junit-4.12/hamcrest-core-1.3.jar:/home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/jvmtest/system/openj9-systemtest/openj9.test.daa/bin net.adoptopenjdk.loadTest.LoadTest -resultsDir /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/openjdk-tests/TKG/test_output_16073486251772/DaaLoadTest_daa2_ConcurrentScavenge_0/20201207-084620-DaaLoadTest/results -resultsPrefix 1.DLT. -dumpRequested false -reportFailureLimit 1 -abortAtFailureLimit 10 -maxTotalLogFileSpace 200M -maxSingleLogSize 1/25 -suite.daa.threadCount 14 -suite.daa.totalNumberTests 4200 -suite.daa.inventoryFile /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/openjdk-tests/TKG/test_output_16073486251772/DaaLoadTest_daa2_ConcurrentScavenge_0/20201207-084620-DaaLoadTest/results/1.DLT.inventory/openj9.test.load/config/inventories/daa/daa2.xml -suite.daa.inventoryExcludeFile none -suite.daa.selection random -suite.daa.seed -1 -suite.daa.repeatCount 1 -suite.daa.thinkingTime 0ms..0ms
13:46:24  STF 08:46:23.961 - Redirecting stderr to /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/openjdk-tests/TKG/test_output_16073486251772/DaaLoadTest_daa2_ConcurrentScavenge_0/20201207-084620-DaaLoadTest/results/1.DLT.stderr
13:46:24  STF 08:46:23.961 - Redirecting stdout to /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/openjdk-tests/TKG/test_output_16073486251772/DaaLoadTest_daa2_ConcurrentScavenge_0/20201207-084620-DaaLoadTest/results/1.DLT.stdout
13:46:24  STF 08:46:23.980 - Monitoring processes: DLT
13:46:25  DLT 08:46:25.243 - Load test parameters
13:46:25  DLT 08:46:25.246 -   Time limited         = false
13:46:25  DLT 08:46:25.247 -   abortIfOutOfMemory   = true
13:46:25  DLT 08:46:25.247 -   reportFailureLimit   = 1
13:46:25  DLT 08:46:25.247 -   abortAtFailureLimit  = 10
13:46:25  DLT 08:46:25.247 -   maxTotalLogFileSpace = 209715200
13:46:25  DLT 08:46:25.248 -   maxSingleLogSize     = 8388608
13:46:25  DLT 08:46:25.248 - Parameters for suite 0
13:46:25  DLT 08:46:25.248 -   Suite name     = daa
13:46:25  DLT 08:46:25.248 -   Number threads = 14
13:46:25  DLT 08:46:25.248 -   Supplied seed  = -1
13:46:25  DLT 08:46:25.248 -   Inventory file = /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/openjdk-tests/TKG/test_output_16073486251772/DaaLoadTest_daa2_ConcurrentScavenge_0/20201207-084620-DaaLoadTest/results/1.DLT.inventory/openj9.test.load/config/inventories/daa/daa2.xml
13:46:25  DLT 08:46:25.249 -   Exclude file   = none
13:46:25  DLT 08:46:25.249 -   Number tests   = 4200
13:46:25  DLT 08:46:25.249 -   Repeat count   = 1
13:46:25  DLT 08:46:25.249 -   Thinking time  = 0ms..0ms
13:46:25  DLT 08:46:25.250 -   Selection mode = random
13:46:25  DLT 08:46:25.250 -   Actual seed    = 1609351252360
13:46:25  DLT 08:46:25.258 - Parsing inventory file. Root=/home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/openjdk-tests/TKG/test_output_16073486251772/DaaLoadTest_daa2_ConcurrentScavenge_0/20201207-084620-DaaLoadTest/results/1.DLT.inventory File=openj9.test.load/config/inventories/daa/daa2.xml
13:46:25  DLT 08:46:25.277 - Final test list:
13:46:25  DLT 08:46:25.278 -   0 ArbitraryJava[net.openj9.test.decimals.DecimalTestRunner invokeTest]  Weighting=1 
13:46:25  DLT 08:46:25.278 -   1 ArbitraryJava[net.openj9.test.MarshallingTestRunner invokeTest]  Weighting=1 
13:46:25  DLT 08:46:25.278 -   2 ArbitraryJava[net.openj9.test.PD2Primitive.I2PD_PD2ITestRunner invokeTest]  Weighting=1 
13:46:25  DLT 08:46:25.278 -   3 JUnit[net.openj9.test.arithmetics.TestArithmeticOperations]  Weighting=1 
13:46:25  DLT 08:46:25.278 -   4 JUnit[net.openj9.test.binaryData.TestBinaryConverters]  Weighting=1 
13:46:25  DLT 08:46:25.279 -   5 JUnit[net.openj9.test.binaryData.TestByteArray2Double]  Weighting=1 
13:46:25  DLT 08:46:25.279 -   6 JUnit[net.openj9.test.binaryData.TestByteArray2Float]  Weighting=1 
13:46:25  DLT 08:46:25.279 -   7 JUnit[net.openj9.test.binaryData.TestByteArray2Integer]  Weighting=1 
13:46:25  DLT 08:46:25.279 -   8 JUnit[net.openj9.test.binaryData.TestByteArray2IntegerNumBytes]  Weighting=1 
13:46:25  DLT 08:46:25.279 -   9 JUnit[net.openj9.test.binaryData.TestByteArray2Long]  Weighting=1 
13:46:25  DLT 08:46:25.280 -   10 JUnit[net.openj9.test.binaryData.TestByteArray2LongNumBytes]  Weighting=1 
13:46:25  DLT 08:46:25.280 -   11 JUnit[net.openj9.test.binaryData.TestByteArray2Short]  Weighting=1 
13:46:25  DLT 08:46:25.280 -   12 JUnit[net.openj9.test.binaryData.TestByteArray2ShortNumBytes]  Weighting=1 
13:46:25  DLT 08:46:25.280 -   13 JUnit[net.openj9.test.binaryData.TestDouble2ByteArray]  Weighting=1 
13:46:25  DLT 08:46:25.280 -   14 JUnit[net.openj9.test.binaryData.TestFloat2ByteArray]  Weighting=1 
13:46:25  DLT 08:46:25.281 -   15 JUnit[net.openj9.test.binaryData.TestInteger2ByteArray]  Weighting=1 
13:46:25  DLT 08:46:25.281 -   16 JUnit[net.openj9.test.binaryData.TestInteger2ByteArrayNumBytes]  Weighting=1 
13:46:25  DLT 08:46:25.281 -   17 JUnit[net.openj9.test.binaryData.TestLong2ByteArray]  Weighting=1 
13:46:25  DLT 08:46:25.281 -   18 JUnit[net.openj9.test.binaryData.TestLong2ByteArrayNumBytes]  Weighting=1 
13:46:25  DLT 08:46:25.281 -   19 JUnit[net.openj9.test.binaryData.TestShort2ByteArray]  Weighting=1 
13:46:25  DLT 08:46:25.282 -   20 JUnit[net.openj9.test.binaryData.TestShort2ByteArrayNumBytes]  Weighting=1 
13:46:25  DLT 08:46:25.282 -   21 JUnit[net.openj9.test.decimals.TestED2PD]  Weighting=1 
13:46:25  DLT 08:46:25.282 -   22 JUnit[net.openj9.test.decimals.TestPD2ED]  Weighting=1 
13:46:25  DLT 08:46:25.282 -   23 JUnit[net.openj9.test.decimals.TestPD2UD]  Weighting=1 
13:46:25  DLT 08:46:25.283 -   24 JUnit[net.openj9.test.decimals.TestUD2PD]  Weighting=1 
13:46:25  DLT 08:46:25.283 -   25 JUnit[net.openj9.test.PD2Primitive.RandomTests]  Weighting=1 
13:46:25  DLT 08:46:25.283 -   26 JUnit[net.openj9.test.PD2Primitive.TestI2PD]  Weighting=1 
13:46:25  DLT 08:46:25.283 -   27 JUnit[net.openj9.test.PD2Primitive.TestPD2I]  Weighting=1 
13:46:25  DLT 08:46:25.336 - Starting thread. Suite=0 thread=0
13:46:25  DLT 08:46:25.338 - Starting thread. Suite=0 thread=1
13:46:25  DLT 08:46:25.339 - Starting thread. Suite=0 thread=2
13:46:25  DLT 08:46:25.340 - Starting thread. Suite=0 thread=3
13:46:25  DLT 08:46:25.340 - Starting thread. Suite=0 thread=4
13:46:25  DLT 08:46:25.341 - Starting thread. Suite=0 thread=5
13:46:25  DLT 08:46:25.342 - Starting thread. Suite=0 thread=6
13:46:25  DLT 08:46:25.343 - Starting thread. Suite=0 thread=7
13:46:25  DLT 08:46:25.343 - Starting thread. Suite=0 thread=8
13:46:25  DLT 08:46:25.344 - Starting thread. Suite=0 thread=9
13:46:25  DLT 08:46:25.344 - Starting thread. Suite=0 thread=10
13:46:25  DLT 08:46:25.345 - Starting thread. Suite=0 thread=11
13:46:25  DLT 08:46:25.345 - Starting thread. Suite=0 thread=12
13:46:25  DLT 08:46:25.346 - Starting thread. Suite=0 thread=13
13:46:26  DLT stderr Unhandled exception
13:46:26  DLT stderr Type=Segmentation error vmState=0x00020002
13:46:26  DLT stderr J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000033
13:46:26  DLT stderr Handler1=09001000A0B47BA0 Handler2=09001000A0A42928
13:46:26  DLT stderr R0=0000000000730074 R1=0000010022529C10 R2=09001000A0A6B350 R3=00000100100A8AE8
13:46:26  DLT stderr R4=0000000000730074 R5=0000000000000000 R6=00000000802F73A0 R7=000001001017EDD0
13:46:26  DLT stderr R8=00000000000002A0 R9=00000000000002A0 R10=00000000000000E0 R11=0000000000000000
13:46:26  DLT stderr R12=0000000000000000 R13=0000010022532800 R14=0000000000000000 R15=0000000000000000
13:46:26  DLT stderr R16=0000000000000000 R17=0000000000000000 R18=0000000000000000 R19=0000000000000000
13:46:26  DLT stderr R20=0000000000000001 R21=0000000000000001 R22=0000000090DF00EC R23=0000000000000160
13:46:26  DLT stderr R24=0000000000000150 R25=0000000000000007 R26=0000010010687750 R27=0000000000000001
13:46:26  DLT stderr R28=0000010010687670 R29=000000000000000C R30=0000000000730074 R31=0000010022530B30
13:46:26  DLT stderr IAR=0900000004FFE5FC LR=0900000005027DF0 MSR=A00000000000D032 CTR=090000000502A600
13:46:26  DLT stderr CR=2400008B40000000 FPSCR=8202000000000000 XER=4000000082020000
13:46:26  DLT stderr FPR0 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:46:26  DLT stderr FPR1 c3e0000000000000 (f: 0.000000, d: -9.223372e+18)
13:46:26  DLT stderr FPR2 41cdcd6500000000 (f: 0.000000, d: 1.000000e+09)
13:46:26  DLT stderr FPR3 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:46:26  DLT stderr FPR4 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:46:26  DLT stderr FPR5 c3e0000000000000 (f: 0.000000, d: -9.223372e+18)
13:46:26  DLT stderr FPR6 4070b23eed1c19ee (f: 3978041856.000000, d: 2.671404e+02)
13:46:26  DLT stderr FPR7 408f400000000000 (f: 0.000000, d: 1.000000e+03)
13:46:26  DLT stderr FPR8 40f567c000000000 (f: 0.000000, d: 8.767600e+04)
13:46:26  DLT stderr FPR9 4530000000000000 (f: 0.000000, d: 1.934281e+25)
13:46:26  DLT stderr FPR10 408f400000000000 (f: 0.000000, d: 1.000000e+03)
13:46:26  DLT stderr FPR11 43300000000003e8 (f: 1000.000000, d: 4.503600e+15)
13:46:26  DLT stderr FPR12 4530000000000000 (f: 0.000000, d: 1.934281e+25)
13:46:26  DLT stderr FPR13 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:46:26  DLT stderr FPR14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:46:26  DLT stderr FPR15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:46:26  DLT stderr FPR16 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:46:26  DLT stderr FPR17 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:46:26  DLT stderr FPR18 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:46:26  DLT stderr FPR19 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:46:26  DLT stderr FPR20 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:46:26  DLT stderr FPR21 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:46:26  DLT stderr FPR22 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:46:26  DLT stderr FPR23 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:46:26  DLT stderr FPR24 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:46:26  DLT stderr FPR25 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:46:26  DLT stderr FPR26 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:46:26  DLT stderr FPR27 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:46:26  DLT stderr FPR28 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:46:26  DLT stderr FPR29 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:46:26  DLT stderr FPR30 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:46:26  DLT stderr FPR31 0000000000000000 (f: 0.000000, d: 0.000000e+00)
13:46:26  DLT stderr Module=/home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/openjdkbinary/j2sdk-image/jre/lib/ppc64/compressedrefs/libj9gc29.so
13:46:26  DLT stderr Module_base_address=0900000004FD6000
13:46:26  DLT stderr Target=2_90_20201207_913 (AIX 7.1)
13:46:26  DLT stderr CPU=ppc64 (16 logical CPUs) (0x200000000 RAM)
13:46:26  DLT stderr ----------- Stack Backtrace -----------
13:46:26  DLT stderr add__34MM_OwnableSynchronizerObjectBufferFP18MM_EnvironmentBaseP8J9Object+0x44c (0x0900000005027DF0 [libj9gc29.so+0x51df0])
13:46:26  DLT stderr scanOwnableSynchronizerObjects__27MM_MarkingSchemeRootClearerFP18MM_EnvironmentBase+0x440 (0x09000000050A48E4 [libj9gc29.so+0xce8e4])
13:46:26  DLT stderr scanClearable__14MM_RootScannerFP18MM_EnvironmentBase+0x248 (0x09000000050940CC [libj9gc29.so+0xbe0cc])
13:46:26  DLT stderr workerCompleteGC__18MM_MarkingDelegateFP18MM_EnvironmentBase+0x1a0 (0x090000000508BA84 [libj9gc29.so+0xb5a84])
13:46:26  DLT stderr markLiveObjectsComplete__16MM_MarkingSchemeFP18MM_EnvironmentBase+0x10 (0x0900000005083C94 [libj9gc29.so+0xadc94])
13:46:26  DLT stderr run__19MM_ParallelMarkTaskFP18MM_EnvironmentBase+0x6c (0x09000000051B54D0 [libj9gc29.so+0x1df4d0])
13:46:26  DLT stderr workerEntryPoint__21MM_ParallelDispatcherFP18MM_EnvironmentBase+0x22c (0x09000000051453B0 [libj9gc29.so+0x16f3b0])
13:46:26  DLT stderr dispatcher_thread_proc2__FP14OMRPortLibraryPv+0x27c (0x0900000005144500 [libj9gc29.so+0x16e500])
13:46:26  DLT stderr omrsig_protect+0x488 (0x0900000004C17CCC [libj9prt29.so+0x59ccc])
13:46:26  DLT stderr dispatcher_thread_proc+0x4c (0x0900000005145130 [libj9gc29.so+0x16f130])
13:46:26  DLT stderr thread_wrapper+0x33c (0x09000000005DC700 [libj9thr29.so+0x4700])
13:46:26  DLT stderr _pthread_body+0xf0 (0x0900000000570E14 [libpthread.a+0x3e14])
13:46:26  DLT stderr ---------------------------------------
13:46:26  DLT stderr JVMDUMP039I Processing dump event "gpf", detail "" at 2020/12/07 08:46:26 - please wait.
13:46:26  DLT stderr JVMDUMP032I JVM requested System dump using '/home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/openjdk-tests/TKG/test_output_16073486251772/DaaLoadTest_daa2_ConcurrentScavenge_0/20201207-084620-DaaLoadTest/results/core.20201207.084626.18743360.0001.dmp' in response to an event
13:46:26  STF 08:46:26.364 - Found dump at: /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/openjdk-tests/TKG/test_output_16073486251772/DaaLoadTest_daa2_ConcurrentScavenge_0/20201207-084620-DaaLoadTest/results/core.20201207.084626.18743360.0001.dmp
13:46:26  DLT stderr core file generated - /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/openjdk-tests/TKG/test_output_16073486251772/DaaLoadTest_daa2_ConcurrentScavenge_0/20201207-084620-DaaLoadTest/results/core.20201207.084626.18743360.0001.dmp
13:46:37  DLT stderr JVMDUMP010I System dump written to /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/openjdk-tests/TKG/test_output_16073486251772/DaaLoadTest_daa2_ConcurrentScavenge_0/20201207-084620-DaaLoadTest/results/core.20201207.084626.18743360.0001.dmp
13:46:37  DLT stderr JVMDUMP032I JVM requested Java dump using '/home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/openjdk-tests/TKG/test_output_16073486251772/DaaLoadTest_daa2_ConcurrentScavenge_0/20201207-084620-DaaLoadTest/results/javacore.20201207.084626.18743360.0002.txt' in response to an event
13:46:37  STF 08:46:36.944 - Found dump at: /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/openjdk-tests/TKG/test_output_16073486251772/DaaLoadTest_daa2_ConcurrentScavenge_0/20201207-084620-DaaLoadTest/results/javacore.20201207.084626.18743360.0002.txt
13:46:37  DLT stderr javacore file generated - /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/openjdk-tests/TKG/test_output_16073486251772/DaaLoadTest_daa2_ConcurrentScavenge_0/20201207-084620-DaaLoadTest/results/javacore.20201207.084626.18743360.0002.txt
13:46:37  DLT stderr JVMDUMP010I Java dump written to /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/openjdk-tests/TKG/test_output_16073486251772/DaaLoadTest_daa2_ConcurrentScavenge_0/20201207-084620-DaaLoadTest/results/javacore.20201207.084626.18743360.0002.txt
13:46:37  DLT stderr JVMDUMP032I JVM requested Snap dump using '/home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/openjdk-tests/TKG/test_output_16073486251772/DaaLoadTest_daa2_ConcurrentScavenge_0/20201207-084620-DaaLoadTest/results/Snap.20201207.084626.18743360.0003.trc' in response to an event
13:46:37  STF 08:46:37.220 - Found dump at: /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/openjdk-tests/TKG/test_output_16073486251772/DaaLoadTest_daa2_ConcurrentScavenge_0/20201207-084620-DaaLoadTest/results/Snap.20201207.084626.18743360.0003.trc
13:46:37  DLT stderr Snap file generated - /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/openjdk-tests/TKG/test_output_16073486251772/DaaLoadTest_daa2_ConcurrentScavenge_0/20201207-084620-DaaLoadTest/results/Snap.20201207.084626.18743360.0003.trc
13:46:37  DLT stderr JVMDUMP010I Snap dump written to /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/openjdk-tests/TKG/test_output_16073486251772/DaaLoadTest_daa2_ConcurrentScavenge_0/20201207-084620-DaaLoadTest/results/Snap.20201207.084626.18743360.0003.trc
13:46:37  DLT stderr JVMDUMP032I JVM requested JIT dump using '/home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/openjdk-tests/TKG/test_output_16073486251772/DaaLoadTest_daa2_ConcurrentScavenge_0/20201207-084620-DaaLoadTest/results/jitdump.20201207.084626.18743360.0004.dmp' in response to an event
13:46:37  DLT stderr JVMDUMP010I JIT dump written to /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/openjdk-tests/TKG/test_output_16073486251772/DaaLoadTest_daa2_ConcurrentScavenge_0/20201207-084620-DaaLoadTest/results/jitdump.20201207.084626.18743360.0004.dmp
13:46:37  DLT stderr JVMDUMP013I Processed dump event "gpf", detail "".
13:46:37  STF 08:46:37.648 - **FAILED** Process DLT ended with exit code (255) and not the expected exit code/s (0)
13:46:37  STF 08:46:37.648 - Monitoring Report Summary:
13:46:37  STF 08:46:37.648 -   o Process DLT has crashed unexpectedly
13:46:37  STF 08:46:37.648 - Killing processes: DLT
13:46:37  STF 08:46:37.648 -   o Process DLT is not running
13:46:37  **FAILED** at step 1 (Run daa load test). Expected return value=0 Actual=1 at /home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix/openjdk-tests/TKG/../TKG/test_output_16073486251772/DaaLoadTest_daa2_ConcurrentScavenge_0/20201207-084620-DaaLoadTest/execute.pl line 94.

pshipton · 2021-01-15T15:16:15Z

https://ci.eclipse.org/openj9/job/Test_openjdk8_j9_sanity.system_ppc64_aix_Nightly/612
DaaLoadTest_daa1_ConcurrentScavenge_0

DLT stderr Unhandled exception
DLT stderr Type=Segmentation error vmState=0x00020002
DLT stderr J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000033
DLT stderr Handler1=09001000A136EC80 Handler2=09001000A1345810
DLT stderr R0=0000000000000001 R1=00000100225C8C10 R2=09001000A1265258 R3=00000100100A9558
DLT stderr R4=0000000000000001 R5=0000000000000000 R6=0000000080A004B8 R7=00000100101805D0
DLT stderr R8=0000000000000090 R9=0000000000000090 R10=0000000000000030 R11=0000000000000000
DLT stderr R12=0000000000000000 R13=00000100225D1800 R14=0000000000000000 R15=0000000000000000
DLT stderr R16=0000000000000000 R17=0000000000000000 R18=0000000000000000 R19=0000000000000000
DLT stderr R20=0000000000000001 R21=0000000000000001 R22=0000000000000000 R23=00000000000000D0
DLT stderr R24=00000000000000C0 R25=0000000000000004 R26=0000010010688450 R27=0000000000000001
DLT stderr R28=0000010010688370 R29=000000000000000C R30=0000000000000001 R31=0000010022583BF0
DLT stderr IAR=0900000013ED06FC LR=0900000013EFA270 MSR=A00000000000D032 CTR=0900000013EFCA80
DLT stderr CR=2400024B20000001 FPSCR=8202000000000000 XER=2000000182020000
DLT stderr FPR0 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR1 c3e0000000000000 (f: 0.000000, d: -9.223372e+18)
DLT stderr FPR2 41cdcd6500000000 (f: 0.000000, d: 1.000000e+09)
DLT stderr FPR3 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR4 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR5 c3e0000000000000 (f: 0.000000, d: -9.223372e+18)
DLT stderr FPR6 4070b1f20d4afc0c (f: 223018000.000000, d: 2.671216e+02)
DLT stderr FPR7 408f400000000000 (f: 0.000000, d: 1.000000e+03)
DLT stderr FPR8 4104161000000000 (f: 0.000000, d: 1.645460e+05)
DLT stderr FPR9 4530000000000000 (f: 0.000000, d: 1.934281e+25)
DLT stderr FPR10 408f400000000000 (f: 0.000000, d: 1.000000e+03)
DLT stderr FPR11 43300000000003e8 (f: 1000.000000, d: 4.503600e+15)
DLT stderr FPR12 4530000000000000 (f: 0.000000, d: 1.934281e+25)
DLT stderr FPR13 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR16 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR17 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR18 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR19 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR20 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR21 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR22 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR23 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR24 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR25 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR26 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR27 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR28 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR29 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR30 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR31 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr Module=/home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix_Nightly_testList_1/openjdkbinary/j2sdk-image/jre/lib/ppc64/compressedrefs/libj9gc29.so
DLT stderr Module_base_address=0900000013EA8000
DLT stderr Target=2_90_20210114_641 (AIX 7.1)
DLT stderr CPU=ppc64 (16 logical CPUs) (0x200000000 RAM)
DLT stderr ----------- Stack Backtrace -----------
DLT stderr add__34MM_OwnableSynchronizerObjectBufferFP18MM_EnvironmentBaseP8J9Object+0x44c (0x0900000013EFA270 [libj9gc29.so+0x52270])
DLT stderr scanOwnableSynchronizerObjects__27MM_MarkingSchemeRootClearerFP18MM_EnvironmentBase+0x440 (0x0900000013F77164 [libj9gc29.so+0xcf164])
DLT stderr scanClearable__14MM_RootScannerFP18MM_EnvironmentBase+0x248 (0x0900000013F6694C [libj9gc29.so+0xbe94c])
DLT stderr workerCompleteGC__18MM_MarkingDelegateFP18MM_EnvironmentBase+0x1a0 (0x0900000013F5E304 [libj9gc29.so+0xb6304])
DLT stderr markLiveObjectsComplete__16MM_MarkingSchemeFP18MM_EnvironmentBase+0x10 (0x0900000013F56514 [libj9gc29.so+0xae514])
DLT stderr run__19MM_ParallelMarkTaskFP18MM_EnvironmentBase+0x6c (0x0900000014088750 [libj9gc29.so+0x1e0750])
DLT stderr workerEntryPoint__21MM_ParallelDispatcherFP18MM_EnvironmentBase+0x22c (0x0900000014017FB0 [libj9gc29.so+0x16ffb0])
DLT stderr dispatcher_thread_proc2__FP14OMRPortLibraryPv+0x27c (0x0900000014017100 [libj9gc29.so+0x16f100])
DLT stderr omrsig_protect+0x488 (0x0900000012920BCC [libj9prt29.so+0x59bcc])
DLT stderr dispatcher_thread_proc+0x4c (0x0900000014017D30 [libj9gc29.so+0x16fd30])
DLT stderr thread_wrapper+0x524 (0x09000000128A7968 [libj9thr29.so+0x4968])
DLT stderr _pthread_body+0xf0 (0x0900000000570E14 [libpthreads.a+0x3e14])
DLT stderr ---------------------------------------

dmitripivkine · 2021-02-02T19:00:51Z

I believe this is PPC specific problem, so adding proper tag

pshipton · 2021-04-20T12:58:11Z

https://ci.eclipse.org/openj9/job/Test_openjdk8_j9_sanity.system_ppc64le_linux_Nightly_testList_1/106 - ub16p8j94
DaaLoadTest_daa2_CS_5m_0
variation: -Xgcpolicy:gencon -Xgc:concurrentScavenge
JVM_OPTIONS: -Xgcpolicy:gencon -Xgc:concurrentScavenge

https://140-211-168-230-openstack.osuosl.org/artifactory/ci-eclipse-openj9/Test/Test_openjdk8_j9_sanity.system_ppc64le_linux_Nightly_testList_1/106/system_test_output.tar.gz

DLT stderr Unhandled exception
DLT stderr Type=Segmentation error vmState=0x00020002
DLT stderr J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
DLT stderr Handler1=00003FFFB0E933E0 Handler2=00003FFFB0DCC1C0
DLT stderr R0=00003FFFB05B1920 R1=00003FFF91BFBDF0 R2=00003FFFB06C5600 R3=00003FFFAC070B70
DLT stderr R4=00000000000001FF R5=0000000000000006 R6=00003FFFA90D0000 R7=00003FFFAC053520
DLT stderr R8=000000000000000C R9=00003FFFAC02A478 R10=00003FFFAC06F1D0 R11=00000000000001FF
DLT stderr R12=00003FFFB0516160 R13=00003FFF91C06900 R14=0000000000000000 R15=0000000000000000
DLT stderr R16=00003FFF91BC0000 R17=0000000000040000 R18=00003FFFB1F64390 R19=00003FFF91BFF900
DLT stderr R20=00003FFFAC1A8C98 R21=00003FFFAC1A9100 R22=00003FFF91BFF260 R23=00003FFF91BFBF30
DLT stderr R24=00003FFF91BFBE58 R25=0000000000000001 R26=00003FFFAC0E7730 R27=00003FFFAC0E7810
DLT stderr R28=00000000000001B0 R29=0000000000000009 R30=00003FFF30001AC8 R31=0000000000000006
DLT stderr NIP=00003FFFB05B18E8 MSR=800000010280F033 ORIG_GPR3=00000000000081C8 CTR=00003FFFB0516160
DLT stderr LINK=00003FFFB05B1920 XER=0000000000000000 CCR=0000000022224244 SOFTE=0000000000000001
DLT stderr TRAP=0000000000000300 DAR=0000000000000012 dsisr=0000000040000000 RESULT=0000000000000000
DLT stderr FPR0 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR1 4054420920000000 (f: 536870912.000000, d: 8.103181e+01)
DLT stderr FPR2 3fe8000000000000 (f: 0.000000, d: 7.500000e-01)
DLT stderr FPR3 bf3d4a1000000000 (f: 0.000000, d: -4.469194e-04)
DLT stderr FPR4 3f499a0000000000 (f: 0.000000, d: 7.812977e-04)
DLT stderr FPR5 bf94724ea0000000 (f: 2684354560.000000, d: -1.996730e-02)
DLT stderr FPR6 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR7 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR8 00bb0b009f000011 (f: 2667577344.000000, d: 3.851048e-305)
DLT stderr FPR9 00bb2800b7b0010a (f: 3081765120.000000, d: 3.867180e-305)
DLT stderr FPR10 3fcf240000000000 (f: 0.000000, d: 2.432861e-01)
DLT stderr FPR11 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR12 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR13 bfb7440000000000 (f: 0.000000, d: -9.088135e-02)
DLT stderr FPR14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR16 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR17 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR18 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR19 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR20 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR21 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR22 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR23 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR24 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR25 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR26 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR27 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR28 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR29 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR30 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR31 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr Module=/home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64le_linux_Nightly_testList_1/openjdkbinary/j2sdk-image/jre/lib/ppc64le/default/libj9gc29.so
DLT stderr Module_base_address=00003FFFB0390000
DLT stderr Target=2_90_20210419_701 (Linux 4.4.0-173-generic)
DLT stderr CPU=ppc64le (16 logical CPUs) (0x1fe090000 RAM)
DLT stderr ----------- Stack Backtrace -----------
DLT stderr (0x00003FFFB05B18E8 [libj9gc29.so+0x2218e8])
DLT stderr (0x00003FFFB05B1920 [libj9gc29.so+0x221920])
DLT stderr (0x00003FFFB03DCCAC [libj9gc29.so+0x4ccac])
DLT stderr (0x00003FFFB05ACAD8 [libj9gc29.so+0x21cad8])
DLT stderr (0x00003FFFB04F772C [libj9gc29.so+0x16772c])
DLT stderr (0x00003FFFB05E3610 [libj9gc29.so+0x253610])
DLT stderr (0x00003FFFB0515344 [libj9gc29.so+0x185344])
DLT stderr (0x00003FFFB0514718 [libj9gc29.so+0x184718])
DLT stderr (0x00003FFFB0DCD5C8 [libj9prt29.so+0x3d5c8])
DLT stderr (0x00003FFFB0513E90 [libj9gc29.so+0x183e90])
DLT stderr (0x00003FFFB0D613E8 [libj9thr29.so+0x113e8])
DLT stderr (0x00003FFFB1F38040 [libpthread.so.0+0x8040])
DLT stderr clone+0x98 (0x00003FFFB1E13BB0 [libc.so.6+0x123bb0])
DLT stderr ---------------------------------------

pshipton · 2022-01-10T17:49:57Z

https://openj9-jenkins.osuosl.org/job/Test_openjdk8_j9_sanity.system_ppc64le_linux_Nightly_testList_0/177 - ub18-ppcle-2
DaaLoadTest_daa2_CS_5m_0 -Xgcpolicy:gencon -Xgc:concurrentScavenge

https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Test/Test_openjdk8_j9_sanity.system_ppc64le_linux_Nightly_testList_0/177/system_test_output.tar.gz

DLT stderr Attempt to access ownable synchronizer object located outside of heap (stack allocated?) 0000000000450065
DLT stderr 03:47:47.558 0x196f00    j9mm.141    *   ** ASSERTION FAILED ** at /home/jenkins/workspace/Build_JDK8_ppc64le_linux_Nightly/openj9/runtime/gc_base/OwnableSynchronizerObjectBuffer.cpp:100: ((false))

JasonFengJ9 · 2022-01-26T14:38:35Z

Appeared at 0.30.0 release build job/Test_openjdk8_j9_sanity.system_x86-64_mac_testList_2/142/ (mac10x64rt04)

openjdk version "1.8.0_322"
IBM Semeru Runtime Open Edition (build 1.8.0_322-b06)
Eclipse OpenJ9 VM (build openj9-0.30.0-rc1, JRE 1.8.0 Mac OS X amd64-64-Bit Compressed References 20220125_305 (JIT enabled, AOT enabled)
OpenJ9   - f441547b8
OMR      - dac962a28
JCL      - 3273d1d5e1 based on jdk8u322-b06)

[2022-01-26T05:36:43.893Z] Running test DaaLoadTest_daa2_CS_5m_0 ...
[2022-01-26T05:36:43.893Z] ===============================================

[2022-01-26T05:36:43.893Z] variation: -Xgcpolicy:gencon -Xgc:concurrentScavenge
[2022-01-26T05:36:43.893Z] JVM_OPTIONS:  -Xgcpolicy:gencon -Xgc:concurrentScavenge 

[2022-01-26T05:36:46.357Z] DLT 00:36:45.942 - Starting thread. Suite=0 thread=5
[2022-01-26T05:36:48.931Z] DLT stderr Attempt to access ownable synchronizer object located outside of heap (stack allocated?) 0000000003580318
[2022-01-26T05:36:48.931Z] DLT stderr 05:36:48.591 0x1221f400    j9mm.141    *   ** ASSERTION FAILED ** at /Users/jenkins/workspace/build-scripts/jobs/jdk8u/jdk8u-mac-x64-openj9/workspace/build/src/openj9/runtime/gc_base/OwnableSynchronizerObjectBuffer.cpp:100: ((false))
[2022-01-26T05:36:48.931Z] DLT stderr JVMDUMP039I Processing dump event "traceassert", detail "" at 2022/01/26 00:36:48 - please wait.

[2022-01-26T05:36:53.288Z] DaaLoadTest_daa2_CS_5m_0_FAILED

pshipton · 2022-03-07T15:38:21Z

https://openj9-jenkins.osuosl.org/job/Test_openjdk8_j9_sanity.system_x86-64_mac_Nightly_testList_1/225 - osx1014-x86-3
DaaLoadTest_daa2_CS_5m_0 -Xgcpolicy:gencon -Xgc:concurrentScavenge

https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Test/Test_openjdk8_j9_sanity.system_x86-64_mac_Nightly_testList_1/225/system_test_output.tar.gz

DLT stderr Unhandled exception
DLT stderr Type=Segmentation error vmState=0x00020002
DLT stderr J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
DLT stderr Handler1=0000000004E36850 Handler2=000000000510E0E0 InaccessibleAddress=000000000328037C
DLT stderr RDI=0000000000000014 RSI=000000000000352A RAX=000000000000000C RBX=00007FCB2F04EE68
DLT stderr RCX=0000000000000003 RDX=0000000003280370 R8=00007FCB2EE04C30 R9=00007FCB3380B620
DLT stderr R10=00007FCB2EE082A8 R11=0000000000000206 R12=0000000003280370 R13=0000000000000030
DLT stderr R14=000070000DD9DC48 R15=0000000000000001
DLT stderr RIP=0000000006D40FEF GS=0000 FS=0000 RSP=000070000DD9DBA0
DLT stderr RFlags=0000000000010212 CS=002B RBP=000070000DD9DC10 ERR=0328037C00000004
DLT stderr TRAPNO=000000040000000E CPU=037C000000040000 FAULTVADDR=000000000328037C
DLT stderr XMM0 0000000000000004 (f: 4.000000, d: 1.976263e-323)
DLT stderr XMM1 000000000000002f (f: 47.000000, d: 2.322109e-322)
DLT stderr XMM2 0000000000000005 (f: 5.000000, d: 2.470328e-323)
DLT stderr XMM3 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr XMM4 000003f6000003fe (f: 1022.000000, d: 2.151704e-311)
DLT stderr XMM5 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr XMM6 3fe0389eefce633b (f: 4023280384.000000, d: 5.069117e-01)
DLT stderr XMM7 402a56ef8ec924cc (f: 2395546880.000000, d: 1.316980e+01)
DLT stderr XMM8 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr XMM9 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr XMM10 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr XMM11 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr XMM12 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr XMM13 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr XMM14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr XMM15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr Module=/Users/jenkins/workspace/Test_openjdk8_j9_sanity.system_x86-64_mac_Nightly_testList_1/openjdkbinary/j2sdk-image/jre/lib/default/libj9gc29.dylib
DLT stderr Module_base_address=0000000006C7B000 Symbol=_ZN27MM_MarkingSchemeRootClearer30scanOwnableSynchronizerObjectsEP18MM_EnvironmentBase
DLT stderr Symbol_address=0000000006D40E90
DLT stderr Target=2_90_20220306_238 (Mac OS X 10.14.6)
DLT stderr CPU=amd64 (12 logical CPUs) (0x400000000 RAM)
DLT stderr ----------- Stack Backtrace -----------
DLT stderr ---------------------------------------

JasonFengJ9 · 2022-03-14T23:07:11Z

job/Test_openjdk8_j9_sanity.system_aarch64_linux_testList_0/137/(cent7-aarch64-2)

openjdk version "1.8.0_332"
IBM Semeru Runtime Open Edition (build 1.8.0_332-b04)
Eclipse OpenJ9 VM (build master-3262dbe44, JRE 1.8.0 Linux aarch64-64-Bit Compressed References 20220312_314 (JIT enabled, AOT enabled)
OpenJ9   - 3262dbe44
OMR      - dc0e8cd96
JCL      - 2dca7f33d based on jdk8u332-b04)

[2022-03-12T04:21:25.260Z] variation: -Xgcpolicy:gencon -Xgc:concurrentScavenge
[2022-03-12T04:21:25.260Z] JVM_OPTIONS:  -Xgcpolicy:gencon -Xgc:concurrentScavenge 

[2022-03-12T04:21:29.913Z] DLT 23:21:29.488 - Starting thread. Suite=0 thread=9
[2022-03-12T04:21:30.347Z] DLT stderr Attempt to access ownable synchronizer object located outside of heap (stack allocated?) 0000000001680368
[2022-03-12T04:21:30.347Z] DLT stderr 04:21:30.110 0x100c00    j9mm.141    *   ** ASSERTION FAILED ** at /home/jenkins/workspace/build-scripts/jobs/jdk8u/jdk8u-linux-aarch64-openj9/workspace/build/src/openj9/runtime/gc_base/OwnableSynchronizerObjectBuffer.cpp:100: ((false))
[2022-03-12T04:21:30.347Z] DLT stderr JVMDUMP039I Processing dump event "traceassert", detail "" at 2022/03/11 23:21:30 - please wait.

[2022-03-12T04:21:31.488Z] DaaLoadTest_daa2_CS_5m_0_FAILED

pshipton · 2022-03-21T12:31:35Z

https://openj9-jenkins.osuosl.org/job/Test_openjdk8_j9_sanity.system_ppc64_aix_Nightly_testList_0/237 - p8-java1-ibm07
DaaLoadTest_daa2_CS_5m_0 -Xgcpolicy:gencon -Xgc:concurrentScavenge

https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Test/Test_openjdk8_j9_sanity.system_ppc64_aix_Nightly_testList_0/237/system_test_output.tar.gz

DLT stderr Unhandled exception
DLT stderr Type=Segmentation error vmState=0x00020002
DLT stderr J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000033
DLT stderr Handler1=09001000A0E9A540 Handler2=09001000A0D554B0
DLT stderr R0=0000000620000000 R1=0000010023511CD0 R2=09001000A0D01550 R3=000001001010BD10
DLT stderr R4=0000010010112390 R5=0000000003900328 R6=000000000000000C R7=000000000000003E
DLT stderr R8=000000000000024B R9=000000000000024B R10=0000000000000003 R11=0000000000000000
DLT stderr R12=090000000D334A08 R13=000001002351A800 R14=0000000000000000 R15=0000000000000000
DLT stderr R16=0000000000000000 R17=0000000000000000 R18=0000000000000001 R19=0000000000000001
DLT stderr R20=0000000003900328 R21=00000000000000A0 R22=0000000000000090 R23=0000000000000003
DLT stderr R24=0000010010195AF0 R25=0000000000000001 R26=0000000000000000 R27=0000010010195A10
DLT stderr R28=00000100235165B8 R29=0000010023511ED0 R30=00000100235165B8 R31=26E8EBFB8C621A96
DLT stderr IAR=090000000D3F1D58 LR=090000000D3F1E04 MSR=A00000000200D032 CTR=090000000D43E580
DLT stderr CR=2400024800000000 FPSCR=8202000000000000 XER=0000000082020000
DLT stderr FPR0 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR1 c3e0000000000000 (f: 0.000000, d: -9.223372e+18)
DLT stderr FPR2 41cdcd6500000000 (f: 0.000000, d: 1.000000e+09)
DLT stderr FPR3 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR4 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR5 c3e0000000000000 (f: 0.000000, d: -9.223372e+18)
DLT stderr FPR6 4070b260a349a258 (f: 2739511808.000000, d: 2.671486e+02)
DLT stderr FPR7 408f400000000000 (f: 0.000000, d: 1.000000e+03)
DLT stderr FPR8 40f364f000000000 (f: 0.000000, d: 7.943900e+04)
DLT stderr FPR9 4530000000000000 (f: 0.000000, d: 1.934281e+25)
DLT stderr FPR10 408f400000000000 (f: 0.000000, d: 1.000000e+03)
DLT stderr FPR11 43300000000003e8 (f: 1000.000000, d: 4.503600e+15)
DLT stderr FPR12 4530000000000000 (f: 0.000000, d: 1.934281e+25)
DLT stderr FPR13 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR16 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR17 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR18 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR19 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR20 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR21 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR22 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR23 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR24 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR25 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR26 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR27 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR28 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR29 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR30 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR31 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr Module=/home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64_aix_Nightly_testList_0/openjdkbinary/j2sdk-image/jre/lib/ppc64/default/libj9gc29.so
DLT stderr Module_base_address=090000000D319000
DLT stderr Target=2_90_20220319_245 (AIX 7.1)
DLT stderr CPU=ppc64 (16 logical CPUs) (0x780000000 RAM)
DLT stderr ----------- Stack Backtrace -----------
DLT stderr scanClearable__14MM_RootScannerFP18MM_EnvironmentBase+0x248 (0x090000000D3E0D0C [libj9gc29.so+0xc7d0c])
DLT stderr workerCompleteGC__18MM_MarkingDelegateFP18MM_EnvironmentBase+0x1b4 (0x090000000D3D7958 [libj9gc29.so+0xbe958])
DLT stderr markLiveObjectsComplete__16MM_MarkingSchemeFP18MM_EnvironmentBase+0x10 (0x090000000D3CF734 [libj9gc29.so+0xb6734])
DLT stderr run__19MM_ParallelMarkTaskFP18MM_EnvironmentBase+0x1c0 (0x090000000D4C09A4 [libj9gc29.so+0x1a79a4])
DLT stderr workerEntryPoint__21MM_ParallelDispatcherFP18MM_EnvironmentBase+0x22c (0x090000000D4F6DF0 [libj9gc29.so+0x1dddf0])
DLT stderr dispatcher_thread_proc2__FP14OMRPortLibraryPv+0x27c (0x090000000D4F5D80 [libj9gc29.so+0x1dcd80])
DLT stderr omrsig_protect+0x4a0 (0x090000000BC04D84 [libj9prt29.so+0x59d84])
DLT stderr dispatcher_thread_proc+0x60 (0x090000000D4F6B44 [libj9gc29.so+0x1ddb44])
DLT stderr thread_wrapper+0x544 (0x090000000BC6EAE8 [libj9thr29.so+0x4ae8])
DLT stderr _pthread_body+0xf0 (0x090000000051FE14 [libpthreads.a+0x3e14])

pshipton · 2022-03-21T13:15:03Z

https://openj9-jenkins.osuosl.org/job/Test_openjdk17_j9_special.system_x86-64_linux_Personal_testList_4/14 - ub18-x86-2
DaaLoadTest_daa3_special_5m_12 -Xjit -Xgcpolicy:balanced -Xnocompressedrefs

https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Test/Test_openjdk17_j9_special.system_x86-64_linux_Personal_testList_4/14/system_test_output.tar.gz

DLT stderr Unhandled exception
DLT stderr Type=Segmentation error vmState=0x00000000
DLT stderr J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000002
DLT stderr Handler1=00007FCDD747F200 Handler2=00007FCDD71DC190 InaccessibleAddress=00007FCC5CD97AA8
DLT stderr RDI=0000000029DC7A10 RSI=0000000000000012 RAX=0000000000000001 RBX=00007FCC5DBD8B78
DLT stderr RCX=0000000029DC7A12 RDX=0000000000000001 R8=00007FCDD8175500 R9=00007FCC5B5C5C00
DLT stderr R10=000000000000000D R11=00007FCC5D9EC528 R12=00007FCDD8956100 R13=0000000000000001
DLT stderr R14=0000000000000001 R15=0000000000000001
DLT stderr RIP=00007FCC5CD97AA8 GS=0000 FS=0000 RSP=00007FCBC800AF98
DLT stderr EFlags=0000000000010216 CS=0033 RBP=00007FCDD8856B00 ERR=0000000000000015
DLT stderr TRAPNO=000000000000000E OLDMASK=0000000000000000 CR2=00007FCC5CD97AA8
DLT stderr xmm0 0000000000000004 (f: 4.000000, d: 1.976263e-323)
DLT stderr xmm1 3ff0000000000000 (f: 0.000000, d: 1.000000e+00)
DLT stderr xmm2 0031003500360038 (f: 3539000.000000, d: 9.457014e-308)
DLT stderr xmm3 0030003800350033 (f: 3473459.000000, d: 8.900771e-308)
DLT stderr xmm4 003900370039002d (f: 3735597.000000, d: 1.390718e-307)
DLT stderr xmm5 0030003800350033 (f: 3473459.000000, d: 8.900771e-308)
DLT stderr xmm6 0031003500360038 (f: 3539000.000000, d: 9.457014e-308)
DLT stderr xmm7 0030003600330032 (f: 3342386.000000, d: 8.900754e-308)
DLT stderr xmm8 3ff0000000000000 (f: 0.000000, d: 1.000000e+00)
DLT stderr xmm9 3ff0000000000000 (f: 0.000000, d: 1.000000e+00)
DLT stderr xmm10 3ff8151824c7587e (f: 617044096.000000, d: 1.505150e+00)
DLT stderr xmm11 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr xmm12 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr xmm13 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr xmm14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr xmm15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr Target=2_90_20220320_149 (Linux 4.15.0-96-generic)
DLT stderr CPU=amd64 (4 logical CPUs) (0x5e2815000 RAM)
DLT stderr ----------- Stack Backtrace -----------
DLT stderr protectedBacktrace+0x12 (0x00007FCDD71D8732 [libj9prt29.so+0x26732])
DLT stderr omrsig_protect+0x1e3 (0x00007FCDD71DCEF3 [libj9prt29.so+0x2aef3])
DLT stderr omrintrospect_backtrace_thread_raw+0xbf (0x00007FCDD71D8C2F [libj9prt29.so+0x26c2f])
DLT stderr omrsig_protect+0x1e3 (0x00007FCDD71DCEF3 [libj9prt29.so+0x2aef3])
DLT stderr omrintrospect_backtrace_thread+0x70 (0x00007FCDD71D85F0 [libj9prt29.so+0x265f0])
DLT stderr generateDiagnosticFiles+0x84 (0x00007FCDD747ED44 [libj9vm29.so+0x3fd44])
DLT stderr omrsig_protect+0x1e3 (0x00007FCDD71DCEF3 [libj9prt29.so+0x2aef3])
DLT stderr vmSignalHandler+0x185 (0x00007FCDD747EFB5 [libj9vm29.so+0x3ffb5])
DLT stderr  (0x00007FCDD7575BA5 [libj9vm29.so+0x136ba5])
DLT stderr ---------------------------------------
DLT stderr JVMDUMP039I Processing dump event "gpf", detail "" at 2022/03/20 17:52:02 - please wait.
DLT stderr Unhandled exception
DLT stderr Type=Segmentation error vmState=0x00020002
DLT stderr J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000080
DLT stderr Handler1=00007FCDD747F240 Handler2=00007FCDD71DC190 InaccessibleAddress=0000000000000000
DLT stderr RDI=78245C8B48CB8B48 RSI=00007FCC328CC02E RAX=0000000000000000 RBX=0000000000000000
DLT stderr RCX=848B48FF8BF88B40 RDX=87DC23428E9495BF R8=00007FCBC800AF08 R9=00007FCBC800AF08
DLT stderr R10=00007FCBC800AF08 R11=00007FCBC800B048 R12=00007FCBF861B7D0 R13=0000000004600006
DLT stderr R14=0000000000000000 R15=00007FCC328CC02E
DLT stderr RIP=00007FCDD74BFCBA GS=0000 FS=0000 RSP=00007FCBF861B750
DLT stderr EFlags=0000000000010293 CS=0033 RBP=00007FCBE8003C00 ERR=0000000000000000
DLT stderr TRAPNO=000000000000000D OLDMASK=0000000000000000 CR2=00007F26BB994F80
DLT stderr xmm0 0000000400000004 (f: 4.000000, d: 8.487983e-314)
DLT stderr xmm1 0000000400000004 (f: 4.000000, d: 8.487983e-314)
DLT stderr xmm2 2d3638785f37314b (f: 1597452672.000000, d: 6.817684e-91)
DLT stderr xmm3 616e6f737265505f (f: 1919242368.000000, d: 2.139469e+161)
DLT stderr xmm4 2e726f746361706d (f: 1667330176.000000, d: 5.931070e-85)
DLT stderr xmm5 6972572f6367686c (f: 1667721344.000000, d: 8.774248e+199)
DLT stderr xmm6 746e75722f396a6e (f: 792291968.000000, d: 6.978447e+252)
DLT stderr xmm7 737265505f78756e (f: 1601729920.000000, d: 1.286217e+248)
DLT stderr xmm8 006f006900740061 (f: 7602273.000000, d: 1.379617e-306)
DLT stderr xmm9 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr xmm10 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr xmm11 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr xmm12 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr xmm13 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr xmm14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr xmm15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr Module=/home/jenkins/workspace/Test_openjdk17_j9_special.system_x86-64_linux_Personal_testList_4/openjdkbinary/j2sdk-image/lib/default/libj9vm29.so
DLT stderr Module_base_address=00007FCDD743F000
DLT stderr Target=2_90_20220320_149 (Linux 4.15.0-96-generic)
DLT stderr CPU=amd64 (4 logical CPUs) (0x5e2815000 RAM)
DLT stderr ----------- Stack Backtrace -----------
DLT stderr walkStackFrames+0xb0a (0x00007FCDD74BFCBA [libj9vm29.so+0x80cba])
DLT stderr _ZN28GC_VMThreadStackSlotIterator9scanSlotsEP10J9VMThreadS1_PvPFvP8J9JavaVMPP8J9ObjectS2_P16J9StackWalkStatePKvEbb+0x76 (0x00007FCDD4717E96 [libj9gc_full29.so+0x41e96])
DLT stderr _ZN14MM_RootScanner13scanOneThreadEP18MM_EnvironmentBaseP10J9VMThreadPv+0xfd (0x00007FCDD471037D [libj9gc_full29.so+0x3a37d])
DLT stderr _ZN14MM_RootScanner11scanThreadsEP18MM_EnvironmentBase+0xbf (0x00007FCDD470F0BF [libj9gc_full29.so+0x390bf])
DLT stderr _ZN22MM_GlobalMarkingScheme20markLiveObjectsRootsEP19MM_EnvironmentVLHGC+0x1c2 (0x00007FCDD47D0EE2 [libj9gc_full29.so+0xfaee2])
DLT stderr _ZN25MM_ParallelGlobalMarkTask3runEP18MM_EnvironmentBase+0x288 (0x00007FCDD47D2E78 [libj9gc_full29.so+0xfce78])
DLT stderr _ZN21MM_ParallelDispatcher16workerEntryPointEP18MM_EnvironmentBase+0x1f7 (0x00007FCDD48043C7 [libj9gc_full29.so+0x12e3c7])
DLT stderr _Z23dispatcher_thread_proc2P14OMRPortLibraryPv+0x109 (0x00007FCDD4803BD9 [libj9gc_full29.so+0x12dbd9])
DLT stderr omrsig_protect+0x1e3 (0x00007FCDD71DCEF3 [libj9prt29.so+0x2aef3])
DLT stderr dispatcher_thread_proc+0x3f (0x00007FCDD48036DF [libj9gc_full29.so+0x12d6df])
DLT stderr thread_wrapper+0x162 (0x00007FCDD6FA54F2 [libj9thr29.so+0xe4f2])
DLT stderr start_thread+0xdb (0x00007FCDDCF966DB [libpthread.so.0+0x76db])
DLT stderr clone+0x3f (0x00007FCDDCABB88F [libc.so.6+0x12188f])
DLT stderr ---------------------------------------

pshipton · 2022-03-28T13:23:09Z

https://openj9-jenkins.osuosl.org/job/Test_openjdk17_j9_special.system_x86-64_linux_Personal_testList_4/14 - ub18-x86-2
DaaLoadTest_daa3_special_5m_12 -Xjit -Xgcpolicy:balanced -Xnocompressedrefs

https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Test/Test_openjdk17_j9_special.system_x86-64_linux_Personal_testList_4/14/system_test_output.tar.gz

DLT 17:51:49.599 - Completed 26.7%. Number of tests started=742 (+204)
DLT stderr Unhandled exception
DLT stderr Type=Segmentation error vmState=0x00000000
DLT stderr J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000002
DLT stderr Handler1=00007FCDD747F200 Handler2=00007FCDD71DC190 InaccessibleAddress=00007FCC5CD97AA8
DLT stderr RDI=0000000029DC7A10 RSI=0000000000000012 RAX=0000000000000001 RBX=00007FCC5DBD8B78
DLT stderr RCX=0000000029DC7A12 RDX=0000000000000001 R8=00007FCDD8175500 R9=00007FCC5B5C5C00
DLT stderr R10=000000000000000D R11=00007FCC5D9EC528 R12=00007FCDD8956100 R13=0000000000000001
DLT stderr R14=0000000000000001 R15=0000000000000001
DLT stderr RIP=00007FCC5CD97AA8 GS=0000 FS=0000 RSP=00007FCBC800AF98
DLT stderr EFlags=0000000000010216 CS=0033 RBP=00007FCDD8856B00 ERR=0000000000000015
DLT stderr TRAPNO=000000000000000E OLDMASK=0000000000000000 CR2=00007FCC5CD97AA8
DLT stderr xmm0 0000000000000004 (f: 4.000000, d: 1.976263e-323)
DLT stderr xmm1 3ff0000000000000 (f: 0.000000, d: 1.000000e+00)
DLT stderr xmm2 0031003500360038 (f: 3539000.000000, d: 9.457014e-308)
DLT stderr xmm3 0030003800350033 (f: 3473459.000000, d: 8.900771e-308)
DLT stderr xmm4 003900370039002d (f: 3735597.000000, d: 1.390718e-307)
DLT stderr xmm5 0030003800350033 (f: 3473459.000000, d: 8.900771e-308)
DLT stderr xmm6 0031003500360038 (f: 3539000.000000, d: 9.457014e-308)
DLT stderr xmm7 0030003600330032 (f: 3342386.000000, d: 8.900754e-308)
DLT stderr xmm8 3ff0000000000000 (f: 0.000000, d: 1.000000e+00)
DLT stderr xmm9 3ff0000000000000 (f: 0.000000, d: 1.000000e+00)
DLT stderr xmm10 3ff8151824c7587e (f: 617044096.000000, d: 1.505150e+00)
DLT stderr xmm11 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr xmm12 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr xmm13 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr xmm14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr xmm15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr Target=2_90_20220320_149 (Linux 4.15.0-96-generic)
DLT stderr CPU=amd64 (4 logical CPUs) (0x5e2815000 RAM)
DLT stderr ----------- Stack Backtrace -----------
DLT stderr protectedBacktrace+0x12 (0x00007FCDD71D8732 [libj9prt29.so+0x26732])
DLT stderr omrsig_protect+0x1e3 (0x00007FCDD71DCEF3 [libj9prt29.so+0x2aef3])
DLT stderr omrintrospect_backtrace_thread_raw+0xbf (0x00007FCDD71D8C2F [libj9prt29.so+0x26c2f])
DLT stderr omrsig_protect+0x1e3 (0x00007FCDD71DCEF3 [libj9prt29.so+0x2aef3])
DLT stderr omrintrospect_backtrace_thread+0x70 (0x00007FCDD71D85F0 [libj9prt29.so+0x265f0])
DLT stderr generateDiagnosticFiles+0x84 (0x00007FCDD747ED44 [libj9vm29.so+0x3fd44])
DLT stderr omrsig_protect+0x1e3 (0x00007FCDD71DCEF3 [libj9prt29.so+0x2aef3])
DLT stderr vmSignalHandler+0x185 (0x00007FCDD747EFB5 [libj9vm29.so+0x3ffb5])
DLT stderr  (0x00007FCDD7575BA5 [libj9vm29.so+0x136ba5])
DLT stderr ---------------------------------------
DLT stderr JVMDUMP039I Processing dump event "gpf", detail "" at 2022/03/20 17:52:02 - please wait.
DLT stderr Unhandled exception
DLT stderr Type=Segmentation error vmState=0x00020002
DLT stderr J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000080
DLT stderr Handler1=00007FCDD747F240 Handler2=00007FCDD71DC190 InaccessibleAddress=0000000000000000
DLT stderr RDI=78245C8B48CB8B48 RSI=00007FCC328CC02E RAX=0000000000000000 RBX=0000000000000000
DLT stderr RCX=848B48FF8BF88B40 RDX=87DC23428E9495BF R8=00007FCBC800AF08 R9=00007FCBC800AF08
DLT stderr R10=00007FCBC800AF08 R11=00007FCBC800B048 R12=00007FCBF861B7D0 R13=0000000004600006
DLT stderr R14=0000000000000000 R15=00007FCC328CC02E
DLT stderr RIP=00007FCDD74BFCBA GS=0000 FS=0000 RSP=00007FCBF861B750
DLT stderr EFlags=0000000000010293 CS=0033 RBP=00007FCBE8003C00 ERR=0000000000000000
DLT stderr TRAPNO=000000000000000D OLDMASK=0000000000000000 CR2=00007F26BB994F80
DLT stderr xmm0 0000000400000004 (f: 4.000000, d: 8.487983e-314)
DLT stderr xmm1 0000000400000004 (f: 4.000000, d: 8.487983e-314)
DLT stderr xmm2 2d3638785f37314b (f: 1597452672.000000, d: 6.817684e-91)
DLT stderr xmm3 616e6f737265505f (f: 1919242368.000000, d: 2.139469e+161)
DLT stderr xmm4 2e726f746361706d (f: 1667330176.000000, d: 5.931070e-85)
DLT stderr xmm5 6972572f6367686c (f: 1667721344.000000, d: 8.774248e+199)
DLT stderr xmm6 746e75722f396a6e (f: 792291968.000000, d: 6.978447e+252)
DLT stderr xmm7 737265505f78756e (f: 1601729920.000000, d: 1.286217e+248)
DLT stderr xmm8 006f006900740061 (f: 7602273.000000, d: 1.379617e-306)
DLT stderr xmm9 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr xmm10 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr xmm11 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr xmm12 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr xmm13 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr xmm14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr xmm15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr Module=/home/jenkins/workspace/Test_openjdk17_j9_special.system_x86-64_linux_Personal_testList_4/openjdkbinary/j2sdk-image/lib/default/libj9vm29.so
DLT stderr Module_base_address=00007FCDD743F000
DLT stderr Target=2_90_20220320_149 (Linux 4.15.0-96-generic)
DLT stderr CPU=amd64 (4 logical CPUs) (0x5e2815000 RAM)
DLT stderr ----------- Stack Backtrace -----------
DLT stderr walkStackFrames+0xb0a (0x00007FCDD74BFCBA [libj9vm29.so+0x80cba])
DLT stderr _ZN28GC_VMThreadStackSlotIterator9scanSlotsEP10J9VMThreadS1_PvPFvP8J9JavaVMPP8J9ObjectS2_P16J9StackWalkStatePKvEbb+0x76 (0x00007FCDD4717E96 [libj9gc_full29.so+0x41e96])
DLT stderr _ZN14MM_RootScanner13scanOneThreadEP18MM_EnvironmentBaseP10J9VMThreadPv+0xfd (0x00007FCDD471037D [libj9gc_full29.so+0x3a37d])
DLT stderr _ZN14MM_RootScanner11scanThreadsEP18MM_EnvironmentBase+0xbf (0x00007FCDD470F0BF [libj9gc_full29.so+0x390bf])
DLT stderr _ZN22MM_GlobalMarkingScheme20markLiveObjectsRootsEP19MM_EnvironmentVLHGC+0x1c2 (0x00007FCDD47D0EE2 [libj9gc_full29.so+0xfaee2])
DLT stderr _ZN25MM_ParallelGlobalMarkTask3runEP18MM_EnvironmentBase+0x288 (0x00007FCDD47D2E78 [libj9gc_full29.so+0xfce78])
DLT stderr _ZN21MM_ParallelDispatcher16workerEntryPointEP18MM_EnvironmentBase+0x1f7 (0x00007FCDD48043C7 [libj9gc_full29.so+0x12e3c7])
DLT stderr _Z23dispatcher_thread_proc2P14OMRPortLibraryPv+0x109 (0x00007FCDD4803BD9 [libj9gc_full29.so+0x12dbd9])
DLT stderr omrsig_protect+0x1e3 (0x00007FCDD71DCEF3 [libj9prt29.so+0x2aef3])
DLT stderr dispatcher_thread_proc+0x3f (0x00007FCDD48036DF [libj9gc_full29.so+0x12d6df])
DLT stderr thread_wrapper+0x162 (0x00007FCDD6FA54F2 [libj9thr29.so+0xe4f2])
DLT stderr start_thread+0xdb (0x00007FCDDCF966DB [libpthread.so.0+0x76db])
DLT stderr clone+0x3f (0x00007FCDDCABB88F [libc.so.6+0x12188f])
DLT stderr ---------------------------------------

pshipton · 2022-05-02T14:15:45Z

https://openj9-jenkins.osuosl.org/job/Test_openjdk8_j9_sanity.system_ppc64le_linux_Nightly_testList_0/264 - ub16p8j92
DaaLoadTest_daa2_CS_5m_0 -Xgcpolicy:gencon -Xgc:concurrentScavenge

https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Test/Test_openjdk8_j9_sanity.system_ppc64le_linux_Nightly_testList_0/264/system_test_output.tar.gz

DLT stderr Unhandled exception
DLT stderr Type=Segmentation error vmState=0x00020002
DLT stderr J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
DLT stderr Handler1=00003FFFB3843B30 Handler2=00003FFFB376C940
DLT stderr R0=00003FFFB2023710 R1=00003FFF982ABDB0 R2=00003FFFB2135500 R3=00003FFFAC0714B0
DLT stderr R4=00000000000001FF R5=000000000000001C R6=00003FFFAA010000 R7=00003FFFAC053DA0
DLT stderr R8=000000000000000C R9=00003FFFAC02ACF0 R10=00003FFFAC06FB10 R11=00000000000001FF
DLT stderr R12=00003FFFB1F94A00 R13=00003FFF982B6900 R14=0000000000000000 R15=0000000000000000
DLT stderr R16=00003FFF98230000 R17=0000000000080000 R18=00003FFFB4184390 R19=00003FFF982AF900
DLT stderr R20=00003FFFAC1A4AF0 R21=00003FFFAC1A4F58 R22=00003FFF982AF260 R23=00003FFF982ABEF0
DLT stderr R24=00003FFF982ABE18 R25=0000000000000001 R26=00003FFFAC0E7540 R27=00003FFFAC0E7620
DLT stderr R28=0000000000000090 R29=0000000000000003 R30=00003FFF30001AC8 R31=000000000000001C
DLT stderr NIP=00003FFFB20236D8 MSR=800000010280F033 ORIG_GPR3=00000000000081C8 CTR=00003FFFB1F94A00
DLT stderr LINK=00003FFFB2023710 XER=0000000000000000 CCR=0000000028224244 SOFTE=0000000000000001
DLT stderr TRAP=0000000000000300 DAR=0000000000000028 dsisr=0000000040000000 RESULT=0000000000000000
DLT stderr FPR0 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR1 40542d7da0000000 (f: 2684354560.000000, d: 8.071079e+01)
DLT stderr FPR2 3feec70000000000 (f: 0.000000, d: 9.617920e-01)
DLT stderr FPR3 bf3d4a1000000000 (f: 0.000000, d: -4.469194e-04)
DLT stderr FPR4 3f499a0000000000 (f: 0.000000, d: 7.812977e-04)
DLT stderr FPR5 bf86a69e80000000 (f: 2147483648.000000, d: -1.105999e-02)
DLT stderr FPR6 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR7 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR8 002e006500760069 (f: 7733353.000000, d: 8.344456e-308)
DLT stderr FPR9 0050003200440050 (f: 4456528.000000, d: 3.560288e-307)
DLT stderr FPR10 3fc6180000000000 (f: 0.000000, d: 1.726074e-01)
DLT stderr FPR11 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR12 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR13 bfb7440000000000 (f: 0.000000, d: -9.088135e-02)
DLT stderr FPR14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR16 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR17 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR18 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR19 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR20 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR21 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR22 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR23 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR24 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR25 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR26 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR27 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR28 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR29 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR30 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr FPR31 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr Module=/home/jenkins/workspace/Test_openjdk8_j9_sanity.system_ppc64le_linux_Nightly_testList_0/openjdkbinary/j2sdk-image/jre/lib/ppc64le/default/libj9gc29.so
DLT stderr Module_base_address=00003FFFB1E10000
DLT stderr Target=2_90_20220429_275 (Linux 4.4.0-173-generic)
DLT stderr CPU=ppc64le (16 logical CPUs) (0x1fe090000 RAM)
DLT stderr ----------- Stack Backtrace -----------
DLT stderr _ZN27MM_MarkingSchemeRootClearer30scanOwnableSynchronizerObjectsEP18MM_EnvironmentBase+0x318 (0x00003FFFB20236D8 [libj9gc29.so+0x2136d8])
DLT stderr _ZN27MM_MarkingSchemeRootClearer30scanOwnableSynchronizerObjectsEP18MM_EnvironmentBase+0x350 (0x00003FFFB2023710 [libj9gc29.so+0x213710])
DLT stderr _ZN14MM_RootScanner13scanClearableEP18MM_EnvironmentBase+0x17c (0x00003FFFB1E5E30C [libj9gc29.so+0x4e30c])
DLT stderr _ZN18MM_MarkingDelegate16workerCompleteGCEP18MM_EnvironmentBase+0x198 (0x00003FFFB201E7D8 [libj9gc29.so+0x20e7d8])
DLT stderr _ZN16MM_MarkingScheme23markLiveObjectsCompleteEP18MM_EnvironmentBase+0x1c (0x00003FFFB1F7171C [libj9gc29.so+0x16171c])
DLT stderr _ZN19MM_ParallelMarkTask3runEP18MM_EnvironmentBase+0xe0 (0x00003FFFB20503B0 [libj9gc29.so+0x2403b0])
DLT stderr _ZN21MM_ParallelDispatcher16workerEntryPointEP18MM_EnvironmentBase+0x294 (0x00003FFFB1F93BE4 [libj9gc29.so+0x183be4])
DLT stderr _Z23dispatcher_thread_proc2P14OMRPortLibraryPv+0x168 (0x00003FFFB1F92FB8 [libj9gc29.so+0x182fb8])
DLT stderr omrsig_protect+0x358 (0x00003FFFB376DD48 [libj9prt29.so+0x3dd48])
DLT stderr dispatcher_thread_proc+0x50 (0x00003FFFB1F92730 [libj9gc29.so+0x182730])
DLT stderr thread_wrapper+0x1a8 (0x00003FFFB37013E8 [libj9thr29.so+0x113e8])
DLT stderr start_thread+0xf0 (0x00003FFFB4158040 [libpthread.so.0+0x8040])
DLT stderr clone+0x98 (0x00003FFFB4033BB0 [libc.so.6+0x123bb0])
DLT stderr ---------------------------------------

pshipton · 2022-06-10T13:54:36Z

This was hot for a while in March, but hasn't been seen much since. I don't see any progress mentioned, moving it back to the Backlog.

dmitripivkine · 2022-07-14T18:30:39Z

I think two comments #10489 (comment) and #10489 (comment) have been added to this item by mistake. These comments mentioned failure with Balanced crash is java thread stack walk. Just bad O-slot perhaps.
This issue seems specific to -Xgcpolicy:gencon -Xgc:concurrentScavenge.

dmitripivkine · 2022-07-14T18:38:33Z

Also I am not sure it is JIT or platform specific. Higher reproducibility rate on Power might be explained we have more large machines for this platform. There is higher chance that not all GC threads performing GC (limited to smaller amount due heap size). We think that it might cause Ownable Synchronizer object become obsolete in thread local sublist (if this thread has not been involved in GC). We are investigating this possibility

pshipton · 2022-07-22T11:04:44Z

https://openj9-jenkins.osuosl.org/job/Test_openjdk8_j9_sanity.system_aarch64_linux_Nightly_testList_1/209
DaaLoadTest_daa2_CS_5m_0 -Xgcpolicy:gencon -Xgc:concurrentScavenge

https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Test/Test_openjdk8_j9_sanity.system_aarch64_linux_Nightly_testList_1/209/system_test_output.tar.gz

DLT stderr Attempt to access ownable synchronizer object located outside of heap (stack allocated?) 0000000000610074
DLT stderr 00:36:01.339 0x102d00    j9mm.141    *   ** ASSERTION FAILED ** at /home/jenkins/workspace/Build_JDK8_aarch64_linux_Nightly/openj9/runtime/gc_base/OwnableSynchronizerObjectBuffer.cpp:100: ((false))

dmitripivkine · 2022-08-22T14:59:01Z

Should be fixed by eclipse-omr/omr#6651

JasonFengJ9 added the test failure label Aug 31, 2020

dmitripivkine added the comp:gc label Aug 31, 2020

dmitripivkine added the comp:jit label Aug 31, 2020

dmitripivkine removed the comp:gc label Aug 31, 2020

dmitripivkine mentioned this issue Sep 14, 2020

JDK8 DaaLoadTest_all_ConcurrentScavenge_0_FAILED : Segmentation error vmState=0x00020002 #10587

Closed

This was referenced Dec 7, 2020

DaaLoadTest ConcurrentScavenge crash vmState=0x00020002 #11379

Closed

Improve gccheck for ownablesynchronizer in DDR #11384

Closed

pshipton mentioned this issue Jan 15, 2021

DaaLoadTest ConcurrentScavenge segfault or assert #11616

Closed

dmitripivkine mentioned this issue Feb 2, 2021

DaaLoadTest_daa1_CS_5m_0 crash vmState=0x00020002 #11846

Closed

dmitripivkine added the arch:power label Feb 2, 2021

0xdaryl modified the milestones: Release 0.30 (Java 8, 11, 17) Jan refresh, Release 0.31 (Java 18) Dec 14, 2021

pshipton modified the milestones: Release 0.31 (Java 18), Release 0.32 (Java 8, 11, 17, 18) Apr refresh Jan 11, 2022

dmitripivkine mentioned this issue Feb 9, 2022

DaaLoadTest_all_CS_5m crash vmState=0x00020002 #14468

Closed

pshipton modified the milestones: Release 0.32 (Java 8, 11, 17, 18) Apr refresh, Tracking Feb 10, 2022

pshipton modified the milestones: Backlog, Release 0.33 (Java 8, 11, 17, 18) July refresh Mar 21, 2022

cjjdespres mentioned this issue Apr 6, 2022

Internal JITServer test failures #14538

Open

pshipton modified the milestones: Release 0.33 (Java 8, 11, 17, 18) July refresh, Backlog Jun 10, 2022

dmitripivkine added the comp:gc label Jul 14, 2022

dmitripivkine removed arch:power arch:x86 comp:jit labels Jul 14, 2022

dmitripivkine closed this as completed Aug 22, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

JDK8 DaaLoadTest_daa2_ConcurrentScavenge_0_FAILED : ASSERTION FAILED ** at OwnableSynchronizerObjectBuffer.cpp:100 ((false)) #10489

JDK8 DaaLoadTest_daa2_ConcurrentScavenge_0_FAILED : ASSERTION FAILED ** at OwnableSynchronizerObjectBuffer.cpp:100 ((false)) #10489

JasonFengJ9 commented Aug 31, 2020

dmitripivkine commented Aug 31, 2020

dmitripivkine commented Aug 31, 2020

dmitripivkine commented Aug 31, 2020

dmitripivkine commented Sep 14, 2020

dmitripivkine commented Sep 14, 2020

mnalam-p commented Sep 16, 2020

pshipton commented Sep 16, 2020

mnalam-p commented Sep 18, 2020 •

edited

Loading

JasonFengJ9 commented Sep 21, 2020

mnalam-p commented Sep 24, 2020

mnalam-p commented Sep 24, 2020

gita-omr commented Sep 28, 2020

JasonFengJ9 commented Sep 28, 2020

pshipton commented Sep 28, 2020 •

edited

Loading

pshipton commented Nov 10, 2020 •

edited

Loading

pshipton commented Dec 8, 2020

lumpfish commented Dec 8, 2020

pshipton commented Jan 15, 2021

dmitripivkine commented Feb 2, 2021

pshipton commented Apr 20, 2021

pshipton commented Jan 10, 2022

JasonFengJ9 commented Jan 26, 2022

pshipton commented Mar 7, 2022

JasonFengJ9 commented Mar 14, 2022

pshipton commented Mar 21, 2022

pshipton commented Mar 21, 2022

pshipton commented Mar 28, 2022 •

edited

Loading

pshipton commented May 2, 2022

pshipton commented Jun 10, 2022

dmitripivkine commented Jul 14, 2022

dmitripivkine commented Jul 14, 2022

pshipton commented Jul 22, 2022

dmitripivkine commented Aug 22, 2022

JDK8 DaaLoadTest_daa2_ConcurrentScavenge_0_FAILED : ASSERTION FAILED ** at OwnableSynchronizerObjectBuffer.cpp:100 ((false)) #10489

JDK8 DaaLoadTest_daa2_ConcurrentScavenge_0_FAILED : ASSERTION FAILED ** at OwnableSynchronizerObjectBuffer.cpp:100 ((false)) #10489

Comments

JasonFengJ9 commented Aug 31, 2020

Failure link

Optional info

Failure output (captured from console output)

dmitripivkine commented Aug 31, 2020

dmitripivkine commented Aug 31, 2020

dmitripivkine commented Aug 31, 2020

dmitripivkine commented Sep 14, 2020

dmitripivkine commented Sep 14, 2020

mnalam-p commented Sep 16, 2020

pshipton commented Sep 16, 2020

mnalam-p commented Sep 18, 2020 • edited Loading

JasonFengJ9 commented Sep 21, 2020

mnalam-p commented Sep 24, 2020

mnalam-p commented Sep 24, 2020

gita-omr commented Sep 28, 2020

JasonFengJ9 commented Sep 28, 2020

pshipton commented Sep 28, 2020 • edited Loading

pshipton commented Nov 10, 2020 • edited Loading

pshipton commented Dec 8, 2020

lumpfish commented Dec 8, 2020

pshipton commented Jan 15, 2021

dmitripivkine commented Feb 2, 2021

pshipton commented Apr 20, 2021

pshipton commented Jan 10, 2022

JasonFengJ9 commented Jan 26, 2022

pshipton commented Mar 7, 2022

JasonFengJ9 commented Mar 14, 2022

pshipton commented Mar 21, 2022

pshipton commented Mar 21, 2022

pshipton commented Mar 28, 2022 • edited Loading

pshipton commented May 2, 2022

pshipton commented Jun 10, 2022

dmitripivkine commented Jul 14, 2022

dmitripivkine commented Jul 14, 2022

pshipton commented Jul 22, 2022

dmitripivkine commented Aug 22, 2022

mnalam-p commented Sep 18, 2020 •

edited

Loading

pshipton commented Sep 28, 2020 •

edited

Loading

pshipton commented Nov 10, 2020 •

edited

Loading

pshipton commented Mar 28, 2022 •

edited

Loading