Loom: JVMTI support #15183

tajila · 2022-06-01T15:16:10Z

Depends on #15177

JVMTI

can_support_virtual_threads //new capbility
Suspend/ResumeAllVirtualThreads //new APIs
- finds all vthreads and suspends or resumes them
StopThread
- disallowerd on vthreads
GetThreadInfo
- vthread is always daemon
- vthread priotity is always normal
RunAgentThread
- cannot be vthread
GetThreadGroupChildren
- does not include vthreads
GetAllStackTraces
- does not include vthreads
PopFrame/ForceEarlyReturnXXX/SetLocalXXX
- allowed to fail if vthread
- intial approach, do not allow for unmounted vthreads
- if mounted, should just work, although should behave as pinned
  - cant modify pin state here, since effectively threadlocal
GetCurrentThreadCpuTime/GetThreadCpuTime
- disallowed on vthreads
JVMTI_THREAD_STATE_PARKED
- includes sleep state for vthreads

See https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html

babsingh · 2022-06-20T17:57:58Z

Revised List

Potentially, the following JVMTI methods are impacted by Loom:

SetThreadLocalStorage (assigned to @EricYangIBM; covered by the TLS work): https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#SetThreadLocalStorage
GetThreadLocalStorage (assigned to @EricYangIBM; covered by the TLS work): https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#GetThreadLocalStorage
SetEventCallbacks (assigned to @EricYangIBM; covered by the TLS work): https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#SetEventCallbacks
SetEventNotificationMode (assigned to @EricYangIBM; covered by the TLS work): https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#SetEventNotificationMode
For JVMTI event callbacks, each jvmtiHook* function invokes prepareForEvent, which accesses J9JVMTIThreadData->threadEventEnable (assigned to @EricYangIBM; covered by the TLS work): https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#jvmtiEventCallbacks.
GetThreadState: https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#GetThreadState
- Implementation details: See Loom: JVMTI support #15183 (comment).
~~GetCurrentThread: https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#GetCurrentThread~~
- No change needed. See Loom: JVMTI support #15183 (comment).
SuspendThread: https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#SuspendThread
SuspendThreadList: https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#SuspendThreadList
SuspendAllVirtualThreads: https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#SuspendAllVirtualThreads
ResumeThread: https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#ResumeThread
ResumeThreadList: https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#ResumeThreadList
ResumeAllVirtualThreads: https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#ResumeAllVirtualThreads
~~StopThread: https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#StopThread~~
- Completed. See Add error handling for JVMTI functions for Loom #15418.
InterruptThread: https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#InterruptThread
- Phase 1: Block Virtual Threads for JVMTI Inspection #15690.
- Phase 2: Ignore the interrupt code for (NULL == targetThread) i.e. a yielded virtual thread, and avoid pinning a yielded virtual thread in jvmtiHelpers.c::getVMThread to evade the omthread_monitor_* costs.
GetThreadInfo: https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#GetThreadInfo
- Implementation details: See Loom: JVMTI support #15183 (comment).
GetOwnedMonitorInfo: https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#GetOwnedMonitorInfo
- Phase 1: Block Virtual Threads for JVMTI Inspection #15690.
- Phase 2: Loom: JVMTI support #15183 (comment).
GetOwnedMonitorStackDepthInfo: https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#GetOwnedMonitorStackDepthInfo
- Phase 1: Block Virtual Threads for JVMTI Inspection #15690.
- Phase 2: Loom: JVMTI support #15183 (comment).
GetCurrentContendedMonitor: https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#GetCurrentContendedMonitor
- Phase 1: Block Virtual Threads for JVMTI Inspection #15690.
- Phase 2: Loom: JVMTI support #15183 (comment).
~~GetThreadGroupChildren: https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#GetThreadGroupChildren.~~
- Completed. See Update jvmtiGetThreadGroupChildren for java 19 #15539.
GetStackTrace: https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#GetStackTrace
- Phase 1: Block Virtual Threads for JVMTI Inspection #15690.
- Phase 2: Loom: JVMTI support #15183 (comment).
GetThreadListStackTraces: https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#GetThreadListStackTraces
- Phase 1: Block Virtual Threads for JVMTI Inspection #15690.
- Phase 2: Loom: JVMTI support #15183 (comment).
GetFrameCount (assigned to @thallium): https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#GetFrameCount
- Phase 1: Block Virtual Threads for JVMTI Inspection #15690.
- Phase 2: Loom: JVMTI support #15183 (comment).
~~PopFrame: https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#PopFrame~~
- Completed: Return JVMTI_ERROR_OPAQUE_FRAME from ForceEarlyReturn* and PopFrame #15710.
GetFrameLocation: https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#GetFrameLocation
- Phase 1: Block Virtual Threads for JVMTI Inspection #15690.
- Phase 2: Loom: JVMTI support #15183 (comment).
NotifyFramePop: https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#NotifyFramePop
- Phase 1: Block Virtual Threads for JVMTI Inspection #15690.
- Phase 2: Loom: JVMTI support #15183 (comment).
GetLocal(Object|Instance|Int|Long|Float|Double): https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#GetLocalObject
- Phase 1: Block Virtual Threads for JVMTI Inspection #15690.
- Phase 2: Loom: JVMTI support #15183 (comment).
SetLocal(Object|Int|Long|Float|Double): https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#SetLocalObject
- Phase 1: Block Virtual Threads for JVMTI Inspection #15690.
- Phase 2: Loom: JVMTI support #15183 (comment).
~~ForceEarlyReturn(Object|Int|Long|Float|Double|Void): https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#ForceEarlyReturnObject~~
- Completed: Return JVMTI_ERROR_OPAQUE_FRAME from ForceEarlyReturn* and PopFrame #15710.
~~GetCurrentThreadCpuTime: https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#GetCurrentThreadCpuTime~~
- Completed. See Add error handling for JVMTI functions for Loom #15418.
~~GetThreadCpuTime: https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#GetThreadCpuTime~~
- Completed. See Add error handling for JVMTI functions for Loom #15418.
~~RunAgentThread: https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#RunAgentThread~~
- Completed. See Add error handling for JVMTI functions for Loom #15418.
~~GetAllThreads: https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#GetAllThreads~~
- Completed: Ignore VirtualThreads in jvmtiGetAllThreads and Thread.getThreads #15635.
~~can_support_virtual_threads: https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#jvmtiCapabilities.can_support_virtual_threads~~
- Completed. See add JVMTI callback/hook for virtualThread #15254 and Fix padding in jvmtiCapabilities structure #15451.
GetAllStackTraces: https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#GetAllStackTraces
- Phase 1: Block Virtual Threads for JVMTI Inspection #15690.
- Phase 2: Loom: JVMTI support #15183 (comment).

babsingh · 2022-06-20T19:20:36Z

Task 1: Handle error cases

re #15183 (comment)

Examples:
29. GetCurrentThreadCpuTime: https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#GetCurrentThreadCpuTime
30. GetThreadCpuTime: https://download.java.net/java/early_access/jdk19/docs/specs/jvmti.html#GetThreadCpuTime

Return JVMTI_ERROR_UNSUPPORTED_OPERATION for virtual threads

vmconstantpool.xml:
    <classref name="java/lang/VirtualThread"/>

bool isVirtualThread(J9VMThread *vmThread, jthread jthrd) {
   bool result = false;
   if (isSameOrSuperClassOf(
           J9VMJAVALANGVIRTUALTHREAD_OR_NULL(vmThread->javaVM),
           J9OBJECT_CLAZZ(vmThread, *(j9object_t *)jthrd)
   ) {
      result = true;
   }
   return result;
}

babsingh · 2022-06-20T19:57:58Z

@tajila Will J9VMThread->threadObject always be up-to-date for Loom?

jvmtiGetCurrentThreadCpuTime(jvmtiEnv *env, jlong *nanos_ptr) {
    ...
    J9VMThread * currentThread = NULL;
    j9object_t threadObject = NULL;
    jvmtiError rc = JVMTI_ERROR_NONE;

    rc = getCurrentVMThread(vm, &currentThread);
    if (JVMTI_ERROR_NONE != rc) {
        return rc;
    } 
    threadObject = currentThread->threadObject;

    if (isSameOrSuperClassOf(
           J9VMJAVALANGVIRTUALTHREAD_OR_NULL(currentThread->javaVM),
           J9OBJECT_CLAZZ(currentThread, threadObject)
   ) {
       return JVMTI_ERROR_UNSUPPORTED_OPERATION;
   }
   ...
}

tajila · 2022-06-20T20:12:57Z

@tajila Will J9VMThread->threadObject always be up-to-date for Loom?

yes

Return JVMTI_ERROR_UNSUPPORTED_OPERATION from JVMTI functions that do not accept virtual threads as their current thread or parameters. Issue: eclipse-openj9#15183 Signed-off-by: Eric Yang <eric.yang@ibm.com>

EricYangIBM · 2022-06-27T14:29:42Z

Is there a priority for functions to update or should I just go down the list in the spec?

babsingh · 2022-06-27T17:38:40Z

Is there a priority for functions to update or should I just go down the list in the spec?

Some functions will require the changes for #15177. You can support the easiest ones first, and the ones not dependent on #15177.

EricYangIBM · 2022-06-27T17:42:56Z

Looking at the first three thread functions, do they need to be updated?

GetThreadState
Are there any changes needed for getVMThreadStateHelper()? Thread flags used differently for virtual threads?
GetCurrentState
Depends on vm->internalVMFunctions->currentVMThread, I don't think it needs any other changes. https://github.com/eclipse-openj9/openj9/blob/master/runtime/jvmti/jvmtiThread.c#L1096
GetAllThreads
Iterates over J9VMThread.linkNext, does this include virtual threads? https://github.com/eclipse-openj9/openj9/blob/master/runtime/jvmti/jvmtiThread.c#L163

babsingh · 2022-06-27T19:07:15Z

re #15183 (comment):

GetThreadState: There are some differences between Thread.state and VirtualThread.state. Example: A virtual thread that is sleeping, in Thread.sleep, may have JVMTI_THREAD_STATE_PARKED set instead of JVMTI_THREAD_STATE_SLEEPING. JVMTI_THREAD_STATE_RUNNABLE means that a thread is runnable. A virtual thread has two similar states: RUNNABLE and RUNNING. Also, a virtual thread has a PINNED state.
GetCurrentThread: ((J9VMThread *)(vm->internalVMFunctions->currentVMThread()))->threadObject should provide the correct Java programming language thread. No change needed for this method.
GetAllThreads: J9VMThread.linkNext should only include platform and carrier threads. It should not have virtual threads. The JVMTI doc tells us to ignore virtual threads for this method. So, no change is needed for this method.

We can also use the JVMTI tests in the extensions repo to verify our functionality. These tests have been updated for Project Loom.

GetThreadState tests: https://github.com/ibmruntimes/openj9-openjdk-jdk19/tree/openj9/test/hotspot/jtreg/serviceability/jvmti/thread/GetThreadState

Other Project Loom related JVMTI tests: https://github.com/ibmruntimes/openj9-openjdk-jdk19/tree/openj9/test/hotspot/jtreg/serviceability/jvmti/thread

EricYangIBM · 2022-06-30T14:51:25Z

How will we implement Get/SetThreadLocalStorage for virtual threads? The tls for platform threads is attached to omr thread.

babsingh · 2022-06-30T16:17:09Z

How will we implement Get/SetThreadLocalStorage for virtual threads? The tls for platform threads is attached to omr thread.

It can be moved as a hidden field into java.lang.Thread (class). In the Loom world, this will allow us to associate a pointer value with each environment-thread pair.

Old:
   struct J9Thread {
         void* tls[J9THREAD_MAX_TLS_KEYS];

New:
    Add hidden field into java.lang.Thread to store the above TLS array. It can be dynamically initialized.

Old:
    #define THREAD_DATA_FOR_VMTHREAD(j9env, vmThread) \
	((J9JVMTIThreadData *) omrthread_tls_get((vmThread)->osThread, (j9env)->tlsKey))

New:
    Get TLS from java.lang.Thread's hidden field
    (J9JVMTIThreadData *)READU(TLS[j9env->tlsKey - 1]);

We will also need to account for the initialization and destruction of the native structures associated to this hidden field.

@tajila @gacholio Can you provide feedback on the above approach?

gacholio · 2022-06-30T16:52:19Z

At a glance, no, this is not correct. Our TLS is effectively useless in the presence of vthreads, so we will simply have to stop using it in many places.

Correct answer here is probably to add a hidden field to Thread to contain the JVMTI data itself, not some fake TLS implementation.

babsingh · 2022-06-30T16:58:28Z

Correct answer here is probably to add a hidden field to Thread to contain the JVMTI data itself, not some fake TLS implementation.

Will JVMTI data be a struct as follows?

struct JVMTIThreadData {
   List of {J9JVMTIEnv, J9JVMTIThreadData} pairs;
   Other thread specific JVMTI details if needed;
}

gacholio · 2022-06-30T17:35:46Z

To avoid searching a list, we could essentially reimplement TLS - store a fixed-size array in the Thread hidden field, and manage the keys ourselves. This will mean freeing the TLS when the Thread is collected (which may be a pain).

EricYangIBM · 2022-07-05T18:00:57Z

Currently the TLS data struct gets allocated for each existing os thread when a jvmti env is allocated (allocateEnvironment in jvmtiHelpers.c) and new threads when a J9VMThread is created. TLS gets deallocated upon jvmti env deallocation or J9VMThread death. (J9HOOK_VM_THREAD_CREATED and J9HOOK_VM_THREAD_DESTROY)
Could we change these hooks to when a Thread is started/destroyed? Or maybe use the J9HOOK_VM_THREAD_STARTED/J9HOOK_VM_THREAD_END events instead?
Also I don't think the jvmtiEventThreadStart ThreadStart; and jvmtiEventThreadEnd ThreadEnd; callback functions exist/are used (same with the virtual thread versions), could they be used?

Also note the TLS only has to exist while the thread is alive (JVMTI_ERROR_THREAD_NOT_ALIVE)

gacholio · 2022-07-05T18:32:04Z

J9HOOK_VM_THREAD_CREATED is special in that it's fired under lock (it does not report any JVMTI events) and is allowed to fail (e.g. malloc fail for TLS).

The VirtualThread natives are needed to keep a list of every live virtual thread for JVM TI. Since JVM TI agents can be attached at any time, enable these natives by default. Issue: eclipse-openj9/openj9#15183 Signed-off-by: Eric Yang <eric.yang@ibm.com>

Traverse the thread list before acquireExclusiveVMAccess and after releaseExclusiveVMAccess to ensure virtual thread transition completion. Update live thread check for virtual thread Update surrounding code style Issues: eclipse-openj9#15759 eclipse-openj9#15183 Signed-off-by: Gengchen Tuo <gengchen.tuo@ibm.com>

Adds virtual thread support for GetOrSetLocal and NotifyFramePop As walkState->walkThread will be used by some jit functions after stackwalk, we can't use walkContinuationStackFrames which will cause dangling pointer. Instead we need to stack-allocate manually if a J9VMContinuation needs to be walked, so we created a helper getJ9VMContinuationToWalk to get the J9VMContinuation. Also fix coding style of surrounding code Issues: eclipse-openj9#15759 eclipse-openj9#15183 Co-authored-by: Babneet Singh <sbabneet@ca.ibm.com> Signed-off-by: Gengchen Tuo <gengchen.tuo@ibm.com>

targetThread is NULL only for virtual threads, as per the assertion in getVMThread, when mustBeAlive is TRUE. vmThreadForTLS is only used to acquire J9JavaVM in createThreadData and jvmtiTLSGet. If targetThread is NULL, currentThread is passed to createThreadData and jvmtiTLSGet for retrieving J9JavaVM in JDK19+. Related: eclipse-openj9#15541 Related: eclipse-openj9#15183 Signed-off-by: Babneet Singh <sbabneet@ca.ibm.com>

tajila added AQA Test failures from AdoptOpenJDK Quality Assurance test set project:loom Used to track Project Loom related work comp:vm and removed AQA Test failures from AdoptOpenJDK Quality Assurance test set labels Jun 1, 2022

tajila assigned babsingh Jun 2, 2022

tajila added this to the Release 0.34 (Java 19) milestone Jun 7, 2022

fengxue-IS mentioned this issue Jun 7, 2022

add JVMTI callback/hook for virtualThread #15254

Merged

babsingh assigned EricYangIBM Jun 20, 2022

EricYangIBM mentioned this issue Jun 23, 2022

Add error handling for JVMTI functions for Loom #15418

Merged

babsingh mentioned this issue Sep 22, 2022

Update jvmtiGetOrSetLocal and jvmtiNotifyFramePop to support virtual threads #15857

Merged

babsingh mentioned this issue Sep 26, 2022

Fix setEventNotificationMode for unmounted virtual threads #15969

Merged

babsingh mentioned this issue Oct 3, 2022

Avoid double scan stacks in continuation for compact cases #16022

Merged

tajila closed this as completed Oct 18, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Loom: JVMTI support #15183

Loom: JVMTI support #15183

tajila commented Jun 1, 2022 •

edited

Loading

babsingh commented Jun 20, 2022 •

edited

Loading

babsingh commented Jun 20, 2022

babsingh commented Jun 20, 2022

tajila commented Jun 20, 2022

EricYangIBM commented Jun 27, 2022

babsingh commented Jun 27, 2022 •

edited

Loading

EricYangIBM commented Jun 27, 2022

babsingh commented Jun 27, 2022

EricYangIBM commented Jun 30, 2022

babsingh commented Jun 30, 2022 •

edited

Loading

gacholio commented Jun 30, 2022

babsingh commented Jun 30, 2022 •

edited

Loading

gacholio commented Jun 30, 2022

EricYangIBM commented Jul 5, 2022

gacholio commented Jul 5, 2022

Loom: JVMTI support #15183

Loom: JVMTI support #15183

Comments

tajila commented Jun 1, 2022 • edited Loading

babsingh commented Jun 20, 2022 • edited Loading

Revised List

babsingh commented Jun 20, 2022

Task 1: Handle error cases

babsingh commented Jun 20, 2022

tajila commented Jun 20, 2022

EricYangIBM commented Jun 27, 2022

babsingh commented Jun 27, 2022 • edited Loading

EricYangIBM commented Jun 27, 2022

babsingh commented Jun 27, 2022

EricYangIBM commented Jun 30, 2022

babsingh commented Jun 30, 2022 • edited Loading

gacholio commented Jun 30, 2022

babsingh commented Jun 30, 2022 • edited Loading

gacholio commented Jun 30, 2022

EricYangIBM commented Jul 5, 2022

gacholio commented Jul 5, 2022

tajila commented Jun 1, 2022 •

edited

Loading

babsingh commented Jun 20, 2022 •

edited

Loading

babsingh commented Jun 27, 2022 •

edited

Loading

babsingh commented Jun 30, 2022 •

edited

Loading

babsingh commented Jun 30, 2022 •

edited

Loading