Use JikesRVM-specific weak reference processing and finalisation mechanism

[wks]

Parent issue: mmtk/mmtk-core#694

The task

When we rewrote JikesRVM's MMTk in Rust, we also reimplemented its weak reference and finalisation processors, namely ReferenceProcessor and FinalizableProcessor, in Rust, and included them in mmtk-core (See reference_processor.rs and finalizable_processor.rs). Currently, both mmtk-openjdk and mmtk-jikesrvm use the ref/final processors in mmtk-core.

We later decided that this approach is problematic. The main reason is that it is not general enough for languages other than Java. See mmtk/mmtk-core#694 for details.

We have designed a general language-neutral weak reference processing API, and have included it in mmtk-core (See: mmtk/mmtk-core#700). The existing ref/final processors in mmtk-core are deprecated for removal.

Now, we should implement the reference/finalisation processors in the mmtk-jikesrvm or jikesrvm repository in a JikesRVM-specific manner.

Reuse the Rust implementation or bridge with the Java implementation?

There are two ways to implement ref/final processors for mmtk-jikesrvm.

1. Copy and paste the ref/final processors from the mmtk-core repository to mmtk-jikesrvm and make changes to adapt to the new API.
   • See: https://github.com/wks/mmtk-openjdk/tree/gen-weakref-api
2. Connect the new API to JikesRVM's existing ReferenceProcessor and FinalizableProcessor written in Java.

I once tried method (1) for mmtk-openjdk for the purpose of testing whether the new language-neutral weak reference processing API is general enough for Java. It worked. It showed that the API is general enough to support Java, and it only requires a minimum amount of changes to make the ref/final processors working in the VM binding repo (mmtk-openjdk). It should be just as easy to do the same for JikesRVM.

However, because the ref/final processors are rewritten in Rust, it behaves differently from the existing ref/final processors in JikesRVM written in Java. This is bad because if we compare the vanilla JikesRVM (using its own MMTk written in Java) against JikesRVM with Rust MMTk, the two VMs will be using two different ref/final processor implementations. That's not an apple-to-apple comparison. Any performance advantage or disadvantage shown in the comparison may not be the result of using the Rust MMTk as the GC, but the result of different ref/final processor implementations.

It is therefore more advisable to use the existing Java version of ref/final processor in JikesRVM.

More details

The main part of the new API is the Scanning::process_weak_refs function which the VM binding should implement. It is executed after the first transitive closure is computed (i.e. after all strongly reachable objects are visited), and allows the VM binding to (1) query if any object is live and (2) add any object to the transitive closure. JikesRVM's existing ReferenceProcessor.scan method relies on the TraceLocal argument for those capabilities. We probably need to provide a TraceLocal implementation to provide those capabilities to the Java code.

Java has multiple reference strengths (soft/weak/final/phantom references). My previous work demonstrated a way to build a state machine to handle soft/weak/final/phantom references while expanding the transitive closure multiple times. (See: https://github.com/wks/mmtk-openjdk/blob/gen-weakref-api/mmtk/src/weak_processor/mod.rs).

[wks]

@fepicture implemented VM-side weakref processing by

1. routing the Scanning::process_weak_refs to a state machine implemented in Java in JikesRVM, and
2. routing relevant methods in the TraceLocal class used by the reference processors in JikesRVM, including isLive, getForwarded{Finalizable,Referent,Reference} and retain{Referent,ForFinalize}, to corresponding functions provided by MMTk core, including ObjectReference::is_reachable, ObjectReference::get_forwarded_object and ObjectTracer::trace_object.

As of 25 October 2023, two PRs that implemented these changes have been merged

• Refactor finalize/reference processor #150
• Refactor finalize/reference processor jikesrvm#16

But VM-side weak ref processing is not enabled by default due to remaining bugs that prevent some tests to fail.