JITServer AOT cache testing

[AlexeyKhrabrov]

The JITServer AOT cache is currently not covered by the tests, which can easily result in code changes breaking its implementation. We need to add tests that exercise the AOT cache. One option is to simply run the same tests that already exist for JITServer with the AOT cache enabled, possibly restarting the JITServer at some points to clear the AOT cache.

[llxia]

For JITServer AOT testing, we can just set both in TEST_FLAG (i.e., TEST_GLAG=JITAAS,AOT)
See Grinder: https://openj9-jenkins.osuosl.org/view/Test/job/Grinder/1414/

13:49:23  /home/jenkins/workspace/Grinder/openjdkbinary/j2sdk-image/bin/jitserver &
...
13:49:25  JVM_OPTIONS: -Xshareclasses:name=test_aot -Xscmx400M -Xscmaxaot256m -XX:+UseJITServer 

[llxia]

Also, I noticed that we are exporting TR_Options (see below) in TKG. Can they be replaced by jvm option -Xjit:forceaot?

	export TR_Options=forceAOT
	export TR_OptionsAOT=forceAOT

[AlexeyKhrabrov]

13:49:25  JVM_OPTIONS: -Xshareclasses:name=test_aot -Xscmx400M -Xscmaxaot256m -XX:+UseJITServer 

These options do not enable the JITServer AOT cache, only the local SCC at the client JVMs. Either the JITAAS test flag needs to be updated to add the AOT cache options to both the JITServer and the client JVMs, or we need a new separate flag.

[AlexeyKhrabrov]

Also, how exactly does the JITAAS mode run JITServer instances? If it's a single JITServer instance for the duration of the whole test suite, it might not work as well with the AOT cache because it will shared by all the client JVMs across all test cases. Although it should be good enough as a starting point. IIRC, there are also dedicated JITServer test cases. Those should also be extended to properly exercise the AOT cache.

[llxia]

Either the JITAAS test flag needs to be updated to add the AOT cache options to both the JITServer and the client JVMs, or we need a new separate flag.

Could you please list all the JVM options that we need for JITServer AOT cache testing?

[AlexeyKhrabrov]

Could you please list all the JVM options that we need for JITServer AOT cache testing?

In addition to existing JITServer options, -XX:+JITServerUseAOTCache on both the server and the client.

[AlexeyKhrabrov]

Also, I noticed that we are exporting TR_Options (see below) in TKG. Can they be replaced by jvm option -Xjit:forceaot?

Only if none of the test cases themselves pass any -Xjit: options because only the last one takes effect.

[llxia]

Thanks @AlexeyKhrabrov .
@renfeiw could we add -XX:+JITServerUseAOTCache on both the server and the client when TEST_FLAG contains both JITAAS and AOT?

[AlexeyKhrabrov]

could we add -XX:+JITServerUseAOTCache on both the server and the client when TEST_FLAG contains both JITAAS and AOT?

I'm not sure if we want exactly that. JITServer + AOT (local SCC only) is still a valid testing scenario which would be eliminated by this change.

@mpirvu Do you have an opinion on this?

[mpirvu]

I agree that we don't want every JITServer test that uses AOT to use -XX:+JITServerUseAOTCache because both modes can be found in practice. Of all the JITServer tests, a subset should be executed with AOT, and a subset of that subset should enable -XX:+JITServerUseAOTCache.
I view -XX:+JITServerUseAOTCache like a new mode because what we want is:

• Start JITServer with -XX:+JITServerUseAOTCache
• Start client with -Xshareclasses and -XX:+JITServerUseAOTCache
• Run test once
• Destroy the sharedclasses at the client
• Start client with with -Xshareclasses and -XX:+JITServerUseAOTCache again
• Run the test a second time taking advantage of the cache that was built at the server
• Clean up

[SajinaKandy]

Of all the JITServer tests, a subset should be executed with AOT, and a subset of that subset should enable -XX:+JITServerUseAOTCache.

Currently with AOT testing, only a subset of all the tests would run with aot options. Those marked as "explicit" for AOT would run with default options. In case of TEST_FLAG=AOT,JITAAS, those marked as explicit for aot, would run with only JVM_OPTIONS: -XX:+UseJITServer and not JVM_OPTIONS: -Xshareclasses:name=test_aot -Xscmx400M -Xscmaxaot256m -XX:+UseJITServer. [AOT tests marked for testing with: #5767]

So the new mode or flag (probably jitaot) can be used as a third option of TEST_FLAG which will enable -XX:+JITServerUseAOTCache at the server and find a subset of tests to run with the additional options.

Another thing to note is, currently all tests marked for AOT tests are run twice using the aotIterations=2, but without destroying the cache between iterations. [AOT iteration feature added with: #4096]

[AlexeyKhrabrov]

Thanks for the explanation. Do I understand correctly that tests marked as "explicit for AOT" already pass their own options that enable the SCC and AOT compilation? Do we also assume that such tests already handle the need for separate "cold" and "warm" runs themselves? If so, I think we should still run such tests twice with the same JITServer instance (making sure the SCC is destroyed in between) to exercise the JITServer AOT cache.

I would call the the new mode/flag "jitserveraot" or "jitaasaot". Note that eventually (after #15949 and #16075 are delivered) we might want to add yet another mode that stops the JITServer instance in between the 2 runs to test the AOT cache persistence.

Note that enabling forceAOT is not necessary (but also wouldn't break anything) since the client JVMs using the JITServer AOT cache already generate AOT compilation requests whenever possible.