Provides built-in GraalVM Reachability Metadata and nativeTest on Ela…

…sticjob Bootstrap

.github/workflows/graalvm.yml

@@ -0,0 +1,47 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+name: NativeTest CI - GraalVM Native Image
+
+on:
+  pull_request:
+    branches: [ master ]
+    paths:
+      - '.github/workflows/graalvm.yml'
+      - 'reachability-metadata/src/**'
+      - 'test/native/native-image-filter/**'
+      - 'test/native/src/**'
+
+jobs:
+  build:
+    strategy:
+      matrix:
+        java: [ '22.0.2' ]
+        os: [ 'ubuntu-latest' ]
+    runs-on: ${{ matrix.os }}
+    steps:
+      - uses: actions/checkout@v3
+      - name: Set up GraalVM CE ${{ matrix.java }}
+        uses: graalvm/setup-graalvm@v1
+        with:
+          java-version: ${{ matrix.java }}
+          distribution: 'graalvm-community'
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+          cache: 'maven'
+      - name: Run nativeTest with GraalVM CE for ${{ matrix.java-version }}
+        continue-on-error: true
+        run: ./mvnw -PnativeTestInElasticJob -T1C -B -e clean test

bootstrap/pom.xml

@@ -62,6 +62,11 @@
             <artifactId>elasticjob-tracing-rdb</artifactId>
             <version>${project.parent.version}</version>
         </dependency>
+        <dependency>
+            <groupId>org.apache.shardingsphere.elasticjob</groupId>
+            <artifactId>elasticjob-reachability-metadata</artifactId>
+            <version>${project.version}</version>
+        </dependency>
 
         <dependency>
             <groupId>org.apache.shardingsphere.elasticjob</groupId>

docs/content/user-manual/configuration/graalvm-native-image.cn.md

@@ -0,0 +1,164 @@
++++
+title = "GraalVM Native Image"
+weight = 6
+chapter = true
++++
+
+## 背景信息
+
+ElasticJob 已在 GraalVM Native Image 下完成可用性验证。
+
+构建包含 `org.apache.shardingsphere.elasticjob:elasticjob-bootstrap:${elasticjob.version}` 的 Maven 依赖的 GraalVM Native Image，
+你需要借助于 GraalVM Native Build Tools。
+GraalVM Native Build Tools 提供了 Maven Plugin 和 Gradle Plugin 来简化 GraalVM CE 的 `native-image` 命令行工具的长篇大论的 shell 命令。
+
+ElasticJob 要求在如下或更高版本的 `GraalVM CE` 完成构建 GraalVM Native Image。使用者可通过 `SDKMAN!` 快速切换 JDK。这同理
+适用于 https://sdkman.io/jdks#graal ， https://sdkman.io/jdks#nik 和 https://sdkman.io/jdks#mandrel 等 `GraalVM CE` 的下游发行版。
+
+- GraalVM CE For JDK 22.0.2，对应于 SDKMAN! 的 `22.0.2-graalce`
+
+用户依然可以使用 SDKMAN! 上的 `21.0.2-graalce` 等旧版本的 GraalVM CE 来构建 ShardingSphere 的 GraalVM Native Image 产物。
+ElasticJob 不为已停止维护的 GraalVM CE 版本设置 CI。
+
+## 使用 ElasticJob 的 Java API
+
+### Maven 生态
+
+使用者需要主动使用 GraalVM Reachability Metadata 中央仓库。
+如下配置可供参考，以配置项目额外的 Maven Profiles，以 GraalVM Native Build Tools 的文档为准。
+
+```xml
+<project>
+    <dependencies>
+        <dependency>
+            <groupId>org.apache.shardingsphere.elasticjob</groupId>
+            <artifactId>elasticjob-bootstrap</artifactId>
+            <version>${elasticjob.version}</version>
+            <scope>test</scope>
+        </dependency>
+    </dependencies>
+
+    <build>
+        <plugins>
+            <plugin>
+                <groupId>org.graalvm.buildtools</groupId>
+                <artifactId>native-maven-plugin</artifactId>
+                <version>0.10.2</version>
+                <extensions>true</extensions>
+                <executions>
+                    <execution>
+                        <id>build-native</id>
+                        <goals>
+                            <goal>compile-no-fork</goal>
+                        </goals>
+                        <phase>package</phase>
+                    </execution>
+                    <execution>
+                        <id>test-native</id>
+                        <goals>
+                            <goal>test</goal>
+                        </goals>
+                        <phase>test</phase>
+                    </execution>
+                </executions>
+            </plugin>
+        </plugins>
+    </build>
+</project>
+```
+
+### Gradle 生态
+
+使用者需要主动使用 GraalVM Reachability Metadata 中央仓库。
+如下配置可供参考，以配置项目额外的 Gradle Tasks，以 GraalVM Native Build Tools 的文档为准。
+由于 https://github.com/gradle/gradle/issues/17559 的限制，用户需要通过 Maven 依赖的形式引入 Metadata Repository 的 JSON 文件。
+参考 https://github.com/graalvm/native-build-tools/issues/572 。
+
+```groovy
+plugins {
+   id 'org.graalvm.buildtools.native' version '0.10.2'
+}
+
+dependencies {
+   implementation 'org.apache.shardingsphere.elasticjob:elasticjob-bootstrap:${elasticjob.version}'
+   implementation(group: 'org.graalvm.buildtools', name: 'graalvm-reachability-metadata', version: '0.10.2', classifier: 'repository', ext: 'zip')
+}
+
+graalvmNative {
+   metadataRepository {
+        enabled.set(false)
+   }
+}
+```
+
+## 对于 sbt 等不被 GraalVM Native Build Tools 支持的构建工具
+
+此类需求需要在 https://github.com/graalvm/native-build-tools 打开额外的 issue 并提供对应构建工具的 Plugin 实现。
+
+## 使用限制
+
+1. 使用者依然需要在 `src/main/resources/META-INF/native-image` 文件夹或 `src/test/resources/META-INF/native-image` 文件夹配置独立文件的 GraalVM Reachability Metadata。
+使用者可通过 GraalVM Native Build Tools 的 GraalVM Tracing Agent 来快速采集 GraalVM Reachability Metadata。
+
+2. ElasticJob 的 Spring 命名空间集成模块 `org.apache.shardingsphere.elasticjob:elasticjob-spring-namespace` 尚未在 GraalVM Native Image 下可用。
+
+3. ElasticJob 的 Spring Boot Starter 集成模块 `org.apache.shardingsphere.elasticjob:elasticjob-spring-boot-starter` 尚未在 GraalVM Native Image 下可用。
+
+## 贡献 GraalVM Reachability Metadata
+
+ElasticJob 对在 GraalVM Native Image 下的可用性的验证，是通过 GraalVM Native Build Tools 的 Maven Plugin 子项目来完成的。
+通过在 JVM 下运行单元测试，为单元测试打上 `junit-platform-unique-ids*` 标签，此后构建为 GraalVM Native Image 进行 nativeTest 来测试
+在 GraalVM Native Image 下的单元测试覆盖率。请贡献者不要使用 `io.kotest:kotest-runner-junit5-jvm:5.5.4` 等在 `test listener` mode 下
+failed to discover tests 的测试库。
+
+ElasticJob 定义了 `elasticjob-test-native` 的 Maven Module 用于为 native Test 提供小型的单元测试子集，
+此单元测试子集避免了使用 Mockito 等 native Test 下无法使用的第三方库。
+
+ElasticJob 定义了 `nativeTestInElasticJob` 的 Maven Profile 用于为 `elasticjob-test-native` 模块执行 nativeTest 。
+
+假设贡献者处于新的 Ubuntu 22.04.4 LTS 实例下，其可通过如下 bash 命令通过 SDKMAN! 管理 JDK 和工具链，
+并为 `elasticjob-test-native` 子模块执行 nativeTest。
+
+贡献者必须安装 Docker Engine 以执行 `testcontainers-java` 相关的单元测试。
+
+```bash
+sudo apt install unzip zip curl sed -y
+curl -s "https://get.sdkman.io" | bash
+source "$HOME/.sdkman/bin/sdkman-init.sh"
+sdk install java 22.0.2-graalce
+sdk use java 22.0.2-graalce
+sudo apt-get install build-essential zlib1g-dev -y
+
+git clone git@github.com:apache/shardingsphere-elasticjob.git
+cd ./shardingsphere-elasticjob/
+./mvnw -PnativeTestInElasticJob -T1C -e clean test
+```
+
+当贡献者发现缺少与 ElasticJob 无关的第三方库的 GraalVM Reachability Metadata 时，应当在
+https://github.com/oracle/graalvm-reachability-metadata 打开新的 issue， 并提交包含依赖的第三方库缺失的 GraalVM Reachability
+Metadata 的 PR。ElasticJob 在 `elasticjob-reachability-metadata` 子模块主动托管了部分第三方库的 GraalVM Reachability Metadata。
+
+如果 nativeTest 执行失败， 应为单元测试生成初步的 GraalVM Reachability Metadata，
+并手动调整 `elasticjob-reachability-metadata` 子模块的 classpath 的 `META-INF/native-image/org.apache.shardingsphere.elasticjob/elasticjob-reachability-metadata/` 文件夹下的内容以修复 nativeTest。
+如有需要，请使用 `org.junit.jupiter.api.condition.DisabledInNativeImage` 注解或 `org.graalvm.nativeimage.imagecode` 的
+System Property 屏蔽部分单元测试在 GraalVM Native Image 下运行。
+
+ElasticJob 定义了 `generateMetadata` 的 Maven Profile 用于在 GraalVM JIT Compiler 下携带 GraalVM Tracing Agent 执行单元测试，
+并在 `elasticjob-reachability-metadata` 子模块的 classpath 的 `META-INF/native-image/org.apache.shardingsphere.elasticjob/generated-reachability-metadata/` 文件夹下，
+生成或覆盖已有的 GraalVM Reachability Metadata 文件。可通过如下 bash 命令简单处理此流程。
+贡献者仍可能需要手动调整具体的 JSON 条目，并适时调整 Maven Profile 和 GraalVM Tracing Agent 的 Filter 链。
+针对 `elasticjob-reachability-metadata` 子模块，
+手动增删改动的 JSON 条目应位于 `META-INF/native-image/org.apache.shardingsphere.elasticjob/elasticjob-reachability-metadata/` 文件夹下，
+而 `META-INF/native-image/org.apache.shardingsphere.elasticjob/generated-reachability-metadata/` 中的条目仅应由 `generateMetadata` 的 Maven Profile 生成。
+
+以下命令仅为 `elasticjob-test-native` 生成 Conditional 形态的 GraalVM Reachability Metadata 的一个举例。
+生成的 GraalVM Reachability Metadata 位于 `elasticjob-reachability-metadata` 子模块下。
+
+对于测试类和测试文件独立使用的 GraalVM Reachability Metadata，贡献者应该放置到 `shardingsphere-test-native` 子模块的 classpath 的
+`META-INF/native-image/elasticjob-test-native-test-metadata/` 下。
+
+```bash
+git clone git@github.com:apache/shardingsphere.git
+cd ./shardingsphere/
+./mvnw -PgenerateMetadata -DskipNativeTests -e -T1C clean test native:metadata-copy
+```

docs/content/user-manual/configuration/graalvm-native-image.en.md

@@ -0,0 +1,168 @@
++++
+title = "GraalVM Native Image"
+weight = 6
+chapter = true
++++
+
+## Background information
+
+ElasticJob has been verified for availability under GraalVM Native Image.
+
+To build a GraalVM Native Image with the Maven dependency `org.apache.shardingsphere.elasticjob:elasticjob-bootstrap:${elasticjob.version}`,
+you need to use GraalVM Native Build Tools.
+
+GraalVM Native Build Tools provides Maven Plugin and Gradle Plugin to simplify the long-winded shell commands of GraalVM CE's `native-image` command line tool.
+
+ElasticJob requires the following or higher versions of `GraalVM CE` to build the GraalVM Native Image. Users can quickly switch JDKs through `SDKMAN!`. 
+This also applies to downstream distributions of `GraalVM CE` such as https://sdkman.io/jdks#graal, https://sdkman.io/jdks#nik and https://sdkman.io/jdks#mandrel.
+
+- GraalVM CE For JDK 22.0.2, corresponding to `22.0.2-graalce` of SDKMAN!
+
+Users can still use old versions of GraalVM CE such as `21.0.2-graalce` on SDKMAN! to build ShardingSphere's GraalVM Native Image product.
+ElasticJob does not set CI for GraalVM CE versions that have stopped maintenance.
+
+## Using ElasticJob's Java API
+
+### Maven Ecosystem
+
+Users need to actively use the GraalVM Reachability Metadata Central Repository.
+The following configuration is for reference. To configure additional Maven Profiles for the project, 
+refer to the documentation of GraalVM Native Build Tools.
+
+```xml
+<project>
+    <dependencies>
+        <dependency>
+            <groupId>org.apache.shardingsphere.elasticjob</groupId>
+            <artifactId>elasticjob-bootstrap</artifactId>
+            <version>${elasticjob.version}</version>
+            <scope>test</scope>
+        </dependency>
+    </dependencies>
+
+    <build>
+        <plugins>
+            <plugin>
+                <groupId>org.graalvm.buildtools</groupId>
+                <artifactId>native-maven-plugin</artifactId>
+                <version>0.10.2</version>
+                <extensions>true</extensions>
+                <executions>
+                    <execution>
+                        <id>build-native</id>
+                        <goals>
+                            <goal>compile-no-fork</goal>
+                        </goals>
+                        <phase>package</phase>
+                    </execution>
+                    <execution>
+                        <id>test-native</id>
+                        <goals>
+                            <goal>test</goal>
+                        </goals>
+                        <phase>test</phase>
+                    </execution>
+                </executions>
+            </plugin>
+        </plugins>
+    </build>
+</project>
+```
+
+### Gradle Ecosystem
+
+Users need to actively use the GraalVM Reachability Metadata Central Repository.
+The following configuration is for reference. To configure additional Gradle Tasks for the project, refer to the documentation of GraalVM Native Build Tools.
+Due to the limitations of https://github.com/gradle/gradle/issues/17559, users need to introduce the Metadata Repository JSON file in the form of Maven dependencies.
+Refer to https://github.com/graalvm/native-build-tools/issues/572.
+
+```groovy
+plugins {
+   id 'org.graalvm.buildtools.native' version '0.10.2'
+}
+
+dependencies {
+   implementation 'org.apache.shardingsphere.elasticjob:elasticjob-bootstrap:${elasticjob.version}'
+   implementation(group: 'org.graalvm.buildtools', name: 'graalvm-reachability-metadata', version: '0.10.2', classifier: 'repository', ext: 'zip')
+}
+
+graalvmNative {
+   metadataRepository {
+        enabled.set(false)
+   }
+}
+```
+
+## For build tools such as sbt that are not supported by GraalVM Native Build Tools
+
+Such requirements require opening additional issues at https://github.com/graalvm/native-build-tools and providing plugin implementations for the corresponding build tools.
+
+## Usage restrictions
+
+1. Users still need to configure GraalVM Reachability Metadata in separate files in the `src/main/resources/META-INF/native-image` folder or the `src/test/resources/META-INF/native-image` folder.
+Users can quickly collect GraalVM Reachability Metadata through the GraalVM Tracing Agent of GraalVM Native Build Tools.
+
+2. The Spring namespace integration module `org.apache.shardingsphere.elasticjob:elasticjob-spring-namespace` of ElasticJob is not yet available under GraalVM Native Image.
+
+3. The Spring Boot Starter integration module `org.apache.shardingsphere.elasticjob:elasticjob-spring-boot-starter` for ElasticJob is not yet available under GraalVM Native Image.
+
+## Contribute GraalVM Reachability Metadata
+
+ElasticJob's usability verification under GraalVM Native Image is done by the Maven Plugin subproject of GraalVM Native Build Tools.
+
+Unit test coverage under GraalVM Native Image is tested by running unit tests under JVM, 
+tagging unit tests with `junit-platform-unique-ids*`, and then building GraalVM Native Image for nativeTest. 
+Contributors are requested not to use test libraries such as `io.kotest:kotest-runner-junit5-jvm:5.5.4` that failed to discover tests in `test listener` mode.
+
+ElasticJob defines the `elasticjob-test-native` Maven Module to provide a small subset of unit tests for native Test,
+which avoids the use of third-party libraries such as Mockito that cannot be used under native Test.
+
+ElasticJob defines the `nativeTestInElasticJob` Maven profile to execute nativeTest for the `elasticjob-test-native` module.
+
+Assuming the contributor is on a fresh Ubuntu 22.04.4 LTS instance, he can use SDKMAN! to manage JDK and toolchains with the following bash command,
+and execute nativeTest for the `elasticjob-test-native` submodule.
+
+Contributors must install Docker Engine to execute the `testcontainers-java` related unit tests.
+
+```bash
+sudo apt install unzip zip curl sed -y
+curl -s "https://get.sdkman.io" | bash
+source "$HOME/.sdkman/bin/sdkman-init.sh"
+sdk install java 22.0.2-graalce
+sdk use java 22.0.2-graalce
+sudo apt-get install build-essential zlib1g-dev -y
+
+git clone git@github.com:apache/shardingsphere-elasticjob.git
+cd ./shardingsphere-elasticjob/
+./mvnw -PnativeTestInElasticJob -T1C -e clean test
+```
+
+When contributors find that GraalVM Reachability Metadata for third-party libraries not related to ElasticJob is missing, 
+they should open a new issue at https://github.com/oracle/graalvm-reachability-metadata, 
+and submit a PR with missing GraalVM Reachability Metadata for dependent third-party libraries. 
+ElasticJob proactively hosts GraalVM Reachability Metadata for some third-party libraries in the `elasticjob-reachability-metadata` submodule.
+
+If nativeTest fails, generate preliminary GraalVM Reachability Metadata for unit tests, 
+and manually adjust the contents of the `META-INF/native-image/org.apache.shardingsphere.elasticjob/elasticjob-reachability-metadata/` folder in the classpath of the `elasticjob-reachability-metadata` submodule to fix nativeTest.
+If necessary, 
+use the `org.junit.jupiter.api.condition.DisabledInNativeImage` annotation or the `org.graalvm.nativeimage.imagecode` System Property to shield some unit tests from running under the GraalVM Native Image.
+
+ElasticJob defines the `generateMetadata` Maven Profile to execute unit tests with the GraalVM Tracing Agent under the GraalVM JIT Compiler,
+and generates or overwrites the existing GraalVM Reachability Metadata file in the `META-INF/native-image/org.apache.shardingsphere.elasticjob/generated-reachability-metadata/` folder in the classpath of the `elasticjob-reachability-metadata` submodule. 
+This process can be easily handled by the following bash command. 
+Contributors may still need to manually adjust specific JSON entries and adjust the filter chain of the Maven Profile and GraalVM Tracing Agent as appropriate. 
+For the `elasticjob-reachability-metadata` submodule,
+manually added, deleted, and modified JSON entries should be located in the `META-INF/native-image/org.apache.shardingsphere.elasticjob/elasticjob-reachability-metadata/` folder,
+while the entries in `META-INF/native-image/org.apache.shardingsphere.elasticjob/generated-reachability-metadata/` should only be generated by the `generateMetadata` Maven Profile.
+
+The following command is just an example of generating Conditional GraalVM Reachability Metadata for `elasticjob-test-native`.
+The generated GraalVM Reachability Metadata is located in the `elasticjob-reachability-metadata` submodule.
+
+For GraalVM Reachability Metadata used independently by test classes and test files, 
+contributors should place it in the classpath of the shardingsphere-test-native submodule under `META-INF/native-image/elasticjob-test-native-test-metadata/`.
+
+```bash
+git clone git@github.com:apache/shardingsphere.git
+cd ./shardingsphere/
+./mvnw -PgenerateMetadata -DskipNativeTests -e -T1C clean test native:metadata-copy
+```