Skip to content

Commit

Permalink
Merge branch 'master' into e2e-php
Browse files Browse the repository at this point in the history
  • Loading branch information
heyanlong authored Sep 27, 2023
2 parents 74ebe24 + a182220 commit 9eb87c1
Show file tree
Hide file tree
Showing 49 changed files with 3,779 additions and 1,076 deletions.
40 changes: 40 additions & 0 deletions .github/workflows/native-image.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

name: GraalVM Community Edition build
on:
pull_request:
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
submodules: true
- uses: graalvm/setup-graalvm@v1
with:
java-version: '17.0.8'
distribution: 'graalvm-community'
components: 'native-image'
github-token: ${{ secrets.GITHUB_TOKEN }}
- name: Native Compile
run: |
./mvnw -Pbackend,native clean package -Dmaven.test.skip
- uses: actions/upload-artifact@v3
name: Upload distribution tar
with:
name: native-pre-dist
path: distribution/graal/dist
2 changes: 0 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,6 @@ microservices, cloud native and container-based architectures.
[![Twitter Follow](https://img.shields.io/twitter/follow/asfskywalking.svg?style=for-the-badge&label=Follow&logo=twitter)](https://twitter.com/AsfSkyWalking)

[![Maven Central](https://img.shields.io/maven-central/v/org.apache.skywalking/apache-skywalking-apm.svg)](http://skywalking.apache.org/downloads/)
[![CI/IT Tests](https://github.com/apache/skywalking/workflows/CI%20AND%20IT/badge.svg?branch=master)](https://github.com/apache/skywalking/actions?query=workflow%3ACI%2BAND%2BIT+event%3Aschedule+branch%3Amaster)
[![E2E Tests](https://github.com/apache/skywalking/workflows/E2E/badge.svg?branch=master)](https://github.com/apache/skywalking/actions?query=branch%3Amaster+event%3Aschedule+workflow%3AE2E)

# Abstract
**SkyWalking** is an open source APM system, including monitoring, tracing, diagnosing capabilities for distributed system
Expand Down
9 changes: 8 additions & 1 deletion docs/en/changes/changes.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

#### Project

* Add distribution/graal module to make preliminary preparations for supporting native-image.

#### OAP Server

Expand All @@ -11,10 +12,16 @@
* [Breaking Change] Elasticsearch storage merge all management data indices into one index `management`,
including `ui_template,ui_menu,continuous_profiling_policy`.
* Add a release mechanism for alarm windows when it is expired in case of OOM.
* Fix Zipkin trace receiver response: make the HTTP status code from `200` to `202`.

#### UI


* Add new menu for kafka monitoring.
* Fix independent widget duration.
* Fix the display height of the link tree structure.
* Replace the name by shortName on service widget.
* Refactor: update pagination style. No visualization style change.
* Apply MQE on K8s layer UI-templates

#### Documentation

Expand Down
4 changes: 3 additions & 1 deletion docs/en/concepts-and-designs/manual-sdk.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
# Manual instrument SDK
Our incredible community has contributed to the manual instrument SDK.
- [Rust](https://github.com/apache/skywalking-rust). Rust SDK follows the SkyWalking format.
- [Go2Sky](https://github.com/SkyAPM/go2sky). Go SDK follows the SkyWalking format.
- [C++](https://github.com/SkyAPM/cpp2sky). C++ SDK follows the SkyWalking format.

Below is the archived list.
- [Go2Sky](https://github.com/SkyAPM/go2sky). Since Jun 14, 2023.

## What are the SkyWalking format and the propagation protocols?
- [Tracing APIs](../api/trace-data-protocol-v3.md)
- [Meter APIs](../api/meter.md)
Expand Down
49 changes: 49 additions & 0 deletions docs/en/setup/backend/backend-graalvm.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# GraalVM native-image for SkyWalking (Experimental)

## Native-Image
Native Image is a technology to ahead-of-time compile Java code to a standalone executable, called a native image.
This executable includes the application classes, classes from its dependencies, runtime library classes,
and statically linked native code from JDK. It does not run on the Java VM, but includes necessary components like memory management, thread scheduling, and so on from a different runtime system, called “Substrate VM”.
Substrate VM is the name for the runtime components (like the deoptimizer, garbage collector, thread scheduling etc.).
The resulting program has faster startup time and lower runtime memory overhead compared to a JVM.

SkyWalking currently offers support for OAP servers running as native-image. However, please note that the OAP started in this manner does not have the same functionality as the regular OAP, and some features are not yet supported.

## Compile Guide
Notice: If you are not familiar with the compilation process, please read [How-to-build](https://skywalking.apache.org/docs/main/next/en/guides/how-to-build/) first.

The native-image compilation is not enabled by default. To enable it, we need to activate `native` profile during compilation, such as:

```shell

./mvnw -backend,naative clean package -Dmaven.test.skip

```

Then, 2 packages are in `distribution/graal/dist`,
As this is an experimental feature, a package named `apache-skywalking-apm-native-pre-bin.tar.gz` is a tarball for GraalVMization friendly, including original classes to be compiled by GraalVM.
The package named `apache-skywalking-apm-native-bin.tar.gz` includes the final compiled native binary, relative configuration files, and booting shells. Read `Package Structure` doc for more details.

## Package Structure

SkyWalking’s native-image distribution package consists of the following parts:

* bin/cmd scripts: Located in the /bin folder. Includes startup Linux shell and Windows cmd scripts for the backend server.

* Backend config: Located in the /config folder. Includes settings files of the backend. Running native-image does not require additional configuration, so you can refer to [backend-setup](https://skywalking.apache.org/docs/main/next/en/setup/backend/backend-setup/) to learn how to configure it.

* Native image executable: Located in /image folder. It can be run directly, but it is not recommended to do so, as the absence of some environment variables can lead to startup failure.

## How To Use
By executing following:

```shell
./bin/oapService-native.sh
```
we can successfully start SkyWalking-oap.

## Differences and TODO
With native-image, some features are not yet supported.

1. [LAL](https://skywalking.apache.org/docs/main/next/en/concepts-and-designs/lal/), [MAL](https://skywalking.apache.org/docs/main/next/en/concepts-and-designs/mal/), and some other features related to them are not supported at the moment.
2. The [OAL](https://skywalking.apache.org/docs/main/next/en/concepts-and-designs/oal/) files are used in the compiling stage, which means that users would not see these files inside the native image package, and can't change it. Consider recompiling and packaging from the source codes including your OAL file changes.
2 changes: 2 additions & 0 deletions docs/menu.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,8 @@ catalog:
path: "/en/setup/backend/backend-docker"
- name: "Run with Kubernetes"
path: "/en/setup/backend/backend-k8s"
- name: "Run with GraalVM (Experimental)"
path: "/en/setup/backend/backend-graalvm"
- name: "Configuration Vocabulary"
path: "/en/setup/backend/configuration-vocabulary"
- name: "Advanced Setup"
Expand Down
107 changes: 107 additions & 0 deletions graal/apache-skywalking-native-apm/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ Licensed to the Apache Software Foundation (ASF) under one or more
~ contributor license agreements. See the NOTICE file distributed with
~ this work for additional information regarding copyright ownership.
~ The ASF licenses this file to You under the Apache License, Version 2.0
~ (the "License"); you may not use this file except in compliance with
~ the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
~
-->

<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.skywalking</groupId>
<artifactId>graal</artifactId>
<version>9.7.0-SNAPSHOT</version>
</parent>

<artifactId>apache-skywalking-native-apm</artifactId>

<dependencies>
<dependency>
<groupId>org.apache.skywalking</groupId>
<artifactId>graal-server-starter</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>

<profiles>
<profile>
<id>native</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>copy-dependencies</id>
<phase>package</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<includeScope>runtime</includeScope>
<outputDirectory>${project.build.directory}/oap-libs</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<executions>
<execution>
<id>dist</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
<configuration>
<finalName>apache-skywalking-apm-native-pre-bin</finalName>
<descriptors>
<descriptor>${project.basedir}/src/main/assembly/binary.xml</descriptor>
</descriptors>
</configuration>
</execution>
</executions>
<configuration>
<attach>true</attach>
<tarLongFileMode>posix</tarLongFileMode>
<runOnlyAtExecutionRoot>false</runOnlyAtExecutionRoot>
<appendAssemblyId>false</appendAssemblyId>
</configuration>
</plugin>
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<id>dist</id>
<phase>package</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<target>
<copy file="${project.build.directory}/apache-skywalking-apm-native-pre-bin.tar.gz" tofile="${project.basedir}/../dist/apache-skywalking-apm-native-pre-bin.tar.gz" overwrite="true" />
</target>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>
129 changes: 129 additions & 0 deletions graal/apache-skywalking-native-apm/src/main/assembly/binary.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
<!--
~ Licensed to the Apache Software Foundation (ASF) under one or more
~ contributor license agreements. See the NOTICE file distributed with
~ this work for additional information regarding copyright ownership.
~ The ASF licenses this file to You under the Apache License, Version 2.0
~ (the "License"); you may not use this file except in compliance with
~ the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
~
-->

<assembly
xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2 http://maven.apache.org/xsd/assembly-1.1.2.xsd">
<id>dist</id>
<formats>
<format>tar.gz</format>
</formats>
<fileSets>
<fileSet>
<directory>${project.basedir}/../dist-material/bin</directory>
<outputDirectory>bin</outputDirectory>
<includes>
<include>*.sh</include>
<include>*.bat</include>
</includes>
<fileMode>0755</fileMode>
</fileSet>
<fileSet>
<directory>${project.basedir}/../../dist-material</directory>
<outputDirectory>config</outputDirectory>
<includes>
<include>log4j2.xml</include>
<include>alarm-settings.yml</include>
</includes>
</fileSet>
<fileSet>
<directory>${project.basedir}/../../dist-material</directory>
<outputDirectory/>
<includes>
<include>config-examples/*</include>
</includes>
</fileSet>
<fileSet>
<directory>${project.basedir}/../../oap-server/server-starter/src/main/resources</directory>
<includes>
<include>component-libraries.yml</include>
<include>gateways.yml</include>
<include>service-apdex-threshold.yml</include>
<include>endpoint-name-grouping.yml</include>
<include>metadata-service-mapping.yaml</include>
<include>trace-sampling-policy-settings.yml</include>
<include>oal/*.oal</include>
<include>fetcher-prom-rules/*.yaml</include>
<include>envoy-metrics-rules/**</include>
<include>meter-analyzer-config/*.yaml</include>
<include>zabbix-rules/*.yaml</include>
<include>openapi-definitions/*/*.yaml</include>
<include>otel-rules/**</include>
<include>ui-initialized-templates/*/*.json</include>
<include>ui-initialized-templates/menu.yaml</include>
<include>lal/*</include>
<include>log-mal-rules/**</include>
<include>telegraf-rules/*</include>
</includes>
<outputDirectory>config</outputDirectory>
</fileSet>
<fileSet>
<directory>${project.basedir}/target/oap-libs</directory>
<outputDirectory>oap-libs</outputDirectory>
</fileSet>

<fileSet>
<directory>${project.basedir}/../../oap-server/server-tools/profile-exporter/tool-profile-snapshot-exporter/target/oap-libs</directory>
<outputDirectory>oap-libs</outputDirectory>
</fileSet>

<!-- data generator -->
<fileSet>
<directory>${project.basedir}/../../oap-server/server-tools/data-generator/target/</directory>
<includes>
<include>data-generator-${project.version}.jar</include>
</includes>
<outputDirectory>oap-libs</outputDirectory>
</fileSet>
<fileSet>
<directory>${project.basedir}/../../oap-server/server-tools/data-generator/src/main/assembly/bin</directory>
<outputDirectory>tools/data-generator/bin</outputDirectory>
<includes>
<include>*.sh</include>
</includes>
<fileMode>0755</fileMode>
</fileSet>
<fileSet>
<directory>${project.basedir}/../../oap-server/server-tools/data-generator/src/main/resources</directory>
<outputDirectory>tools/data-generator/config</outputDirectory>
<includes>
<include>application.yml</include>
</includes>
</fileSet>
<!-- data generator -->

<!-- Profile exporter tools -->
<fileSet>
<directory>${project.basedir}/../../tools/profile-exporter</directory>
<outputDirectory>tools/profile-exporter</outputDirectory>
</fileSet>

<fileSet>
<directory>${project.basedir}/../../dist-material/release-docs</directory>
<outputDirectory/>
</fileSet>
</fileSets>
<files>
<file>
<source>${project.basedir}/../graal-server-starter/src/main/resources/application.yml</source>
<outputDirectory>config</outputDirectory>
<fileMode>0644</fileMode>
</file>
</files>
</assembly>
Loading

0 comments on commit 9eb87c1

Please sign in to comment.