Skip to content

Commit

Permalink
feat: init
Browse files Browse the repository at this point in the history
  • Loading branch information
@Z-Beatles
Z-Beatles committed Jan 25, 2021
0 parents commit a55fc06
Show file tree
Hide file tree
Showing 46 changed files with 2,550 additions and 0 deletions.
104 changes: 104 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# biz-boot-starter-logger

### 项目介绍
该模块用于将日志上传到elk中(支持RabbitMQ或Kafka两种上传方式)，将异常堆栈信息上传到sentry中。

1. 将logback日志通过Appender发送到指定RabbitMQ消息队列/Kafka消息队列中，然后通过配置logstash的input为
RabbitMQ、output为elasticsearch即可将日志收集到ES中并在Kibana中展示
2. 将error级别日志通过SentryAppender发送到指定的Sentry DSN地址，便于错误日志汇总、Bug排查定位，还能及时收到应用的错误报警
3. 添加调用链路信息到请求头和MDC上下文中，实现微服务的全链路追踪
4. 定制化banner样式
5. 添加默认的 `logback-spring.xml` 配置，实际项目中无需再添加日志配置文件
6. elk中记录的信息除当前服务的基础信息之外，还包含调用方信息、微服务调用链路追踪信息
```
---------- 基础信息 ----------
parentProjectVersion - 父项目版本号。该值为 `biz.logger.version.parent-project` 的值，用于定位基础服务框架的版本问题
appId - 项目唯一标识。该值为 `spring.application.name` 的值
appName - 项目名称。该值为 `spring.application.name` 的值
hostName - 服务器名
hostAddress - 服务器IP
logger - logger名称
threadName - 线程名称
level - 日志级别
time - 日志时间。格式为 yyyy-MM-dd HH:mm:ss.SSS
message - 日志内容
```
示例：`requestId`追踪微服务调用日志
![调用链路追踪](./docs/elk-1.png "调用链路追踪")
![ELK记录信息](./docs/elk-2.png "ELK记录信息")
### 调用链路追踪
1. MDCFilter过滤器
该过滤器用于请求头中的调用方信息添加到 MDC上下文中，并且完善调用链路追踪信息
```
---------- 调用方信息 ----------
api-version - 请求的API版本号
channel - 渠道，调用方标识
device-id - 设备id
---------- 链路追踪信息 ----------
request-id - 请求跟踪号，全链路唯一标识。格式为UUID(32个字符)，来自header或者由该过滤器初始化
sc-client-ip - 请求来源客户端ip
origin-url - 请求来源地址
trace-app-ids - appId调用链路追踪记录。来自header并由该过滤器追加，以`,`分割
trace-app-names - appName调用链路追踪记录。来自header并由该过滤器追加，以`,`分割
trace-host-names - hostName调用链路追踪记录。来自header并由该过滤器追加，以`,`分割
trace-host-addresses - hostAddress调用链路追踪记录。来自header并由该过滤器追加，以`,`分割
```

### 使用方式
1. 添加依赖
```
<dependency>
<groupId>cn.waynechu</groupId>
<artifactId>biz-boot-starter-logger</artifactId>
</dependency>
```
2. 添加配置
```
## sentry
sentry.enable=true
sentry.dsn=http://a1c395c85d244742ae2a50b90f1535b8@sentry.waynechu.cn:9000/2
sentry.stacktrace-app-packages=
## elk-rabbit (两种方式二选一即可)
elk.rabbitmq.enable=true
elk.rabbitmq.host=mq.waynechu.cn
elk.rabbitmq.port=5672
elk.rabbitmq.username=waynechu
elk.rabbitmq.password=123456
elk.rabbitmq.application-id=${spring.application.name}
elk.rabbitmq.virtual-host=/logback
elk.rabbitmq.exchange=topic.loggingExchange
elk.rabbitmq.routing-key=logback.#
elk.rabbitmq.connection-name=biz|${spring.application.name}
## elk-kafka (两种方式二选一即可)
elk.kafka.enable=false
elk.kafka.host=kafka.waynechu.cn
elk.kafka.port=9092
elk.kafka.topic=logback
```
### 其他说明
1. 如果抛出 `org.springframework.amqp.AmqpConnectException` Rabbit health check failed，这是因为`org.springframework.boot.autoconfigure.amqp.RabbitAutoConfiguration`生效。
需添加Rabbitmq配置：
```
spring.rabbitmq.host=mq.waynechu.cn
spring.rabbitmq.port=5672
spring.rabbitmq.username=waynechu
spring.rabbitmq.password=123456
spring.rabbitmq.virtual-host=/logback
spring.rabbitmq.publisher-confirm-type=correlated
spring.rabbitmq.publisher-returns=true
spring.rabbitmq.template.mandatory=true
```
2. (可选)如需要自定义日志相关配置，可新建`src/main/resources/logback-custom-spring.xml`来增加相关配置
```
<?xml version="1.0" encoding="UTF-8"?>
<included>
自定义配置
</included>
```
108 changes: 108 additions & 0 deletions biz-boot-starter-logger.iml
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
<?xml version="1.0" encoding="UTF-8"?>
<module org.jetbrains.idea.maven.project.MavenProjectsManager.isMavenModule="true" type="JAVA_MODULE" version="4">
<component name="FacetManager">
<facet type="Spring" name="Spring">
<configuration />
</facet>
</component>
<component name="NewModuleRootManager" LANGUAGE_LEVEL="JDK_1_8">
<output url="file://$MODULE_DIR$/target/classes" />
<output-test url="file://$MODULE_DIR$/target/test-classes" />
<content url="file://$MODULE_DIR$">
<sourceFolder url="file://$MODULE_DIR$/src/main/java" isTestSource="false" />
<sourceFolder url="file://$MODULE_DIR$/src/main/resources" type="java-resource" />
<excludeFolder url="file://$MODULE_DIR$/target" />
</content>
<orderEntry type="inheritedJdk" />
<orderEntry type="sourceFolder" forTests="false" />
<orderEntry type="library" name="Maven: org.springframework.boot:spring-boot-configuration-processor:2.3.3.RELEASE" level="project" />
<orderEntry type="library" name="Maven: org.springframework:spring-web:5.2.8.RELEASE" level="project" />
<orderEntry type="library" name="Maven: org.springframework:spring-beans:5.2.8.RELEASE" level="project" />
<orderEntry type="library" name="Maven: org.springframework:spring-core:5.2.8.RELEASE" level="project" />
<orderEntry type="library" name="Maven: org.springframework:spring-jcl:5.2.8.RELEASE" level="project" />
<orderEntry type="library" name="Maven: javax.servlet:javax.servlet-api:4.0.1" level="project" />
<orderEntry type="library" name="Maven: io.github.openfeign:feign-core:10.10.1" level="project" />
<orderEntry type="library" name="Maven: io.sentry:sentry-logback:1.7.27" level="project" />
<orderEntry type="library" name="Maven: io.sentry:sentry:1.7.27" level="project" />
<orderEntry type="library" name="Maven: com.fasterxml.jackson.core:jackson-core:2.11.2" level="project" />
<orderEntry type="library" name="Maven: org.slf4j:slf4j-api:1.7.30" level="project" />
<orderEntry type="library" name="Maven: ch.qos.logback:logback-core:1.2.3" level="project" />
<orderEntry type="library" name="Maven: ch.qos.logback:logback-classic:1.2.3" level="project" />
<orderEntry type="library" name="Maven: org.springframework.boot:spring-boot-starter-amqp:2.3.3.RELEASE" level="project" />
<orderEntry type="library" name="Maven: org.springframework.boot:spring-boot-starter:2.3.3.RELEASE" level="project" />
<orderEntry type="library" name="Maven: org.springframework.boot:spring-boot:2.3.3.RELEASE" level="project" />
<orderEntry type="library" name="Maven: org.springframework.boot:spring-boot-autoconfigure:2.3.3.RELEASE" level="project" />
<orderEntry type="library" name="Maven: org.springframework.boot:spring-boot-starter-logging:2.3.3.RELEASE" level="project" />
<orderEntry type="library" name="Maven: org.apache.logging.log4j:log4j-to-slf4j:2.13.3" level="project" />
<orderEntry type="library" name="Maven: org.apache.logging.log4j:log4j-api:2.13.3" level="project" />
<orderEntry type="library" name="Maven: org.slf4j:jul-to-slf4j:1.7.30" level="project" />
<orderEntry type="library" name="Maven: jakarta.annotation:jakarta.annotation-api:1.3.5" level="project" />
<orderEntry type="library" name="Maven: org.yaml:snakeyaml:1.26" level="project" />
<orderEntry type="library" name="Maven: org.springframework:spring-messaging:5.2.8.RELEASE" level="project" />
<orderEntry type="library" name="Maven: org.springframework.amqp:spring-rabbit:2.2.10.RELEASE" level="project" />
<orderEntry type="library" name="Maven: com.rabbitmq:amqp-client:5.9.0" level="project" />
<orderEntry type="library" name="Maven: org.springframework.amqp:spring-amqp:2.2.10.RELEASE" level="project" />
<orderEntry type="library" name="Maven: org.springframework.retry:spring-retry:1.2.5.RELEASE" level="project" />
<orderEntry type="library" name="Maven: org.springframework:spring-context:5.2.8.RELEASE" level="project" />
<orderEntry type="library" name="Maven: org.springframework:spring-aop:5.2.8.RELEASE" level="project" />
<orderEntry type="library" name="Maven: org.springframework:spring-expression:5.2.8.RELEASE" level="project" />
<orderEntry type="library" name="Maven: org.springframework:spring-tx:5.2.8.RELEASE" level="project" />
<orderEntry type="library" name="Maven: com.github.danielwegener:logback-kafka-appender:0.2.0-RC2" level="project" />
<orderEntry type="library" name="Maven: org.apache.kafka:kafka-clients:2.5.1" level="project" />
<orderEntry type="library" name="Maven: com.github.luben:zstd-jni:1.4.4-7" level="project" />
<orderEntry type="library" name="Maven: org.lz4:lz4-java:1.7.1" level="project" />
<orderEntry type="library" name="Maven: org.xerial.snappy:snappy-java:1.1.7.3" level="project" />
<orderEntry type="library" name="Maven: org.codehaus.janino:janino:3.1.2" level="project" />
<orderEntry type="library" name="Maven: org.codehaus.janino:commons-compiler:3.1.2" level="project" />
<orderEntry type="module" module-name="biz-spring-cloud-common" />
<orderEntry type="module" module-name="biz-facade-common" />
<orderEntry type="library" name="Maven: org.hibernate.validator:hibernate-validator:6.1.5.Final" level="project" />
<orderEntry type="library" name="Maven: jakarta.validation:jakarta.validation-api:2.0.2" level="project" />
<orderEntry type="library" name="Maven: org.jboss.logging:jboss-logging:3.4.1.Final" level="project" />
<orderEntry type="library" name="Maven: com.fasterxml:classmate:1.5.1" level="project" />
<orderEntry type="library" name="Maven: io.springfox:springfox-swagger2:2.9.2" level="project" />
<orderEntry type="library" name="Maven: io.swagger:swagger-annotations:1.5.20" level="project" />
<orderEntry type="library" name="Maven: io.swagger:swagger-models:1.5.20" level="project" />
<orderEntry type="library" name="Maven: com.fasterxml.jackson.core:jackson-annotations:2.11.2" level="project" />
<orderEntry type="library" name="Maven: io.springfox:springfox-spi:2.9.2" level="project" />
<orderEntry type="library" name="Maven: io.springfox:springfox-core:2.9.2" level="project" />
<orderEntry type="library" name="Maven: net.bytebuddy:byte-buddy:1.10.14" level="project" />
<orderEntry type="library" name="Maven: io.springfox:springfox-schema:2.9.2" level="project" />
<orderEntry type="library" name="Maven: io.springfox:springfox-swagger-common:2.9.2" level="project" />
<orderEntry type="library" name="Maven: io.springfox:springfox-spring-web:2.9.2" level="project" />
<orderEntry type="library" name="Maven: com.google.guava:guava:29.0-jre" level="project" />
<orderEntry type="library" name="Maven: com.google.guava:failureaccess:1.0.1" level="project" />
<orderEntry type="library" name="Maven: com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava" level="project" />
<orderEntry type="library" name="Maven: com.google.code.findbugs:jsr305:3.0.2" level="project" />
<orderEntry type="library" name="Maven: org.checkerframework:checker-qual:2.11.1" level="project" />
<orderEntry type="library" name="Maven: com.google.errorprone:error_prone_annotations:2.3.4" level="project" />
<orderEntry type="library" name="Maven: com.google.j2objc:j2objc-annotations:1.3" level="project" />
<orderEntry type="library" name="Maven: org.springframework.plugin:spring-plugin-core:1.2.0.RELEASE" level="project" />
<orderEntry type="library" name="Maven: org.springframework.plugin:spring-plugin-metadata:1.2.0.RELEASE" level="project" />
<orderEntry type="library" name="Maven: org.mapstruct:mapstruct:1.2.0.Final" level="project" />
<orderEntry type="library" name="Maven: com.github.pagehelper:pagehelper-spring-boot-starter:1.2.12" level="project" />
<orderEntry type="library" name="Maven: com.github.pagehelper:pagehelper:5.1.10" level="project" />
<orderEntry type="library" name="Maven: com.github.jsqlparser:jsqlparser:2.0" level="project" />
<orderEntry type="library" name="Maven: com.alibaba:fastjson:1.2.71" level="project" />
<orderEntry type="library" name="Maven: com.alibaba:easyexcel:2.2.6" level="project" />
<orderEntry type="library" name="Maven: org.apache.poi:poi:3.17" level="project" />
<orderEntry type="library" name="Maven: commons-codec:commons-codec:1.14" level="project" />
<orderEntry type="library" name="Maven: org.apache.commons:commons-collections4:4.1" level="project" />
<orderEntry type="library" name="Maven: org.apache.poi:poi-ooxml:3.17" level="project" />
<orderEntry type="library" name="Maven: com.github.virtuald:curvesapi:1.04" level="project" />
<orderEntry type="library" name="Maven: org.apache.poi:poi-ooxml-schemas:3.17" level="project" />
<orderEntry type="library" name="Maven: org.apache.xmlbeans:xmlbeans:2.6.0" level="project" />
<orderEntry type="library" name="Maven: stax:stax-api:1.0.1" level="project" />
<orderEntry type="library" name="Maven: cglib:cglib:3.1" level="project" />
<orderEntry type="library" name="Maven: org.ow2.asm:asm:4.2" level="project" />
<orderEntry type="library" name="Maven: org.ehcache:ehcache:3.8.1" level="project" />
<orderEntry type="library" name="Maven: org.glassfish.jaxb:jaxb-runtime:2.3.3" level="project" />
<orderEntry type="library" name="Maven: jakarta.xml.bind:jakarta.xml.bind-api:2.3.3" level="project" />
<orderEntry type="library" name="Maven: org.glassfish.jaxb:txw2:2.3.3" level="project" />
<orderEntry type="library" name="Maven: com.sun.istack:istack-commons-runtime:3.0.11" level="project" />
<orderEntry type="library" scope="RUNTIME" name="Maven: com.sun.activation:jakarta.activation:1.2.2" level="project" />
<orderEntry type="library" name="Maven: org.apache.commons:commons-lang3:3.10" level="project" />
<orderEntry type="library" name="Maven: cn.hutool:hutool-core:5.3.8" level="project" />
<orderEntry type="library" name="Maven: org.projectlombok:lombok:1.18.12" level="project" />
</component>
</module>
Binary file added docs/elk-1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/elk-2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
91 changes: 91 additions & 0 deletions pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<parent>
<groupId>cn.waynechu</groupId>
<artifactId>biz-parent</artifactId>
<version>1.0.0-RELEASE</version>
</parent>

<artifactId>biz-boot-starter-logger</artifactId>
<version>1.0.0-RELEASE</version>
<name>biz-boot-starter-logger</name>
<description>biz-boot-starter-logger module</description>

<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
</properties>

<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-configuration-processor</artifactId>
<optional>true</optional>
</dependency>

<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-web</artifactId>
</dependency>

<dependency>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
</dependency>

<dependency>
<groupId>io.github.openfeign</groupId>
<artifactId>feign-core</artifactId>
</dependency>

<!-- Sentry -->
<dependency>
<groupId>io.sentry</groupId>
<artifactId>sentry-logback</artifactId>
</dependency>

<!-- Rabbit MQ for Logback AmqpAppender-->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>

<dependency>
<groupId>com.github.danielwegener</groupId>
<artifactId>logback-kafka-appender</artifactId>
<version>0.2.0-RC2</version>
</dependency>

<!-- Logback condition表达式支持 -->
<dependency>
<groupId>org.codehaus.janino</groupId>
<artifactId>janino</artifactId>
</dependency>

<dependency>
<groupId>cn.waynechu</groupId>
<artifactId>biz-spring-cloud-common</artifactId>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
<executions>
<execution>
<id>attach-sources</id>
<goals>
<goal>jar-no-fork</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
40 changes: 40 additions & 0 deletions src/main/java/cn/waynechu/bootstarter/logger/LoggerAutoConfiguration.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
package cn.waynechu.bootstarter.logger;

import cn.waynechu.bootstarter.logger.aware.SentryContextAware;
import cn.waynechu.bootstarter.logger.filter.MDCFilter;
import cn.waynechu.bootstarter.logger.properties.ElkProperty;
import cn.waynechu.bootstarter.logger.properties.LoggerProperty;
import cn.waynechu.bootstarter.logger.properties.SentryProperties;
import cn.waynechu.bootstarter.logger.provider.ApplicationProvider;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.boot.web.servlet.FilterRegistrationBean;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Import;

/**
* @author zhuwei
* @since 2019/1/8 13:50
*/
@Configuration
@EnableConfigurationProperties({SentryProperties.class, ElkProperty.class, LoggerProperty.class})
@Import({ApplicationProvider.class})
public class LoggerAutoConfiguration {

@Bean
@ConditionalOnProperty(value = SentryContextAware.SENTRY_ENABLE, havingValue = "true")
public SentryContextAware sentryInitializer() {
return new SentryContextAware();
}

@Bean
public FilterRegistrationBean<MDCFilter> mdcFilterRegistrationBean() {
FilterRegistrationBean<MDCFilter> registrationBean = new FilterRegistrationBean<>();
MDCFilter mdcFilter = new MDCFilter();

registrationBean.setFilter(mdcFilter);
registrationBean.setOrder(1);
return registrationBean;
}
}
Loading

0 comments on commit a55fc06

Please sign in to comment.