adding slf4j logging adapter #1655

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

jabubake wants to merge 2 commits into googleapis:master from jabubake:master

google-cloud-contrib/google-cloud-logging-adapters/README.md

-Original file line number
+Diff line change
@@ -0,0 +1,8 @@
+    # cloud-logging-adapters
+    Cloud logging adapters with standard Java frameworks
+    - Write structured logs to Cloud logging, stdout or via socket (Fluentd forwarding port)
+    Slf4j  :
+    - [README](cloud-logging-over-slf4j/README.md)
+    - Example application [here](test-adapters)

...-cloud-contrib/google-cloud-logging-adapters/cloud-logging-over-slf4j/README.md

-Original file line number
+Diff line change
@@ -0,0 +1,67 @@
+    ## Slf4j Adapter for Google Cloud Logging
+    Built with [Slf4j](https://www.slf4j.org/) v1.7.21 and
+    [Google cloud logging library](https://github.com/GoogleCloudPlatform/google-cloud-java/tree/master/google-cloud-logging)
+    Adapter library for [Google Cloud logging](https://cloud.google.com/logging/docs/view/logs_viewer_v2) over Slf4j.
+    ## Configuration and setup
+    Install [Apache Maven](https://maven.apache.org/) <p>
+    `mvn clean install`
+    If you are using Maven, add this to your pom.xml file
+    ```xml
+    <dependency>
+      <groupId>com.example</groupId>
+      <artifactId>cloud-logging-over-slf4j</artifactId>
+      <version>0.1.0-SNAPSHOT</version>
+    </dependency>
+    ```
+    ### Configuration file
+    Set system property or environment variable `CLOUD_LOGGING_CONFIG` to the absolute path of .yaml configuration file. <p>
+    Here is a sample configuration file : [google-cloud-logging.yaml](src/test/resources/google-cloud-logging.yaml)<p>
+    #### Fields
+    - `destination` (optional) : defaults to direct writes via API. <p>
+    Allowed values : <p>
+     -`"stdout"` : logs to standard out <p>
+     -`"fluentd"` logs to fluentd forwarding port (`localhost:24224`)
+    - `loggers` (list) : (optional)
+      ##### Required fields for custom loggers :
+      - `name` : "default" or package/class name
+      ##### Optional fields :
+      - `log_name` defaults to `name` if not specified <p>
+      - `resource` defaults to `global`, restricted to
+      [Supported ResourceTypes](https://cloud.google.com/logging/docs/api/v2/resource-list)<p>
+      - `enable`  : boolean [default : true] <p>
+      - `level` : Min log level. Supported log levels [here](https://www.slf4j.org/api/org/apache/commons/logging/Log.html)<p>
+      - `labels` (optional) : Adds custom labels to logs <p>
+         - Format list of key, value pairs, eg `project_id: "$GOOGLE_CLOUD_PROJECT"`
+         - prefix values with `$` to use system/environment variable values.
+       <p> Use `default` logger name to update default logging parameters <p>
+    ## Usage
+    Initialize the service in a class with
+    `Logger logger = LoggerFactory.getLogger(loggerName)`
+    - Log using standard Slf4j formatting : <p>
+      - `logger.info("Hello world {}: INFO", "app")` <p>
+      - `logger.error("Hello world : {}", "exception", new Throwable("illegal argument"))` <p>
+      - `logger.debug("Hello world : DEBUG")` <p>
+      - `logger.trace("Hello world : TRACE")` <p>
+     - Markers can be used to set Cloud logging log levels that are not supported via Slf4j
+        : `NOTICE`, `CRITICAL`, `ALERT`, `EMERGENCY`<p>
+        Examples : <p>
+        - `logger.info(MarkerFactory.getMarker("NOTICE"), Hello world : INFO_NOTICE")` <p>
+        - `logger.error(MarkerFactory.getMarker("CRITICAL"), "Hello world : Error")` <p>
+     - Markers can also be used to add custom labels <p>
+     `logger.info(MarkerFactory.getMarker("label1:value1"), "Hello world : labels")` <p>
+    ## Todo
+    <p> Add support for fluentd port forwarding config, currently hardcoded

google-cloud-contrib/google-cloud-logging-adapters/cloud-logging-over-slf4j/pom.xml

-Original file line number
+Diff line change
@@ -0,0 +1,42 @@
+    <?xml version="1.0" encoding="UTF-8"?>
+    <project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+      xmlns="http://maven.apache.org/POM/4.0.0"
+      xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+      <modelVersion>4.0.0</modelVersion>
+      <groupId>com.example</groupId>
+      <artifactId>cloud-logging-over-slf4j</artifactId>
+      <version>0.1.0-SNAPSHOT</version>
+      <properties>
+        <slf4jVersion>1.7.21</slf4jVersion>
+        <googleCloudLoggingVersion>0.9.3-beta</googleCloudLoggingVersion>
+        <maven.compiler.target>1.7</maven.compiler.target>
+        <maven.compiler.source>1.7</maven.compiler.source>
+      </properties>
+      <dependencies>
+        <dependency>
+          <groupId>org.slf4j</groupId>
+          <artifactId>slf4j-api</artifactId>
+          <version>${slf4jVersion}</version>
+        </dependency>
+        <dependency>
+          <groupId>org.yaml</groupId>
+          <artifactId>snakeyaml</artifactId>
+          <version>1.11</version>
+        </dependency>
+        <dependency>
+          <groupId>com.google.cloud</groupId>
+          <artifactId>google-cloud-logging</artifactId>
+          <version>${googleCloudLoggingVersion}</version>
+        </dependency>
+        <dependency>
+          <groupId>junit</groupId>
+          <artifactId>junit</artifactId>
+          <version>4.12</version>
+        </dependency>
+        <dependency>
+          <groupId>com.fasterxml.jackson.core</groupId>
+          <artifactId>jackson-databind</artifactId>
+          <version>2.8.5</version>
+        </dependency>
+      </dependencies>
+    </project>

...d-logging-adapters/cloud-logging-over-slf4j/src/main/java/org/slf4j/impl/CloudLogger.java

-Original file line number
+Diff line change
@@ -0,0 +1,134 @@
+    /*
+     * Copyright 2017 Google Inc. All Rights Reserved.
+     *
+     * Licensed under the Apache License, Version 2.0 (the "License");
+     * you may not use this file except in compliance with the License.
+     * You may obtain a copy of the License at
+     *
+     *       http://www.apache.org/licenses/LICENSE-2.0
+     *
+     * Unless required by applicable law or agreed to in writing, software
+     * distributed under the License is distributed on an "AS IS" BASIS,
+     * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+     * See the License for the specific language governing permissions and
+     * limitations under the License.
+     */
+    package org.slf4j.impl;
+    import com.google.cloud.MonitoredResource;
+    import com.google.cloud.logging.LogEntry;
+    import com.google.cloud.logging.Payload.StringPayload;
+    import com.google.cloud.logging.Severity;
+    import com.google.common.base.Splitter;
+    import java.io.PrintWriter;
+    import java.io.StringWriter;
+    import java.util.ArrayList;
+    import java.util.List;
+    import java.util.Map;
+    import org.slf4j.Marker;
+    import org.slf4j.impl.config.ConfigLoader;
+    import org.slf4j.impl.logging.ILogging;
+    import org.slf4j.impl.logging.ILoggingFactory;
+    /** Cloud logging service with external configuration */
+    class CloudLogger {
+      private static final String CONFIGURATION_FILE = "google-cloud-logging.yaml";
+      private static final String CLOUD_LOGGING_CONFIG = "CLOUD_LOGGING_CONFIG";
+      // 100kb payload limit : limit string payload to 95 kb (assume 2 bytes per character)
+      private static final int MAX_LOG_SIZE_IN_CHARS = 1024 * 95 / 2;
+      private static ConfigLoader configLoader;
+      private static ILogging logging;
+      private MonitoredResource resource;
+      private String fileName;
+      private String loggerName;
+      private Severity minSeverity;
+      private boolean isEnabled;
+      private Map<String, String> labels;
+      CloudLogger(String loggerName) throws Exception {
+          init(loggerName);
+      }
+      static void init() {
+          String configFile = ConfigLoader.getValue(CLOUD_LOGGING_CONFIG, CONFIGURATION_FILE);
+          configLoader = new ConfigLoader(configFile);
+          logging = ILoggingFactory.get(configLoader.getDestination());
+          logging.init();
+      }
+      String getName() {
+        return loggerName;
+      }
+      boolean isEnabled(Severity severity) {
+        return (isEnabled && severity.compareTo(minSeverity) >= 0);
+      }
+      void log(Marker marker, Severity severity, String text, Throwable throwable) {
+        writeLogEntries(marker, severity, text, throwable);
+      }
+      void log(Severity severity, String text, Throwable throwable) {
+        writeLogEntries(null, severity, text, throwable);
+      }
+      void log(Marker marker, Severity severity, String text) {
+        writeLogEntries(marker, severity, text, null);
+      }
+      void log(Severity severity, String text) {
+        log(null, severity, text, null);
+      }
+      private void updateConfig() {
+        minSeverity = configLoader.getSeverity(loggerName);
+        labels = configLoader.getLabels(loggerName);
+        fileName = configLoader.getFileName(loggerName);
+        this.resource = MonitoredResource.newBuilder(configLoader.getResource(loggerName)).build();
+        this.isEnabled = configLoader.isEnabled(loggerName);
+      }
+      private void init(String loggerName) throws Exception {
+        this.loggerName = loggerName;
+        updateConfig();
+      }
+      private LogEntry createLogEntry(Marker marker, Severity severity, String text) {
+        LogDetailsTuple logDetailsTuple = new LogDetailsTuple(marker, labels, severity);
+        return LogEntry.newBuilder(StringPayload.of(text))
+            .setResource(resource)
+            .setLabels(logDetailsTuple.getLabels())
+            .setLogName(fileName)
+            .setSeverity(logDetailsTuple.getLevel())
+            .build();
+      }
+      private Iterable<String> getChunks(String text) {
+        return Splitter.fixedLength(MAX_LOG_SIZE_IN_CHARS)
+            .omitEmptyStrings()
+            .split(text);
+      }
+      private String getStackTrace(Throwable throwable) {
+        StringWriter errors = new StringWriter();
+        throwable.printStackTrace(new PrintWriter(errors));
+        return errors.toString();
+      }
+      private void writeLogEntries(Marker marker, Severity level, String text, Throwable t) {
+        if (!isEnabled(level)) {
+          return;
+        }
+        if (t != null) {
+          text = text + "\n" + getStackTrace(t);
+        }
+        Iterable<String> chunks = getChunks(text);
+        List<LogEntry> logEntries = new ArrayList<>();
+        for (String chunk : chunks) {
+          logEntries.add(createLogEntry(marker, level, chunk));
+        }
+        logging.write(logEntries);
+      }
+    }

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

adding slf4j logging adapter #1655

Uh oh!

Diff view

Diff view

There are no files selected for viewing

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

Uh oh!