Skip to content
This repository was archived by the owner on Feb 22, 2022. It is now read-only.

Add fluentd-cloudwatch log forwarder #211

Closed
wants to merge 7 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions incubator/fluentd-cloudwatch/Chart.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
name: fluentd-cloudwatch
version: 0.1.0
description: A Fluentd CloudWatch Helm chart for Kubernetes.
keywords:
- fluentd
- cloudwatch
- logging
sources:
- https://github.com/kubernetes/charts
- https://github.com/18F/cg-deploy-kubernetes
maintainers:
- name: Josh Carp
email: jm.carp@gmail.com
- name: Michael Haselton
email: michael.haselton@gmail.com
engine: gotpl
72 changes: 72 additions & 0 deletions incubator/fluentd-cloudwatch/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Fluentd CloudWatch

* Installs [Fluentd](https://www.fluentd.org/) [Cloudwatch](https://aws.amazon.com/cloudwatch/) log forwarder.

## TL;DR;

```console
$ helm install incubator/fluentd-cloudwatch
```

## Introduction

This chart bootstraps a [Fluentd](https://www.fluentd.org/) [Cloudwatch](https://aws.amazon.com/cloudwatch/) deployment on a [Kubernetes](http://kubernetes.io) cluster using the [Helm](https://helm.sh) package manager.

## Prerequisites

- Kubernetes 1.4+ with Beta APIs enabled

## Installing the Chart

To install the chart with the release name `my-release`:

```console
$ helm install --name my-release incubator/fluentd-cloudwatch
```

The command deploys Fluentd Cloudwatch on the Kubernetes cluster in the default configuration. The [configuration](#configuration) section lists the parameters that can be configured during installation.

## Uninstalling the Chart

To uninstall/delete the `my-release` deployment:

```console
$ helm delete my-release
```

The command removes all the Kubernetes components associated with the chart and deletes the release.

## Configuration

The following tables lists the configurable parameters of the Fluentd Cloudwatch chart and their default values.

| Parameter | Description | Default |
| -------------------------- | ------------------------------------------ | ---------------------------------------------------------- |
| `image` | Image | `18fgsa/fluentd-cloudwatch` |
| `imageTag` | Image tag | `0.1.0` |
| `imagePullPolicy` | Image pull policy | `Always` if `imageTag` is `latest`, else `IfNotPresent` |
| `resources.limits.cpu` | CPU limit | `100m` |
| `resources.limits.memory` | Memory limit | `200Mi` |
| `resources.requests.cpu` | CPU request | `100m` |
| `resources.requests.memory`| Memory request | `200Mi` |
| `hostNetwork` | Host network | `false` |
| `annotations` | Annotations | `nil` |
| `awsRegion` | AWS Cloudwatch region | `us-east-1` |
| `logGroupName` | AWS Cloudwatch log group | `kubernetes` |

Specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. For example,

```console
$ helm install --name my-release \
--set awsRegion=us-east-1 \
incubator/fluentd-cloudwatch
```

Alternatively, a YAML file that specifies the values for the above parameters can be provided while installing the chart. For example,

```console
$ helm install --name my-release -f values.yaml stable/fluentd-cloudwatch
```

### ConfigMap Files
FluentD is configured through [td-agent.conf](http://docs.fluentd.org/articles/config-file). This file (and any others listed in `tdAgentFiles`) will be mounted into the `fluentd` pod.
6 changes: 6 additions & 0 deletions incubator/fluentd-cloudwatch/templates/NOTES.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
To verify that Fluentd Cloudwatch has started, run:

kubectl --namespace={{ .Release.Namespace }} get pods -l "app={{ template "fullname" . }}"

THIS APPLICATION CAPTURES ALL CONSOLE OUTPUT AND FORWARDS IT TO AWS CLOUDWATCH. Anything that might be identifying,
including things like IP addresses, container images, and object names will NOT be anonymized.
16 changes: 16 additions & 0 deletions incubator/fluentd-cloudwatch/templates/_helpers.tpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{{/* vim: set filetype=mustache: */}}
{{/*
Expand the name of the chart.
*/}}
{{- define "name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 24 -}}
{{- end -}}

{{/*
Create a default fully qualified app name.
We truncate at 24 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
*/}}
{{- define "fullname" -}}
{{- $name := default .Chart.Name .Values.nameOverride -}}
{{- printf "%s-%s" .Release.Name $name | trunc 24 -}}
{{- end -}}
11 changes: 11 additions & 0 deletions incubator/fluentd-cloudwatch/templates/configmap.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ template "fullname" . }}
labels:
app: {{ template "fullname" . }}
chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
heritage: {{ .Release.Service | quote }}
release: {{ .Release.Name | quote }}
data:
{{ toYaml .Values.tdAgentFiles | indent 2 }}
52 changes: 52 additions & 0 deletions incubator/fluentd-cloudwatch/templates/daemonset.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: {{ template "fullname" . }}
labels:
app: {{ template "fullname" . }}
chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
heritage: "{{ .Release.Service }}"
release: "{{ .Release.Name }}"
spec:
selector:
matchLabels:
app: {{ template "fullname" . }}
template:
metadata:
labels:
app: {{ template "fullname" . }}
release: "{{ .Release.Name }}"
annotations:
{{ toYaml .Values.annotations | indent 8 }}
spec:
containers:
- name: fluentd-cloudwatch
image: "{{ .Values.image }}:{{ .Values.imageTag }}"
imagePullPolicy: {{ default "" .Values.imagePullPolicy | quote }}
hostNetwork: {{ default false .Values.hostNetwork }}
env:
- name: AWS_REGION
value: {{ .Values.awsRegion }}
- name: LOG_GROUP_NAME
value: {{ .Values.logGroupName }}
resources:
{{ toYaml .Values.resources | indent 10 }}
volumeMounts:
- name: config-volume
mountPath: /etc/td-agent
- name: varlog
mountPath: /var/log
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
readOnly: true
terminationGracePeriodSeconds: 30
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you consider adding hostNetwork: {{ default false .Values.hostNetwork }} here? We rely on it to create journald log streams using the hostname.

volumes:
- name: config-volume
configMap:
name: {{ template "fullname" . }}
- name: varlog
hostPath:
path: /var/log
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
Loading