Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add NATS client as option for publishing container logs #369

Merged
merged 2 commits into from
Nov 22, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion engine/admin/logging/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,5 @@ title: Logging drivers
* [Amazon CloudWatch Logs logging driver](awslogs.md)
* [Splunk logging driver](splunk.md)
* [ETW logging driver](etwlogs.md)
* [Google Cloud Logging driver](gcplogs.md)
* [Google Cloud Logging driver](gcplogs.md)
* [NATS Logging driver](nats.md)
88 changes: 88 additions & 0 deletions engine/admin/logging/nats.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
---
description: Describes how to use NATS for publishing log entries
keywords: NATS, nats.io, messaging, docker, logging, driver
title: NATS logging driver
---

Docker logging driver for sending container the logs as events published to NATS in JSON format.

## Usage

You can configure the default logging driver by passing the `--log-driver`
option to the Docker daemon:

```bash
$ dockerd --log-driver=nats
```

You can set the logging driver for a specific container by using the
`--log-driver` option to `docker run`:

```bash
$ docker run --log-driver=nats ...
```

This log driver does not implement a reader so it is incompatible with `docker logs`.

## nats options

You can use the `--log-opt NAME=VALUE` flag to customize the logging driver
for NATS:

| Option | Required | Description |
|-----------------------------|----------|---------------------------------------------------------------------------------------------------------------------------------------------|
| `labels` | optional | Comma-separated list of keys of labels, which should be included in message, if these labels are specified for container. |
| `env` | optional | Comma-separated list of keys of environment variables, which should be included in message, if these variables are specified for container. |
| `tag` | optional | Specify tag for message. Refer to the [log tag option documentation](log_tags.md) for customizing the log tag format. |
| `nats-servers` | optional | NATS cluster nodes separated by commas. e.g. `nats://127.0.0.1:4222,nats://127.0.0.1:4223`. Defaults to `localhost:4222` |
| `nats-max-reconnect` | optional | Maximum attempts that the driver will try to connect before giving up. Defaults to infinite (`-1`) |
| `nats-subject` | optional | Specific subject to which logs will be published. Defaults to using `tag` if not specified |
| `nats-user` | optional | Specify user in case of authentication required |
| `nats-pass` | optional | Specify password in case of authentication required |
| `nats-token` | optional | Specify token in case of authentication required |
| `nats-tls-ca-cert` | optional | Specified the absolute path to the trust certificates signed by the CA |
| `nats-tls-cert` | optional | Specifies the absolute path to the TLS certificate file |
| `nats-tls-key` | optional | Specifies the absolute path to the TLS key file |
| `nats-tls-skip-verify` | optional | Specifies whether to skip verification by setting it to `true` |

Below is an example usage of the driver for sending logs to a node in a
NATS cluster to the `docker.logs` subject:

```bash
$ docker run --log-driver=nats \
--log-opt nats-subject=docker.logs \
--log-opt nats-servers=nats://nats-node-1:4222,nats://nats-node-2:4222,nats://nats-node-3:4222 \
your/application
```

By default, the tag is used as the subject for NATS, so it has to be a valid
subject in case subject it is left unspecified:

```bash
{% raw %}
$ docker run --log-driver nats \
--log-opt tag="docker.{{.ID}}.{{.ImageName}}"
your/application
{% endraw %}
```

Secure connection to NATS using TLS can be customized by setting `tls://` scheme
in the URI and absolute paths to the certs and key files:

```bash
docker run --log-driver nats \
--log-opt nats-tls-key=/srv/configs/certs/client-key.pem \
--log-opt nats-tls-cert=/srv/configs/certs/client-cert.pem \
--log-opt nats-tls-ca-cert=/srv/configs/certs/ca.pem \
--log-opt nats-servers="tls://127.0.0.1:4223,tls://127.0.0.1:4222" \
your/application
```

Skip verify is enabled by default, in order to deactivate we can specify `nats-tls-skip-verify`:

```bash
docker run --log-driver nats \
--log-opt nats-tls-skip-verify \
--log-opt nats-servers="tls://127.0.0.1:4223,tls://127.0.0.1:4222" \
your/application
```
98 changes: 58 additions & 40 deletions engine/admin/logging/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,46 +25,44 @@ supported:
| `splunk` | Splunk logging driver for Docker. Writes log messages to `splunk` using HTTP Event Collector. |
| `etwlogs` | ETW logging driver for Docker on Windows. Writes log messages as ETW events. |
| `gcplogs` | Google Cloud Logging driver for Docker. Writes log messages to Google Cloud Logging. |
| `nats` | NATS logging driver for Docker. Publishes log entries to a NATS server. |

The `docker logs` command is available only for the `json-file` and `journald`
The `docker logs` command is only available for the `json-file` and `journald`
logging drivers.

The `labels` and `env` options add additional attributes for use with logging
drivers that accept them. Each option takes a comma-separated list of keys. If
there is collision between `label` and `env` keys, the value of the `env` takes
precedence.
there is a collision between `label` and `env` keys, the value of the `env`
takes precedence.

To use attributes, specify them when you start the Docker daemon. For example,
to manually start the daemon with the `json-file` driver, and include additional
attributes in the output, run the following command:
Specify attributes when you start the Docker daemon. For example, the following
command starts the daemon with the `json-file` driver and includes additional
attributes in the output.

```bash
$ dockerd \
--log-driver=json-file \
--log-opt labels=foo \
--log-opt env=foo,fizz
$ dockerd --log-driver=json-file \
--log-opt labels=foo \
--log-opt env=foo,fizz
```

Then, run a container and specify values for the `labels` or `env`. For
example, you might use this:
This command runs a container with specific values for the `labels` or `env`.

```bash
$ docker run -dit --label foo=bar -e fizz=buzz alpine sh
```

This adds additional fields to the log depending on the driver, e.g. for
`json-file` that looks like:
This adds additional fields to the log depending on the driver. If you use
`json-file`, it might add the attributes as follows:

```json
"attrs":{"fizz":"buzz","foo":"bar"}
```


## json-file options

The following logging options are supported for the `json-file` logging driver:

```no-highlight
```none
--log-opt max-size=[0-9]+[kmg]
--log-opt max-file=[0-9]+
--log-opt labels=label1,label2
Expand All @@ -87,7 +85,7 @@ from the newest log file.

The following logging options are supported for the `syslog` logging driver:

```no-highlight
```none
--log-opt syslog-address=[tcp|udp|tcp+tls]://host:port
--log-opt syslog-address=unix://path
--log-opt syslog-address=unixgram://path
Expand Down Expand Up @@ -205,7 +203,7 @@ By default, Docker uses the first 12 characters of the container ID to tag log
messages. Refer to the [log tag option documentation](log_tags.md) for
customizing the log tag format.

The `labels` and `env` options are supported by the gelf logging
The `labels` and `env` options are supported by the `gelf` logging
driver. It adds additional key on the `extra` fields, prefixed by an
underscore (`_`).
```json
Expand Down Expand Up @@ -240,24 +238,24 @@ For example, to specify both additional options:
```bash
{% raw %}
$ docker run -dit \
--log-driver=fluentd \
--log-opt fluentd-address=localhost:24224 \
--log-opt tag="docker.{{.Name}}" \
alpine sh
--log-driver=fluentd \
--log-opt fluentd-address=localhost:24224 \
--log-opt tag="docker.{{.Name}}" \
alpine sh
{% endraw %}
```

If container cannot connect to the Fluentd daemon on the specified address and
`fluentd-async-connect` is not enabled, the container stops immediately.
For detailed information on working with this logging driver,
see [the fluentd logging driver](fluentd.md)

If cthe ontainer cannot connect to the Fluentd daemon on the specified address
and `fluentd-async-connect` is not enabled, the container stops immediately. For
detailed information about working with this logging driver, see [the fluentd
logging driver](fluentd.md)

## Amazon CloudWatch Logs options

The Amazon CloudWatch Logs logging driver supports the following options:

```bash

```none
{% raw %}
--log-opt awslogs-region=<aws_region>
--log-opt awslogs-group=<log_group_name>
Expand All @@ -266,41 +264,61 @@ The Amazon CloudWatch Logs logging driver supports the following options:
{% endraw %}
```

For detailed information on working with this logging driver, see [the awslogs
logging driver](awslogs.md) reference documentation.
For detailed information about working with this logging driver, see
[the awslogs logging driver](awslogs.md) reference documentation.

## Splunk options

The Splunk logging driver requires the following options:
The `splunk`` logging driver requires the following options:

```no-highlight
```none
--log-opt splunk-token=<splunk_http_event_collector_token>
--log-opt splunk-url=https://your_splunk_instance:8088
```

For detailed information about working with this logging driver, see the
For detailed information about working with the `splunk` logging driver, see the
[Splunk logging driver](splunk.md) reference documentation.

## ETW logging driver options

The etwlogs logging driver does not require any options to be specified. This
logging driver forwards each log message as an ETW event. An ETW listener
can then be created to listen for these events.
The `etwlogs` logging driver forwards each log message as an ETW event. An ETW
listener can then be created to listen for these events. This driver does not
accept any options.

The ETW logging driver is only available on Windows. For detailed information
on working with this logging driver, see [the ETW logging driver](etwlogs.md)
about working with this logging driver, see [the ETW logging driver](etwlogs.md)
reference documentation.

## Google Cloud Logging options

The Google Cloud Logging driver supports the following options:

```no-highlight
```none
--log-opt gcp-project=<gcp_projext>
--log-opt labels=<label1>,<label2>
--log-opt env=<envvar1>,<envvar2>
--log-opt log-cmd=true
```

For detailed information about working with this logging driver, see the
[Google Cloud Logging driver](gcplogs.md). reference documentation.
For detailed information about working with the Google Cloud logging driver, see
the [Google Cloud Logging driver](gcplogs.md). reference documentation.

## NATS logging options

The NATS logging driver supports the following options:

```none
--log-opt labels=<label1>,<label2>
--log-opt env=<envvar1>,<envvar2>
--log-opt tag=<tag>
--log-opt nats-servers="<comma separated list of nats servers uris>"
--log-opt nats-max-reconnect="<max attempts to connect to a server>"
--log-opt nats-subject="<subject where logs are sent>"
--log-opt nats-tls-ca-cert="<absolute path to cert>"
--log-opt nats-tls-cert="<absolute path to cert>"
--log-opt nats-tls-key="<absolute path to cert>"
--log-opt nats-tls-skip-verify="<value>"
```

For detailed information, see [the NATS logging driver](nats.md) reference
documentation.