Skip to content
This repository was archived by the owner on Aug 23, 2023. It is now read-only.

add docker-cluster quickstart guide #1040

Merged
merged 3 commits into from
Sep 14, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
117 changes: 117 additions & 0 deletions docs/quick-start-docker-cluster.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
# Quick start for docker-cluster

The docker-cluster image is similar to docker-standard, but with the following changes and features:

* Cluster of 4 metrictanks (with 2x replication)
* Separate Graphite monitoring server
* The metrictank binary is mounted from `metrictank/build`, so `make bin` must be run before spinning up the stack (benefits: speeds up development and testing)
* Loads custom configurations and scripts
* Supports tags
* More dashboards are available
* Kafka input and clustering backend
* Short chunkspan & numchunks (benefits: easy to trigger mem and mem_and_cass requests, frequent cass saves, notifier messages etc)
* Prometheus monitoring
* Jaeger tracing
* caddy-proxied endpooints/datasources to check with different org-id auth headers
* tsdb-gw


The following programs are required to build and run the docker-cluster image:

* [Docker](https://docs.docker.com/install/)
* [docker-compose](https://docs.docker.com/compose) >= version 1.6
* [Go](https://golang.org/doc/install) (needed to build metrictank)
* Ensure that `$GOPATH/bin` is added to your `$PATH`

## Getting the repository

Pull down the repository and its dependencies:

```
go get github.com/grafana/metrictank/...
cd $GOPATH/src/github.com/grafana/metrictank
```

## Build metrictank

Build metrictank and the docker images:
```
make
```

`make` uses/creates the directory `$GOPATH/src/github.com/grafana/metrictank/build`. If you tried to spin up the stack prior to building metrictank then docker already created this directory, as root, and it will need to be deleted (then run `make` again).

## Bring up the stack

The stack will listen on the following ports:

* 80 tcp (carbon)
* 2003 tcp (metrictank's carbon input)
* 2181 tcp (kafka)
* 3000 tcp (grafana's http port)
* 6060 tcp (metrictank's internal endpoint)
* 6061 tcp (caddy)
* 6062 tcp (caddy)
* 6063 tcp (caddy)
* 6831 udp (jaeger)
* 8080 tcp (the graphite query endpoint)
* 8081 tcp (caddy)
* 8082 tcp (caddy)
* 8125 udp (statsd endpoint)
* 9042 tcp (cassandra)
* 9090 tcp (prometheus)
* 9092 tcp (kafka)
* 9100 tcp (prometheus node exporter)
* 9999 tcp (kafka)
* 16686 tcp (jaeger)

If you already have something else listening on those ports (such as a carbon or grafana server), shut it down, as it will conflict.


Inside your copy of the repository, you can bring up the stack like so:

```
cd docker/docker-cluster
docker-compose up
```

If this gives you the error `service 'version' doesn't have any configuration options`,
your version of `docker-compose` is too old and you need to update to >=1.6.

Wait until the stack is up.

## Working with Grafana and metrictank

In your browser, open Grafana at http://localhost:3000 (or your docker-machine address) and log in as `admin:admin`.
If Grafana prompts you to change the password, you can skip it, since it doesn't matter for a local test setup.

### Working with the cluster

This stack has some unique properties not found in the standard stack, such as multiple data sources to choose from including, but not limited to:
* monitoring and prometheus for data about the stack
* metrictank to see data in metrictank
* graphite to see data in metrictank via graphite

When using [fakemetrics](https://github.com/raintank/fakemetrics) to generate data you should now use kafka
* `fakemetrics feed --kafka-mdm-addr localhost:9092`
* stats will still show up under `some.id.of.a.metric.*`

If anything doesn't work, please let us know via a ticket on github or reach out on slack. See
[Community](https://github.com/grafana/metrictank/blob/master/docs/community.md)


## Shut down the stack

You can shut down the stack by just hitting `Ctrl-C` on your running `docker-compose up` command.


You can remove the containers like so:
```
docker-compose stop
```

To clean up all data so you can start fresh, run this after you stopped the stack:
```
docker rm -v $(docker ps -a -q -f status=exited)
```
This will remove the stopped containers and their data volumes.
32 changes: 24 additions & 8 deletions docs/quick-start-docker.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,27 @@
# Quick start using Docker

[Docker](docker.io) is a toolkit and a daemon which makes running foreign applications convenient, via containers.
This quick start covers a minimal setup without clustering, if you would prefer to use a cluster please follow the [docker-cluster quick start](quick-start-docker-cluster.md)


[Docker](https://www.docker.com) is a toolkit and a daemon which makes running foreign applications convenient, via containers.


This tutorial will help you run metrictank, its dependencies, and grafana for dashboarding, with minimal hassle.

[docker installation instructions](https://www.docker.com/products/overview)
## Install Docker

[Docker installation instructions](https://docs.docker.com/install/)
You will also need to install version >=1.6 of [docker-compose](https://docs.docker.com/compose/)

## Optional: install Go

[GO installation instructions](https://golang.org/doc/install)

Ensure that `$GOPATH/bin` is added to your `$PATH`

## Getting the repository

If you already have go installed, you can just:
If you already have Go installed, you can just:

```
go get github.com/grafana/metrictank/...
Expand All @@ -24,7 +37,7 @@ cd metrictank

If you have neither, just [download the zip](https://github.com/grafana/metrictank/archive/master.zip), extract it somewhere and cd into the metrictank directory.

## Bring up the stack
## Bring up the stack (docker-standard)

The stack will listen on the following ports:

Expand All @@ -35,7 +48,7 @@ The stack will listen on the following ports:
* 8125 udp (statsd endpoint)
* 9042 tcp (cassandra)

If you already have something else listen to that port (such as a carbon or grafana server), shut it down, as it will conflict.
If you already have something else listening on those ports (such as a carbon or grafana server), shut it down, as it will conflict.


Inside your copy of the repository, you can bring up the stack like so:
Expand Down Expand Up @@ -71,19 +84,22 @@ If you're running Docker engine natively, you can connect using `localhost`. If

```
$ curl http://localhost:6060/node
```
Output will be similar to the following:
```
{"name":"default","version":"0.9.0-342-g9c5a12c","primary":true,"primaryChange":"2018-09-07T19:29:10.200078519Z","state":"NodeReady","priority":0,"started":"2018-09-07T19:29:10.196587689Z","stateChange":"2018-09-07T19:29:10.200078795Z","partitions":[0],"apiPort":6060,"apiScheme":"http","updated":"2018-09-07T19:29:11.26049517Z","remoteAddr":""}
```

## Working with Grafana and metrictank

In your browser, open Grafana at `http://localhost:3000` (or your docker-machine address) and log in as `admin:admin`.
If grafana prompts to change the password, you can skip it, since it doesn't matter for a local test setup.
In your browser, open Grafana at http://localhost:3000 (or your docker-machine address) and log in as `admin:admin`.
If Grafana prompts you to change the password, you can skip it, since it doesn't matter for a local test setup.

### Sending and visualizing data

In the "+" (Create) menu, hit `Dashboard`.
This opens the dashboard editor and has a selector open to add a new panel. Hit "Graph" for graph panel.
The panel will appear but not contain data yet. ([Grafana documentation for graph panel](http://docs.grafana.org/features/panels/graph/))
The panel will appear but not contain data yet. ([Grafana documentation for graph panel](http://docs.grafana.org/features/panels/graph/)).
Click on the title of the panel and hit 'edit'.
In the metrics tab you should see a bunch of metrics already in the root hierarchy:

Expand Down