Skip to content

Commit 6872a73

Browse files
ConnorDoyleTobi Knaup
ConnorDoyle
authored and
Tobi Knaup
committed
Getting started guide for Mesos on GCP.
1 parent 0cf3590 commit 6872a73

File tree

4 files changed

+302
-0
lines changed

4 files changed

+302
-0
lines changed

‎docs/getting-started-guides/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ If you are considering contributing a new guide, please read the
88
IaaS Provider | Config. Mgmt | OS | Networking | Docs | Support Level | Notes
99
-------------- | ------------ | ------ | ---------- | ---------------------------------------------------- | ---------------------------- | -----
1010
GCE | Saltstack | Debian | GCE | [docs](../../docs/getting-started-guides/gce.md) | Project | Tested with 0.13.2 by @brendandburns
11+
Mesos/GCE | | | | [docs](../../docs/getting-started-guides/mesos.md) | [Community](https://github.com/mesosphere/kubernetes-mesos) ([@jdef](https://github.com/jdef)) | Uses K8s v0.11.0
1112
Vagrant | Saltstack | Fedora | OVS | [docs](../../docs/getting-started-guides/vagrant.md) | Project |
1213
Bare-metal | custom | Fedora | _none_ | [docs](../../docs/getting-started-guides/fedora/fedora_manual_config.md) | Project | Uses K8s v0.13.2
1314
Bare-metal | Ansible | Fedora | flannel | [docs](../../docs/getting-started-guides/fedora/fedora_ansible_config.md) | Project | Uses K8s v0.13.2

‎docs/getting-started-guides/mesos.md

Lines changed: 301 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,301 @@
1+
## Getting started with Kubernetes on Mesos
2+
3+
<!-- TODO: Update, clean up. -->
4+
5+
Mesos allows dynamic sharing of cluster resources between Kubernetes and other first-class Mesos frameworks such as [Hadoop][1], [Spark][2], and [Chronos][3].
6+
Mesos ensures applications from different frameworks running on your cluster are isolated and that resources are allocated fairly.
7+
8+
Running Kubernetes on Mesos allows you to easily move Kubernetes workloads from one cloud provider to another to your own physical datacenter.
9+
10+
This tutorial will walk you through setting up Kubernetes on a Mesos cluster on [Google Cloud Plaform][4].
11+
It provides a step by step walk through of adding Kubernetes to a Mesos cluster and running the classic GuestBook demo application.
12+
The walkthrough presented here is based on the v0.4.x series of the Kubernetes-Mesos project, which itself is based on Kubernetes v0.11.0.
13+
14+
### Prerequisites
15+
16+
* Understanding of [Apache Mesos][11]
17+
* Mesos cluster on [Google Compute Engine][5]
18+
* Identify the Mesos master node external IP from Mesosphere [cluster launch pad][12]
19+
* A [VPN connection to the cluster][6].
20+
21+
### Deploy Kubernetes-Mesos
22+
23+
Log into the master node over SSH, replacing the placeholder below with the correct IP address.
24+
25+
```bash
26+
ssh jclouds@${ip_address_of_master_node}
27+
```
28+
29+
Build Kubernetes-Mesos.
30+
31+
```bash
32+
$ git clone https://github.com/mesosphere/kubernetes-mesos k8sm
33+
$ mkdir -p bin && sudo docker run --rm -v $(pwd)/bin:/target \
34+
-v $(pwd)/k8sm:/snapshot -e GIT_BRANCH=release-0.4 \
35+
mesosphere/kubernetes-mesos:build
36+
```
37+
38+
Set some environment variables.
39+
The internal IP address of the master is visible via the cluster details page on the Mesosphere launchpad, or may be obtained via `hostname -i`.
40+
41+
```bash
42+
$ export servicehost=$(hostname -i)
43+
$ export mesos_master=${servicehost}:5050
44+
$ export KUBERNETES_MASTER=http://${servicehost}:8888
45+
```
46+
### Deploy etcd
47+
Start etcd and verify that it is running:
48+
49+
```bash
50+
$ sudo docker run -d --hostname $(hostname -f) --name etcd -p 4001:4001 -p 7001:7001 coreos/etcd
51+
```
52+
53+
```bash
54+
$ sudo docker ps
55+
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
56+
fd7bac9e2301 coreos/etcd:latest "/etcd" 5s ago Up 3s 2379/tcp, 2380/... etcd
57+
```
58+
It's also a good idea to ensure your etcd instance is reachable by testing it
59+
```bash
60+
curl -L http://$servicehost:4001/v2/keys/
61+
```
62+
If connectivity is OK, you will see an output of the available keys in etcd (if any).
63+
64+
### Start Kubernetes-Mesos Services
65+
Start the kubernetes-mesos API server, controller manager, and scheduler on a Mesos master node:
66+
67+
```bash
68+
$ ./bin/km apiserver \
69+
--address=${servicehost} \
70+
--mesos_master=${mesos_master} \
71+
--etcd_servers=http://${servicehost}:4001 \
72+
--portal_net=10.10.10.0/24 \
73+
--port=8888 \
74+
--cloud_provider=mesos \
75+
--v=1 >apiserver.log 2>&1 &
76+
77+
$ ./bin/km controller-manager \
78+
--master=$servicehost:8888 \
79+
--mesos_master=${mesos_master} \
80+
--v=1 >controller.log 2>&1 &
81+
82+
$ ./bin/km scheduler \
83+
--address=${servicehost} \
84+
--mesos_master=${mesos_master} \
85+
--etcd_servers=http://${servicehost}:4001 \
86+
--mesos_user=root \
87+
--api_servers=$servicehost:8888 \
88+
--v=2 >scheduler.log 2>&1 &
89+
```
90+
91+
Kubernetes-mesos will start up kubelets automatically, but currently the service
92+
proxy needs to be started manually. Start the service proxy on each Mesos slave:
93+
94+
```bash
95+
$ sudo ./bin/km proxy \
96+
--bind_address=${servicehost} \
97+
--etcd_servers=http://${servicehost}:4001 \
98+
--logtostderr=true >proxy.log 2>&1 &
99+
```
100+
101+
Disown your background jobs so that they'll stay running if you log out.
102+
103+
```bash
104+
$ disown -a
105+
```
106+
#### Validate KM Services
107+
Interact with the kubernetes-mesos framework via `kubectl`:
108+
109+
```bash
110+
$ bin/kubectl get pods
111+
POD IP CONTAINER(S) IMAGE(S) HOST LABELS STATUS
112+
```
113+
114+
```bash
115+
$ bin/kubectl get services # your service IPs will likely differ
116+
NAME LABELS SELECTOR IP PORT
117+
kubernetes component=apiserver,provider=kubernetes <none> 10.10.10.2 443
118+
kubernetes-ro component=apiserver,provider=kubernetes <none> 10.10.10.1 80
119+
```
120+
Lastly, use the Mesos CLI tool to validate the Kubernetes scheduler framework has been registered and running:
121+
```bash
122+
$ mesos state | grep "Kubernetes"
123+
"name": "Kubernetes",
124+
```
125+
Or, open the Mesos web GUI, go to the Frameworks tab, and look for an active
126+
framework named "Kubernetes".
127+
128+
## Spin up a pod
129+
130+
Write a JSON pod description to a local file:
131+
132+
```bash
133+
$ cat <<EOPOD >nginx.json
134+
{ "kind": "Pod",
135+
"apiVersion": "v1beta1",
136+
"id": "nginx-id-01",
137+
"desiredState": {
138+
"manifest": {
139+
"version": "v1beta1",
140+
"containers": [{
141+
"name": "nginx-01",
142+
"image": "dockerfile/nginx",
143+
"ports": [{
144+
"containerPort": 80,
145+
"hostPort": 31000
146+
}],
147+
"livenessProbe": {
148+
"enabled": true,
149+
"type": "http",
150+
"initialDelaySeconds": 30,
151+
"httpGet": {
152+
"path": "/index.html",
153+
"port": "8081"
154+
}
155+
}
156+
}]
157+
}
158+
},
159+
"labels": {
160+
"name": "foo"
161+
} }
162+
EOPOD
163+
```
164+
165+
Send the pod description to Kubernetes using the `kubectl` CLI:
166+
167+
```bash
168+
$ bin/kubectl create -f nginx.json
169+
nginx-id-01
170+
```
171+
172+
Wait a minute or two while `dockerd` downloads the image layers from the internet.
173+
We can use the `kubectl` interface to monitor the status of our pod:
174+
175+
```bash
176+
$ bin/kubectl get pods
177+
POD IP CONTAINER(S) IMAGE(S) HOST LABELS STATUS
178+
nginx-id-01 172.17.5.27 nginx-01 dockerfile/nginx 10.72.72.178/10.72.72.178 cluster=gce,name=foo Running
179+
```
180+
181+
Verify that the pod task is running in the Mesos web GUI. Click on the
182+
Kubernetes framework. The next screen should show the running Mesos task that
183+
started the Kubernetes pod.
184+
185+
## Run the Example Guestbook App
186+
187+
Following the instructions from the kubernetes-mesos [examples/guestbook][7]:
188+
189+
```bash
190+
$ export ex=k8sm/examples/guestbook
191+
$ bin/kubectl create -f $ex/redis-master.json
192+
$ bin/kubectl create -f $ex/redis-master-service.json
193+
$ bin/kubectl create -f $ex/redis-slave-controller.json
194+
$ bin/kubectl create -f $ex/redis-slave-service.json
195+
$ bin/kubectl create -f $ex/frontend-controller.json
196+
197+
$ cat <<EOS >/tmp/frontend-service
198+
{
199+
"id": "frontend",
200+
"kind": "Service",
201+
"apiVersion": "v1beta1",
202+
"port": 9998,
203+
"selector": {
204+
"name": "frontend"
205+
},
206+
"publicIPs": [
207+
"${servicehost}"
208+
]
209+
}
210+
EOS
211+
$ bin/kubectl create -f /tmp/frontend-service
212+
```
213+
214+
Watch your pods transition from `Pending` to `Running`:
215+
216+
```bash
217+
$ watch 'bin/kubectl get pods'
218+
```
219+
220+
Review your Mesos cluster's tasks:
221+
222+
```bash
223+
$ mesos ps
224+
TIME STATE RSS CPU %MEM COMMAND USER ID
225+
0:00:05 R 41.25 MB 0.5 64.45 none root 0597e78b-d826-11e4-9162-42010acb46e2
226+
0:00:08 R 41.58 MB 0.5 64.97 none root 0595b321-d826-11e4-9162-42010acb46e2
227+
0:00:10 R 41.93 MB 0.75 65.51 none root ff8fff87-d825-11e4-9162-42010acb46e2
228+
0:00:10 R 41.93 MB 0.75 65.51 none root 0597fa32-d826-11e4-9162-42010acb46e2
229+
0:00:05 R 41.25 MB 0.5 64.45 none root ff8e01f9-d825-11e4-9162-42010acb46e2
230+
0:00:10 R 41.93 MB 0.75 65.51 none root fa1da063-d825-11e4-9162-42010acb46e2
231+
0:00:08 R 41.58 MB 0.5 64.97 none root b9b2e0b2-d825-11e4-9162-42010acb46e2
232+
```
233+
The number of Kubernetes pods listed earlier (from `bin/kubectl get pods`) should equal to the number active Mesos tasks listed the previous listing (`mesos ps`).
234+
235+
Next, determine the internal IP address of the front end [service portal][8]:
236+
237+
```bash
238+
$ bin/kubectl get services
239+
NAME LABELS SELECTOR IP PORT
240+
kubernetes component=apiserver,provider=kubernetes <none> 10.10.10.2 443
241+
kubernetes-ro component=apiserver,provider=kubernetes <none> 10.10.10.1 80
242+
redismaster <none> name=redis-master 10.10.10.49 10000
243+
redisslave name=redisslave name=redisslave 10.10.10.109 10001
244+
frontend <none> name=frontend 10.10.10.149 9998
245+
```
246+
247+
Interact with the frontend application via curl using the front-end service IP address from above:
248+
249+
```bash
250+
$ curl http://${frontend_service_ip_address}:9998/index.php?cmd=get\&key=messages
251+
{"data": ""}
252+
```
253+
254+
Or via the Redis CLI:
255+
256+
```bash
257+
$ sudo apt-get install redis-tools
258+
$ redis-cli -h ${redis_master_service_ip_address} -p 10000
259+
10.233.254.108:10000> dump messages
260+
"\x00\x06,world\x06\x00\xc9\x82\x8eHj\xe5\xd1\x12"
261+
```
262+
#### Test Guestbook App
263+
Or interact with the frontend application via your browser, in 2 steps:
264+
265+
First, open the firewall on the master machine.
266+
267+
```bash
268+
# determine the internal port for the frontend service portal
269+
$ sudo iptables-save|grep -e frontend # -- port 36336 in this case
270+
-A KUBE-PORTALS-CONTAINER -d 10.10.10.149/32 -p tcp -m comment --comment frontend -m tcp --dport 9998 -j DNAT --to-destination 10.22.183.23:36336
271+
-A KUBE-PORTALS-CONTAINER -d 10.22.183.23/32 -p tcp -m comment --comment frontend -m tcp --dport 9998 -j DNAT --to-destination 10.22.183.23:36336
272+
-A KUBE-PORTALS-HOST -d 10.10.10.149/32 -p tcp -m comment --comment frontend -m tcp --dport 9998 -j DNAT --to-destination 10.22.183.23:36336
273+
-A KUBE-PORTALS-HOST -d 10.22.183.23/32 -p tcp -m comment --comment frontend -m tcp --dport 9998 -j DNAT --to-destination 10.22.183.23:36336
274+
275+
# open up access to the internal port for the frontend service portal
276+
$ sudo iptables -A INPUT -i eth0 -p tcp -m state --state NEW,ESTABLISHED -m tcp \
277+
--dport ${internal_frontend_service_port} -j ACCEPT
278+
```
279+
280+
Next, add a firewall rule in the Google Cloud Platform Console. Choose Compute >
281+
Compute Engine > Networks, click on the name of your mesosphere-* network, then
282+
click "New firewall rule" and allow access to TCP port 9998.
283+
284+
![Google Cloud Platform firewall configuration][9]
285+
286+
Now, you can visit the guestbook in your browser!
287+
288+
![Kubernetes Guestbook app running on Mesos][10]
289+
290+
[1]: http://mesosphere.com/docs/tutorials/run-hadoop-on-mesos-using-installer
291+
[2]: http://mesosphere.com/docs/tutorials/run-spark-on-mesos
292+
[3]: http://mesosphere.com/docs/tutorials/run-chronos-on-mesos
293+
[4]: http://cloud.google.com
294+
[5]: https://google.mesosphere.com
295+
[6]: http://mesosphere.com/docs/getting-started/cloud/google/mesosphere/#vpn-setup
296+
[7]: https://github.com/mesosphere/kubernetes-mesos/tree/v0.4.0/examples/guestbook
297+
[8]: https://github.com/GoogleCloudPlatform/kubernetes/blob/v0.11.0/docs/services.md#ips-and-portals
298+
[9]: mesos/k8s-firewall.png
299+
[10]: mesos/k8s-guestbook.png
300+
[11]: http://mesos.apache.org/
301+
[12]: https://google.mesosphere.com/clusters

‎docs/getting-started-guides/mesos/k8s-firewall.png

86.6 KB
Loading

‎docs/getting-started-guides/mesos/k8s-guestbook.png

43 KB
Loading

0 commit comments

Comments
 (0)