Skip to content

Commit

Permalink
Updated docs to reference Microsoft/SDN GitHub docs
Browse files Browse the repository at this point in the history
  • Loading branch information
Jason Messer committed Nov 3, 2017
1 parent 703c5f8 commit 6a6922c
Showing 1 changed file with 12 additions and 85 deletions.
97 changes: 12 additions & 85 deletions docs/getting-started-guides/windows/index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Kubernetes version 1.5 introduced support for Windows Server Containers based on

**Note:** Windows Server Containers on Kubernetes is an Alpha feature in Kubernetes 1.8.

**Note:** There is one outstanding PR ([51063 Fixes to enable Windows CNI](https://github.com/kubernetes/kubernetes/pull/51063))which has not been merged into v1.8 and is required for Windows CNI to work with kubelet. Users will need to build a private kubelet binary to consume this change. Please refer to instructions here [TODO - George's guide]
**Note:** There is one outstanding PR ([51063 Fixes to enable Windows CNI](https://github.com/kubernetes/kubernetes/pull/51063))which has not been merged into v1.8 and is required for Windows CNI to work with kubelet. Users will need to build a private kubelet binary to consume this change. Please refer to these [instructions](https://github.com/Microsoft/SDN/blob/master/Kubernetes/HOWTO-on-prem.md) for build

## Prerequisites
In Kubernetes version 1.8, Windows Server Containers for Kubernetes is supported using the following:
Expand All @@ -30,6 +30,9 @@ There are several supported network configurations with Windows Server version 1
4. [Future] Overlay - VXLAN or IP-in-IP encapsulation using Flannel
5. [Future] Layer-3 Routing with BGP (Calico)

## CNI Plugins
Microsoft plans to publish code for two CNI plugins - win-l2bridge (host-gateway) and win-overlay (vxlan)) - per this [issue](https://github.com/containernetworking/plugins/issues/80). These two CNI plugins can either be used directly by WinCNI.exe or with Flannel [PR 832](https://github.com/coreos/flannel/pull/832). We have an [outstanding informational PR](https://github.com/containernetworking/plugins/pull/85) needed to complete this work. Windows Server platform work is complete.

The selection of which network configuration and topology to deploy depends on the physical network topolgy and a user's ability to configure routes, performance concerns with encapsulation, and requirement to integrate with third-party network plugins.

### Linux
Expand All @@ -49,14 +52,10 @@ The following diagram illustrates the Windows Server networking setup for Kubern
#### Host-Gateway Topology
This topology is similar to the Upstream L3 Routing topology with the only difference being that static IP routes are configured directly on each cluster node and not in the upstream ToR. Each node uses a local 'l2bridge' network with a Pod CIDR assigned as before and has routing table entries for all other Pod CIDR subnets assigned to the remote cluster nodes.

TODO: Add diagram

#### Overlay using OVN controller and OVS Switch Extension

TODO



## Setting up Windows Server Containers on Kubernetes
To run Windows Server Containers on Kubernetes, you'll need to set up both your host machines and the Kubernetes node components for Windows and depending on your network topology, setup Routes for Pod communication on different nodes.

Expand All @@ -65,22 +64,24 @@ To run Windows Server Containers on Kubernetes, you'll need to set up both your
**Linux Host Setup**

1. Linux hosts should be setup according to their respective distro documentation and the requirements of the Kubernetes version you will be using.
2. Configure Linux Master node using steps here [TODO - Add link to George's page]
2. Configure Linux Master node using steps [here](https://github.com/Microsoft/SDN/blob/master/Kubernetes/HOWTO-on-prem.md)
3. [Optional] CNI network plugin installed.

**Windows Host Setup**

1. Windows Server container host running Windows Server version 1709 and Docker v17.06 or later. Follow the setup instructions outlined by this help topic: https://docs.microsoft.com/en-us/virtualization/windowscontainers/quick-start/quick-start-windows-server.
2. Build or download kubelet.exe, kube-proxy.exe, and kubectl.exe using instructions found here [TODO - George's guide]
2. Build or download kubelet.exe, kube-proxy.exe, and kubectl.exe using instructions found [here](https://github.com/Microsoft/SDN/blob/master/Kubernetes/HOWTO-on-prem.md)
3. Copy Node spec file (config) from Linux master node with X.509 keys
4. Create HNS Network
5. Ensure correct CNI network config
5. Start kubelet.exe using script [TODO - Add link to George's script]
6. Start kube-proxy using script [TODO - Add link to George's script]
5. Start kubelet.exe using this script [start-kubelet.ps1](https://github.com/Microsoft/SDN/blob/master/Kubernetes/windows/start-kubelet.ps1)
6. Start kube-proxy using this script[start-kubeproxy.ps1](https://github.com/Microsoft/SDN/blob/master/Kubernetes/windows/start-kubeproxy.ps1)
7. [Optional] Add static routes on Windows host

**Windows CNI Config Example**
Today, Windows CNI plugin is based on wincni.exe code with the following example, configuration file.

Note: this file assumes that a user previous created 'l2bridge' host networks on each Windows node using <Verb>-HNSNetwork cmdlets as shown in the start-kublet.ps1 and start-kubeproxy.ps1 scripts linked above
```
{
"cniVersion": "0.2.0",
Expand Down Expand Up @@ -132,82 +133,12 @@ To run Windows Server Containers on Kubernetes, you'll need to set up both your
]
}
```
### Component Setup

!Note: TODO - Add link to George's documentation

Requirements

TODO - George Check
* Git
* Go 1.7.1+
* make (if using Linux or MacOS)
* Important notes and other dependencies are listed [here](https://git.k8s.io/community/contributors/devel/development.md#building-kubernetes-on-a-local-osshell-environment)

!Note: Remove if it makes sense and info is in George's documentation

**kubelet**

To build the *kubelet*, run:

1. `cd $GOPATH/src/k8s.io/kubernetes`
2. Build *kubelet*
1. Linux/MacOS: `KUBE_BUILD_PLATFORMS=windows/amd64 make WHAT=cmd/kubelet`
2. Windows: `go build cmd/kubelet/kubelet.go`

**kube-proxy**

To build *kube-proxy*, run:

1. `cd $GOPATH/src/k8s.io/kubernetes`
2. Build *kube-proxy*
1. Linux/MacOS: `KUBE_BUILD_PLATFORMS=windows/amd64 make WHAT=cmd/kube-proxy`
2. Windows: `go build cmd/kube-proxy/proxy.go`

### Route Setup
The below example setup assumes one Linux and two Windows Server 2016 nodes and a cluster CIDR 192.168.0.0/16

| Hostname | Routable IP address | Pod CIDR |
| --- | --- | --- |
| Lin01 | `<IP of Lin01 host>` | 192.168.0.0/24 |
| Win01 | `<IP of Win01 host>` | 192.168.1.0/24 |
| Win02 | `<IP of Win02 host>` | 192.168.2.0/24 |

**Lin01**

```
ip route add 192.168.1.0/24 via <IP of Win01 host>
ip route add 192.168.2.0/24 via <IP of Win02 host>
```


## Starting the Cluster
To start your cluster, you'll need to start both the Linux-based Kubernetes control plane, and the Windows Server-based Kubernetes node components.
To start your cluster, you'll need to start both the Linux-based Kubernetes control plane, and the Windows Server-based Kubernetes node components (kubelet and kube-proxy).

## Starting the Linux-based Control Plane
Use your preferred method to start Kubernetes cluster on Linux. Please note that Cluster CIDR might need to be updated.

## Starting the Windows Node Components
To start kubelet on your Windows node:
Run the following in a PowerShell window. Be aware that if the node reboots or the process exits, you will have to rerun the commands below to restart the kubelet.

TODO - Either link to George's doc or update in-line below

1. Set environment variable *CONTAINER_NETWORK* value to the docker container network to use
`$env:CONTAINER_NETWORK = "<docker network>"`

2. Run *kubelet* executable using the below command
`kubelet.exe --hostname-override=<ip address/hostname of the windows node> --pod-infra-container-image="apprenda/pause" --resolv-conf="" --api_servers=<api server location>`

To start kube-proxy on your Windows node:

Run the following in a PowerShell window with administrative privileges. Be aware that if the node reboots or the process exits, you will have to rerun the commands below to restart the kube-proxy.

1. Set environment variable *INTERFACE_TO_ADD_SERVICE_IP* value to `vEthernet (KubeProxySwitch)` which we created in **_Windows Host Setup_** above
`$env:INTERFACE_TO_ADD_SERVICE_IP = "vEthernet (KubeProxySwitch)"`

2. Run *kube-proxy* executable using the below command
`.\proxy.exe --v=3 --proxy-mode=userspace --hostname-override=<ip address/hostname of the windows node> --master=<api server location> --bind-address=<ip address of the windows node>`
Use your preferred method to setup and start Kubernetes cluster on Linux or follow the directions given in this [link](https://github.com/Microsoft/SDN/blob/master/Kubernetes/HOWTO-on-prem.md). Please note that Cluster CIDR might need to be updated.

## Scheduling Pods on Windows
Because your cluster has both Linux and Windows nodes, you must explicitly set the nodeSelector constraint to be able to schedule Pods to Windows nodes. You must set nodeSelector with the label beta.kubernetes.io/os to the value windows; see the following example:
Expand Down Expand Up @@ -240,7 +171,3 @@ Because your cluster has both Linux and Windows nodes, you must explicitly set t
}
}
```

## Known Limitations:
1. Secrets currently do not work because of a bug in Windows Server Containers described [here](https://github.com/docker/docker/issues/28401).
2. ConfigMaps have not been implemented yet.

0 comments on commit 6a6922c

Please sign in to comment.