Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[create-doks-with-terraform-flux] Main TF module is not able to resize the DOKS cluster (node count) #26

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions create-doks-with-terraform-flux/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -35,3 +35,6 @@ terraform.rc

.terraform.lock*
.DS_Store

backend.tf
*.out
9 changes: 0 additions & 9 deletions create-doks-with-terraform-flux/data.tf
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,6 @@ data "github_repository" "main" {
}
# ==============================================================

# ========================== DOKS ==============================
data "digitalocean_kubernetes_cluster" "primary" {
name = var.doks_cluster_name
depends_on = [
digitalocean_kubernetes_cluster.primary
]
}
# ===============================================================

# =========================== FLUX CD ===========================
data "flux_install" "main" {
target_path = var.git_repository_sync_path
Expand Down
14 changes: 6 additions & 8 deletions create-doks-with-terraform-flux/provider.tf
Original file line number Diff line number Diff line change
Expand Up @@ -31,22 +31,20 @@ provider "digitalocean" {
token = var.do_api_token
}

provider "flux" {}

provider "kubectl" {
host = data.digitalocean_kubernetes_cluster.primary.endpoint
token = data.digitalocean_kubernetes_cluster.primary.kube_config[0].token
host = digitalocean_kubernetes_cluster.primary.endpoint
token = digitalocean_kubernetes_cluster.primary.kube_config[0].token
cluster_ca_certificate = base64decode(
data.digitalocean_kubernetes_cluster.primary.kube_config[0].cluster_ca_certificate
digitalocean_kubernetes_cluster.primary.kube_config[0].cluster_ca_certificate
)
load_config_file = false
}

provider "kubernetes" {
host = data.digitalocean_kubernetes_cluster.primary.endpoint
token = data.digitalocean_kubernetes_cluster.primary.kube_config[0].token
host = digitalocean_kubernetes_cluster.primary.endpoint
token = digitalocean_kubernetes_cluster.primary.kube_config[0].token
cluster_ca_certificate = base64decode(
data.digitalocean_kubernetes_cluster.primary.kube_config[0].cluster_ca_certificate
digitalocean_kubernetes_cluster.primary.kube_config[0].cluster_ca_certificate
)
}

Expand Down
54 changes: 54 additions & 0 deletions netmaker-egress-gateway/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Setting up an Egress Gateway Using Netmaker

## Overview

What is an `Egress Gateway` and more important, why do you need one?

`Ingress` deals with `inbound` traffic, whereas `Egress` deals with `outbound` traffic. Going further, there are two key aspects to keep in mind:

1. Security implications.
2. Having a well defined model and set of measures for your infrastructure (either cloud based or on-premise), so that each entity from the system cannot compromise the others.

Suppose you have a private network and a bunch of machines or nodes inside the network. Do you really trust any machine in the network doing the right thing? In a sense you do, because you set up all those machines, thus knowing what applications are running on each. But even so, you may have many people working with you and accessing those machines, thus performing various operations. Of course, as a system administrator you would limit access and assign roles (RBAC), so that you have a finer control of who has access and to what resources.

What about network traffic ? In the same way, you should consider limiting or restricting both inbound and outbound traffic. Basically, nothing goes in or out without explicitly being allowed to, via explicit network policies in place. On the long term, imagine what would happen if any machine (or application running on it) is allowed to carry network traffic (in our out) without explicitly being allowed to do so. Soon, things can get out of control. By having a dedicated node (or gateway) in your network to control in or out traffic, things become much easier to manage and observe. Not only that, it also allows you to enforce more strict policies, and watch it more thoroughly by performing audits.

You already have `Firewalls` in place to allow or restrict traffic, which is an important security measure as well. `VPCs` isolate your resources (`Droplets`, `Load Balancers`, etc) between clusters in same or different regions. The `Egress` use case, is more related to how you control and route traffic between VPCs. An `Egress Gateway` is just a `NAT Gateway` in essence.

This guide is about setting up an `Egress Gateway` to control and route traffic between a `Kubernetes` cluster (e.g. `DOKS`) or a `Droplet`, from one `VPC` to another, in same or different regions. Another use case is to allow secure connections to an external service, like a managed database for example. The `Egress Gateway` is usually deployed to a dedicated machine with a `static public IP` (or `floating IP`, in case of `DigitalOcean`), and runs dedicated software. Such an example is [Netmaker](https://github.com/gravitl/netmaker), which you will discover in this tutorial.

## Introducing Netmaker

[Netmaker](https://github.com/gravitl/netmaker) is a tool for creating and managing virtual overlay networks. You can create as many overlay networks as you want, and connect different machines from different data centers transparently. At its heart Netmaker is using [WireGuard(https://www.wireguard.com) to do the magic. Being an overlay network, it doesn't affect your existing network setup, it just sits on top of it offering a lot of flexibility and possibilities.

`Netmaker` allows you to define and create `private networks`, just as `VPCs` are. It's built with `security` in mind as well. The way it works is by creating `secure tunnels` across machines, a feature offered by `WireGuard`. Netmaker follows a client-server model, and consists of two parts:

1. The `admin server`, called `Netmaker`.
2. `Agents` (or `clients`) deployed on each machine (or node), called `Netclients`.

The Netmaker server doesn't deal with network traffic (although it can be told to, if needed). The main role of the server is to keep configuration state, and control or manage user defined networks. Each machine participating in the network, is called a `Node`. For the most part, Netmaker serves `configuration` data to `Nodes`, telling them how they should configure themselves. The `Netclient` is the `agent` that actually does that configuration.

Traffic goes between nodes, peer to peer. On the other hand, each node can relay messages as well and improve network resiliency. Netmaker allows you to define and create `mesh networks`, which is a really powerful feature. Mesh networks add resiliency because the network can heal itself. All nodes are interconnected and contribute to traffic. If one node dies, others will take its place, thus offering transparency.

`Netmaker` also allows you to define `Egress Gateways` in a very simple manner. You just select one node from your private network and an interface for outbound traffic, and then tell Netmaker to configure it as an Egress Gateway (all the hard work is handled by Netmaker).

Being based on `WireGuard` it has `VPN` support, so you can create various configurations, like:

1. Personal (Private Browsing).
2. Remote Access.
3. Site-to-Site.
4. Mesh (virtual LAN/WAN).

`WireGuard` is `fast` and industry proven, which is another real advantage. Below picture shows the Netmaker architecture and how it manages networks:

![Netmaker Architecture](assets/images/netmaker_arch.png)

For more information and details about inner workings and architecture, please visit the official Netmaker [design](https://netmaker.readthedocs.io/en/master/architecture.html) page related to this topic.

Next, you're going to learn how to deploy and configure the main `Netmaker` server, as well as `Netclients` on each machine that needs to be part of your egress setup. For now Netmaker doesn't offer HA (`High Availability`) features, but it will handle that as well in a future release.

## Installing Netmaker Server

Netmaker server can be deployed and installed very quickly and painless via the DigitalOcean [Apps](https://marketplace.digitalocean.com/apps) platform.

TBD.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.