Skip to content

Commit

Permalink
Merge pull request #40 from sparkfabrik/feature/2586_prometheus_stack…
Browse files Browse the repository at this point in the history
…_improvements

refs platform/#2586: kube-prometheus-stack improvements
  • Loading branch information
Monska85 authored Nov 30, 2023
2 parents da6f655 + a481000 commit 8818b96
Show file tree
Hide file tree
Showing 5 changed files with 82 additions and 71 deletions.
21 changes: 21 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,23 +7,44 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [2.0.0] - 2023-11-30

[Compare with previous version](https://github.com/sparkfabrik/terraform-aws-eks-bootstrap/compare/1.2.0...2.0.0)

### ⚠️ Breaking changes ⚠️

**ATTENTION:** before applying these changes you must follow the [upgrading instructions](https://github.com/sparkfabrik/terraform-sparkfabrik-prometheus-stack#upgrading-from-2xy-to-300) for the Prometheus Stack module.

### Changed

- refs platform/#2586: update Prometheus Stack module to version `3.0.0` to support multiple values configuration for the Kube Prometheus Stack.

## [1.2.0] - 2023-11-30

[Compare with previous version](https://github.com/sparkfabrik/terraform-aws-eks-bootstrap/compare/1.1.1...1.2.0)

### Added

- refs platform/#2586: add `enhanced_container_insights_enabled` variable to enable/disable enhanced container insights for CloudWatch. Remember that this feature only allows to use **the last 3 hours of collected metrics**. You can find more information about limitations [here](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch-metrics-insights-limits.html).

## [1.1.1] - 2023-11-29

[Compare with previous version](https://github.com/sparkfabrik/terraform-aws-eks-bootstrap/compare/1.1.0...1.1.1)

### Changed

- refs platform/#2586: fix output for `grafana_admin_password` when `enable_kube_prometheus_stack` is `false`.

## [1.1.0] - 2023-11-22

[Compare with previous version](https://github.com/sparkfabrik/terraform-aws-eks-bootstrap/compare/1.0.0...1.1.0)

### Added

- refs #000: add link for CloudWatch Observability EKS addon.

### Changed

- refs platform/#2560: remove local module cluster access and use the one from [GitHub](https://github.com/sparkfabrik/terraform-kubernetes-cluster-access).

## [1.0.0] - 2023-11-22
Expand Down
9 changes: 5 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# terraform-aws-eks-bootstrap

# Terraform aws eks bootstrap

Bootstrap module for AWS EKS cluster.

<!-- BEGIN_TF_DOCS -->
## Providers
Expand Down Expand Up @@ -76,6 +76,7 @@
| <a name="input_metric_server_helm_config"></a> [metric\_server\_helm\_config](#input\_metric\_server\_helm\_config) | Metric Server Helm Chart Configuration | `any` | `{}` | no |
| <a name="input_private_subnet_ids"></a> [private\_subnet\_ids](#input\_private\_subnet\_ids) | n/a | `list(string)` | n/a | yes |
| <a name="input_project"></a> [project](#input\_project) | Project name | `string` | n/a | yes |
| <a name="input_prometheus_stack_additional_values"></a> [prometheus\_stack\_additional\_values](#input\_prometheus\_stack\_additional\_values) | Additional values for Kube Prometheus Stack | `list(string)` | `[]` | no |
| <a name="input_velero_bucket_expiration_days"></a> [velero\_bucket\_expiration\_days](#input\_velero\_bucket\_expiration\_days) | n/a | `number` | `90` | no |
| <a name="input_velero_bucket_glacier_days"></a> [velero\_bucket\_glacier\_days](#input\_velero\_bucket\_glacier\_days) | n/a | `number` | `60` | no |
| <a name="input_velero_bucket_infrequently_access_days"></a> [velero\_bucket\_infrequently\_access\_days](#input\_velero\_bucket\_infrequently\_access\_days) | n/a | `number` | `30` | no |
Expand Down Expand Up @@ -143,9 +144,9 @@
| <a name="module_eks"></a> [eks](#module\_eks) | terraform-aws-modules/eks/aws | ~> 19.13 |
| <a name="module_firestarter_operations"></a> [firestarter\_operations](#module\_firestarter\_operations) | ./modules/firestarter-operations | n/a |
| <a name="module_gitlab_runner"></a> [gitlab\_runner](#module\_gitlab\_runner) | github.com/sparkfabrik/terraform-aws-eks-gitlab-runner | 4e020f8 |
| <a name="module_kube_prometheus_stack"></a> [kube\_prometheus\_stack](#module\_kube\_prometheus\_stack) | github.com/sparkfabrik/terraform-sparkfabrik-prometheus-stack | cd54564 |
| <a name="module_kube_prometheus_stack"></a> [kube\_prometheus\_stack](#module\_kube\_prometheus\_stack) | github.com/sparkfabrik/terraform-sparkfabrik-prometheus-stack | 3.0.0 |
| <a name="module_load_balancer_controller_irsa_role"></a> [load\_balancer\_controller\_irsa\_role](#module\_load\_balancer\_controller\_irsa\_role) | terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks | ~> 5.17 |
| <a name="module_node_termination_handler_irsa_role"></a> [node\_termination\_handler\_irsa\_role](#module\_node\_termination\_handler\_irsa\_role) | terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks | ~> 5.17 |
| <a name="module_velero_irsa_role"></a> [velero\_irsa\_role](#module\_velero\_irsa\_role) | terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks | ~> 5.20 |

<!-- END_TF_DOCS -->
<!-- END_TF_DOCS -->
28 changes: 0 additions & 28 deletions files/kube-prometheus-stack/values.yaml
Original file line number Diff line number Diff line change
@@ -1,27 +1,8 @@
alertmanager:
alertmanagerSpec:
affinity: &prometheusAffinity
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: dedicated-for
operator: In
values:
- kubePrometheus
tolerations: &prometheusTolerations
- key: "dedicated-for"
operator: "Equal"
value: "kubePrometheus"
effect: "NoSchedule"

prometheusOperator:
resources:
requests:
cpu: 50m
memory: 100Mi
affinity: *prometheusAffinity
tolerations: *prometheusTolerations

prometheus:
prometheusSpec:
Expand All @@ -42,12 +23,8 @@ prometheus:
resources:
requests:
storage: 50Gi
affinity: *prometheusAffinity
tolerations: *prometheusTolerations

prometheus-node-exporter:
tolerations: *prometheusTolerations
affinity: *prometheusAffinity
resources:
requests:
cpu: 3m
Expand All @@ -58,14 +35,9 @@ grafana:
requests:
cpu: 50m
memory: 100Mi
tolerations: *prometheusTolerations
affinity: *prometheusAffinity

kube-state-metrics:
resources:
requests:
cpu: 2m
memory: 20Mi
affinity: *prometheusAffinity
tolerations: *prometheusTolerations

89 changes: 50 additions & 39 deletions prometheus.tf
Original file line number Diff line number Diff line change
@@ -1,62 +1,73 @@
# Kube prometheus stack
# https://github.com/sparkfabrik/terraform-sparkfabrik-prometheus-stack
# https://github.com/prometheus-community/helm-charts/tree/kube-prometheus-stack-54.2.2/charts/kube-prometheus-stack
locals {
# https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack
kube_prometheus_namespace = "prometheus"
kube_prometheus_stack_chart_version = "46.4.0"
storage_class_name = "${local.kube_prometheus_namespace}-sc"
}

module "kube_prometheus_stack" {
count = var.enable_kube_prometheus_stack ? 1 : 0
kube_prometheus_stack_chart_version = "54.2.2"
storage_class_name = "prometheus-sc"

source = "github.com/sparkfabrik/terraform-sparkfabrik-prometheus-stack?ref=cd54564"

prometheus_stack_chart_version = local.kube_prometheus_stack_chart_version
create_namespace = true
namespace = local.kube_prometheus_namespace
grafana_ingress_host = var.kube_prometheus_grafana_hostname
grafana_ingress_basic_auth_username = "admin"
regcred = ""
grafana_cluster_issuer_name = local.cert_manager_cluster_issuer_name
grafana_ingress_basic_auth_message = "Restricted Access"

prometheus_stack_additional_values = templatefile(
"${path.module}/files/kube-prometheus-stack/values.yaml",
{
storage_class_name = local.storage_class_name
grafana_hostname = var.kube_prometheus_grafana_hostname
}
prometheus_stack_additional_values = concat(
[
templatefile(
"${path.module}/files/kube-prometheus-stack/values.yaml",
{
storage_class_name = local.storage_class_name
grafana_hostname = var.kube_prometheus_grafana_hostname
}
)
],
var.prometheus_stack_additional_values
)

depends_on = [ kubernetes_manifest.ebs_storageclass ]
}

resource "kubernetes_manifest" "ebs_storageclass" {
count = var.enable_kube_prometheus_stack ? 1 : 0

manifest = {
"apiVersion" = "storage.k8s.io/v1"
"kind" = "StorageClass"
"metadata" = {
"name" = local.storage_class_name
}
"parameters" = {
"csi.storage.k8s.io/fstype" = "xfs"
"encrypted" = "false"
"type" = "gp3"
}
"provisioner" = "ebs.csi.aws.com"
"volumeBindingMode" = "WaitForFirstConsumer"

# The allowedTopologies restrict the node topologies where volumes can be dynamically provisioned.
# This configuration is useful to be sure that the volumes are created in the same AZ of the nodes
# dedicated to the monitoring stack.
"allowedTopologies" = [
{
"matchLabelExpressions" = [
{
"key" = "topology.ebs.csi.aws.com/zone"
"key" = "topology.ebs.csi.aws.com/zone"
"values" = var.kube_prometheus_storage_zone
},
]
},
]
"apiVersion" = "storage.k8s.io/v1"
"kind" = "StorageClass"
"metadata" = {
"name" = local.storage_class_name
}
"parameters" = {
"csi.storage.k8s.io/fstype" = "xfs"
"encrypted" = "false"
"type" = "gp3"
}
"provisioner" = "ebs.csi.aws.com"
"volumeBindingMode" = "WaitForFirstConsumer"
}
}

module "kube_prometheus_stack" {
count = var.enable_kube_prometheus_stack ? 1 : 0

source = "github.com/sparkfabrik/terraform-sparkfabrik-prometheus-stack?ref=3.0.0"

prometheus_stack_chart_version = local.kube_prometheus_stack_chart_version
namespace = local.kube_prometheus_namespace
create_namespace = true
grafana_ingress_host = var.kube_prometheus_grafana_hostname
grafana_ingress_basic_auth_username = "admin"
regcred = ""
grafana_cluster_issuer_name = local.cert_manager_cluster_issuer_name
grafana_ingress_basic_auth_message = "Restricted Access"

prometheus_stack_additional_values = local.prometheus_stack_additional_values

depends_on = [kubernetes_manifest.ebs_storageclass]
}
6 changes: 6 additions & 0 deletions variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -330,6 +330,12 @@ variable "enable_kube_prometheus_stack" {
description = "Enable Kube Prometheus Stack"
}

variable "prometheus_stack_additional_values" {
type = list(string)
description = "Additional values for Kube Prometheus Stack"
default = []
}

variable "kube_prometheus_storage_zone" {
type = list(string)
default = []
Expand Down

0 comments on commit 8818b96

Please sign in to comment.