Skip to content

Commit

Permalink
[bot] Merge automation (#219)
Browse files Browse the repository at this point in the history 
* Update from solo-io/procgen@15d7aa1

---------

Co-authored-by: soloio-bot <>
  • Loading branch information
@soloio-bot
soloio-bot authored Feb 2, 2024
1 parent b50a96d commit af44905
Show file tree
Hide file tree
Showing 776 changed files with 105,765 additions and 32,935 deletions.
2 changes: 1 addition & 1 deletion gloo-mesh/core/2-5/default/scripts/deploy-aws-with-calico.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,7 @@ docker network connect "kind" us-central1-docker || true
docker network connect "kind" quay || true
docker network connect "kind" gcr || true

kubectl --context kind-kind${number} apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.25.0/manifests/calico.yaml
kubectl --context kind-kind${number} apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.27.0/manifests/calico.yaml

# Preload MetalLB images
docker pull quay.io/metallb/controller:v0.13.12
Expand Down
2 changes: 1 addition & 1 deletion gloo-mesh/core/2-5/default/scripts/deploy-multi-with-calico.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ docker network connect "kind" us-central1-docker || true
docker network connect "kind" quay || true
docker network connect "kind" gcr || true

kubectl --context kind-kind${number} apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.25.0/manifests/calico.yaml
kubectl --context kind-kind${number} apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.27.0/manifests/calico.yaml

# Preload MetalLB images
docker pull quay.io/metallb/controller:v0.13.12
Expand Down
2 changes: 1 addition & 1 deletion gloo-mesh/core/2-5/default/scripts/deploy-with-calico.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ docker network connect "kind" us-central1-docker || true
docker network connect "kind" quay || true
docker network connect "kind" gcr || true

kubectl --context kind-kind${number} apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.25.0/manifests/calico.yaml
kubectl --context kind-kind${number} apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.27.0/manifests/calico.yaml

# Preload MetalLB images
docker pull quay.io/metallb/controller:v0.13.12
Expand Down
194 changes: 6 additions & 188 deletions gloo-mesh/enterprise/2-4/airgap/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,8 @@ source ./scripts/assert.sh
* [Lab 12 - Create the Root Trust Policy](#lab-12---create-the-root-trust-policy-)
* [Lab 13 - Leverage Virtual Destinations for east west communications](#lab-13---leverage-virtual-destinations-for-east-west-communications-)
* [Lab 14 - Zero trust](#lab-14---zero-trust-)
* [Lab 15 - See how Gloo Platform can help with observability](#lab-15---see-how-gloo-platform-can-help-with-observability-)
* [Lab 16 - Securing the egress traffic](#lab-16---securing-the-egress-traffic-)
* [Lab 17 - VM integration with Spire](#lab-17---vm-integration-with-spire-)
* [Lab 15 - Securing the egress traffic](#lab-15---securing-the-egress-traffic-)
* [Lab 16 - VM integration with Spire](#lab-16---vm-integration-with-spire-)



Expand Down Expand Up @@ -166,8 +165,6 @@ Pull and push locally the Docker images needed:
```bash
cat <<'EOF' > images.txt
docker.io/curlimages/curl
docker.io/bats/bats:v1.4.1
docker.io/grafana/grafana:10.2.3
docker.io/kennethreitz/httpbin
docker.io/nginx:1.25.3
docker.io/openpolicyagent/opa:0.57.1-debug
Expand All @@ -182,15 +179,7 @@ gcr.io/gloo-mesh/gloo-otel-collector:2.4.7
gcr.io/gloo-mesh/rate-limiter:0.10.3
jimmidyson/configmap-reload:v0.8.0
quay.io/keycloak/keycloak:22.0.5
quay.io/kiwigrid/k8s-sidecar:1.25.2
quay.io/prometheus-operator/prometheus-config-reloader:v0.70.0
quay.io/prometheus-operator/prometheus-operator:v0.70.0
quay.io/prometheus/alertmanager:v0.26.0
quay.io/prometheus/node-exporter:v1.7.0
quay.io/prometheus/prometheus:v2.41.0
quay.io/prometheus/prometheus:v2.48.1
registry.k8s.io/ingress-nginx/kube-webhook-certgen:v20221220-controller-v1.5.1-58-g787ea74b6
registry.k8s.io/kube-state-metrics/kube-state-metrics:v2.10.1
us-docker.pkg.dev/gloo-mesh/istio-workshops/install-cni:1.19.3-solo
us-docker.pkg.dev/gloo-mesh/istio-workshops/operator:1.19.3-solo
us-docker.pkg.dev/gloo-mesh/istio-workshops/pilot:1.19.3-solo
Expand Down Expand Up @@ -1905,7 +1894,8 @@ Let's add the domains to our `/etc/hosts` file:
You can access the `productpage` service
using this URL: [http://cluster1-bookinfo.example.com/productpage](http://cluster1-bookinfo.example.com/productpage).

You should now be able to access the `productpage` application through the browser.<!--bash
You should now be able to access the `productpage` application through the browser.
<!--bash
cat <<'EOF' > ./test.js
const helpers = require('./tests/chai-http');
Expand Down Expand Up @@ -3002,179 +2992,7 @@ kubectl --context ${CLUSTER1} delete accesspolicies -n bookinfo-frontends --all



## Lab 15 - See how Gloo Platform can help with observability <a name="lab-15---see-how-gloo-platform-can-help-with-observability-"></a>
[<img src="https://img.youtube.com/vi/UhWsk4YnOy0/maxresdefault.jpg" alt="VIDEO LINK" width="560" height="315"/>](https://youtu.be/UhWsk4YnOy0 "Video Link")

# Observability with Gloo Platform

Let's take a look at how Gloo Platform can help with observability!

![Gloo Platform OTel arch](images/steps/gloo-platform-observability/metrics-architecture-otel.svg)

Our telemetry pipeline's main goal is to collect all the metrics, and securely forward them to the management cluster, making all the metrics available for our UI to visualize the service graph.

Since our pipeline is leveraging OpenTelemetry, this pipeline can be customized and extended to cover all possible use-cases, e.g. collecting telemetry from other workloads, or integrating with centralized observability platform/SaaS solutions.

## Gloo Platform Operational Dashboard

First let's deploy the usual Prometheus stack, and explore our management plane metrics.

```bash
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

helm upgrade --install kube-prometheus-stack \
prometheus-community/kube-prometheus-stack \
--kube-context ${MGMT} \
--version 55.9.0 \
--namespace monitoring \
--create-namespace \
--values - <<EOF
grafana:
service:
type: LoadBalancer
port: 3000
additionalDataSources:
- name: prometheus-GM
uid: prometheus-GM
type: prometheus
url: http://prometheus-server.gloo-mesh:80
grafana.ini:
auth.anonymous:
enabled: true
defaultDashboardsEnabled: false
global:
imageRegistry: ${registry}
EOF
```
<!--bash
cat <<'EOF' > ./test.js
const helpers = require('./tests/chai-exec');
describe("kube-prometheus-stack deployments are ready", () => {
it('kube-prometheus-stack-kube-state-metrics pods are ready', () => helpers.checkDeployment({ context: process.env.MGMT, namespace: "monitoring", k8sObj: "kube-prometheus-stack-kube-state-metrics" }));
it('kube-prometheus-stack-grafana pods are ready', () => helpers.checkDeployment({ context: process.env.MGMT, namespace: "monitoring", k8sObj: "kube-prometheus-stack-grafana" }));
it('kube-prometheus-stack-operator pods are ready', () => helpers.checkDeployment({ context: process.env.MGMT, namespace: "monitoring", k8sObj: "kube-prometheus-stack-operator" }));
});
describe("kube-prometheus-stack daemonset is ready", () => {
it('kube-prometheus-stack-prometheus-node-exporter pods are ready', () => helpers.checkDaemonSet({ context: process.env.MGMT, namespace: "monitoring", k8sObj: "kube-prometheus-stack-prometheus-node-exporter" }));
});
EOF
echo "executing test dist/gloo-mesh-2-0-workshop/build/templates/steps/gloo-platform-observability/tests/grafana-installed.test.js.liquid"
tempfile=$(mktemp)
echo "saving errors in ${tempfile}"
timeout 2m mocha ./test.js --timeout 10000 --retries=120 --bail 2> ${tempfile} || { cat ${tempfile} && exit 1; }
-->
Let's install a few dashboards!

Now, you can go the the Grafana tab, log in with the default login credentials, admin/prom-operator, and import the dashboard of Istio control plane.

Add the Operational Dashboard
=============================

Our Gloo components are all instrumented with Prometheus compatible metrics, providing an easy way to pinpoint a potential degradation.

You can import the following dashboard to see our Operational Dashboard, covering all of our components in the stack.

Here, you have specific rows for each components, such as the management server, the agent, the telemetry collectors, and some additional information regarding resource usage.

```bash
kubectl --context ${MGMT} -n monitoring create cm operational-dashboard \
--from-file=data/steps/gloo-platform-observability/operational-dashboard.json
kubectl --context ${MGMT} label -n monitoring cm operational-dashboard grafana_dashboard=1
```

Out-of-box alerting
===================

Our Prometheus comes with useful alerts by default, making it easier to get notified if something breaks.

All of the default alerts have corresponding panels on the Operational Dashboard.

You can click the "Bell" icon on the left, and choose "Alert rules", and check "GlooPlatformAlerts" to take a closer look at them.

Let's trigger one of the alerts!

If you scale down the Gloo Agent in let's say `cluster1`, you should have an alert called `GlooPlatformAgentsAreDisconnected` go into first PENDING, then FIRING, let's check this!

```sh
kubectl --context $CLUSTER1 scale deployment.apps/gloo-mesh-agent -n gloo-mesh --replicas=0
```

The alert will fire in 5m, but even before that, it will reach PENDING state, let's wait for this!

Don't forget to scale it up after:

```sh
kubectl --context $CLUSTER1 scale deployment.apps/gloo-mesh-agent -n gloo-mesh --replicas=1
```

Collect remote IstioD metrics securely
======================================

Let's take a look how easy it is to modify the metrics collection in the workload clusters, to collect IstioD metrics, and ship them to the management cluster over TLS.

```bash
helm upgrade --install gloo-platform gloo-platform \
--repo https://storage.googleapis.com/gloo-platform/helm-charts \
--namespace gloo-mesh \
--kube-context ${CLUSTER1} \
--reuse-values \
--version 2.4.7 \
--values - <<EOF
telemetryCollectorCustomization:
extraProcessors:
batch/istiod:
send_batch_size: 10000
timeout: 10s
filter/istiod:
metrics:
include:
match_type: regexp
metric_names:
- "pilot.*"
- "process.*"
- "go.*"
- "container.*"
- "envoy.*"
- "galley.*"
- "sidecar.*"
# - "istio_build.*" re-enable this after this is fixed upstream
extraPipelines:
metrics/istiod:
receivers:
- prometheus
processors:
- memory_limiter
- batch/istiod
- filter/istiod
exporters:
- otlp
EOF
```

This configuration update will
- create a new processor, called `filter/istiod`, that will enable all the IstioD/Pilot related metrics
- create a new pipeline, called `metrics/istiod`, that will have the aforementioned processor to include the control plane metrics

Then, we just need to perform a rollout restart for the metrics collector, so the new pods can pick up the config change.

```bash
kubectl --context $CLUSTER1 rollout restart daemonset/gloo-telemetry-collector-agent -n gloo-mesh
```

Now, let's import the Istio Control Plane Dashboard, and see the metrics!

```bash
kubectl --context ${MGMT} -n monitoring create cm istio-control-plane-dashboard \
--from-file=data/steps/gloo-platform-observability/istio-control-plane-dashboard.json
kubectl --context ${MGMT} label -n monitoring cm istio-control-plane-dashboard grafana_dashboard=1
```



## Lab 16 - Securing the egress traffic <a name="lab-16---securing-the-egress-traffic-"></a>
## Lab 15 - Securing the egress traffic <a name="lab-15---securing-the-egress-traffic-"></a>
[<img src="https://img.youtube.com/vi/tQermml1Ryo/maxresdefault.jpg" alt="VIDEO LINK" width="560" height="315"/>](https://youtu.be/tQermml1Ryo "Video Link")


Expand Down Expand Up @@ -3489,7 +3307,7 @@ kubectl --context ${CLUSTER1} -n istio-gateways delete accesspolicy allow-get-ht



## Lab 17 - VM integration with Spire <a name="lab-17---vm-integration-with-spire-"></a>
## Lab 16 - VM integration with Spire <a name="lab-16---vm-integration-with-spire-"></a>

Let's see how we can configure a VM to be part of the Mesh.

Expand Down
Loading

0 comments on commit af44905

Please sign in to comment.