-
Notifications
You must be signed in to change notification settings - Fork 56
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs: port some of the legacy documentation (#987)
Release-As: v1.0.3
- Loading branch information
Showing
13 changed files
with
226 additions
and
0 deletions.
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+55.7 KB
docs/modules/ROOT/assets/images/explanations/grafana_cert_manager.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
= Cloud Agnostic | ||
|
||
Nobody likes to be locked-in. That's why the DevOps Stack provides a cloud abstraction so you can deploy it on top of multiple Cloud Platforms, in an agnostic way. | ||
|
||
Currently, the DevOps Stack supports the following Cloud Platforms, besides a local test deployment with KinD: | ||
|
||
- Amazon Web Services (AWS) | ||
- Azure | ||
- Exoscale | ||
- Scaleway | ||
Good "agnostic" automation practices offer many benefits. |
17 changes: 17 additions & 0 deletions
17
docs/modules/ROOT/pages/explanations/continuous_deployment.adoc
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
= Continuous Delivery & GitOps | ||
|
||
Change management is at the heart of Infrastructure Operations. The rise of Continuous Integration and Continuous Delivery practices in the Infrastructure world has been so important that the expression DevOps Engineer is often used to describe CI/CD Engineering. | ||
|
||
The DevOps Stack takes advantage of the CI/CD approach to make changes seamless, both at the provisioning and deployment levels. | ||
|
||
== Terraform | ||
|
||
The underlying infrastructure which sets up the Kubernetes cluster underneath the DevOps Stack is managed with Terraform. | ||
|
||
It is strongly encouraged to use a CI/CD approach to manage this Terraform deployment, and the DevOps Stack provides a set of scripts to integrate pair-reviewed change process on your CI/CD platform of choice. | ||
|
||
== Argo CD | ||
|
||
On top of Kubernetes, the DevOps Stack sets up Argo CD to continuously deploy all applications to the cluster. | ||
|
||
Argo CD monitors Kubernetes resources and can keep them synchronized with their target state. |
9 changes: 9 additions & 0 deletions
9
docs/modules/ROOT/pages/explanations/infrastructure_as_code.adoc
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
= Infrastructure as Code | ||
|
||
Deploying Kubernetes clusters from scratch has become an easy task with IaaS interfaces. However, the proper management of all components that make up the cluster and its ecosystem can quickly become a nightmare if everything was done manually. | ||
|
||
The Infrastructure as Code approach has long proved its worth to deploy and maintain complex environments on the long run. In particular, in the context of cloud resources, it allows to make sure all resources are managed centrally and destroyed in their proper context. This is important to control the financial consequences of numerous deployments. | ||
|
||
The DevOps Stack is strongly founded on the idea that every one of its components should be managed as code. *As such, the underlying Kubernetes cluster and its components are managed with Terraform and the cluster applications are deployed using Helm charts orchestrated by Argo CD.* | ||
|
||
This allows you to keep track of all changes made at any level of the stack, using xref:ROOT:explanations/continuous_deployment.adoc[Continuous Deployment] in a GitOps approach. |
57 changes: 57 additions & 0 deletions
57
docs/modules/ROOT/pages/explanations/lifecycle_operations.adoc
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,57 @@ | ||
= Lifecycle Operations | ||
|
||
It is not easy to describe the different phases of an IT project, especially when you also want to talk about operational aspects. One way to do this, which seems to be used more and more, is to talk about "days". | ||
|
||
== What "days" are all about | ||
|
||
To make it very simple, here are some explanations: | ||
|
||
- *Day 0* is related to software development, questions about requirements, architecture and design. | ||
- *Day 1* is more about deploying software developed in the *Day 0* and starting to build and configure the hosting infrastructure. | ||
- *Day 2* is the time to go live and hope that the efforts made in *Day 1* are adequate. This is a crucial moment, when it is imperative to have metrics, monitoring, tools and best practices to do efficient troubleshooting. | ||
|
||
For now, the DevOps Stack is very focused on the points relating to days 1 and 2, in particular, the aspects related to the provisioning of a Kubernetes platform as well as the mandatory services related to its operational management. | ||
|
||
We must also take into account that using the Cloud and Cloud Native development practices make it much easier to handle changes related to software life cycle. The boundaries and responsibilities between these different phases have become more blurred, less strict. | ||
|
||
== Developers need love | ||
|
||
In the near future, we really want to invest significantly in aspects related to day 0 by improving the DevOps Stack in that direction, especially the software factory and development best practices related to Kubernetes integration. | ||
|
||
This is closely linked to the generalization of the https://12factor.net/[Twelve-Factor App] methodology, as well as breaking silos and embracing DevOps principles. These good practices should be applied as early as possible in projects. It is absolutely necessary. | ||
|
||
== Sysadmins want control | ||
|
||
The DevOps Stack provisions our Kubernetes clusters with a set of administration and monitoring tools out-of-the-box. Below you will find some of the services' dashboards. | ||
|
||
=== Identity Management Dashboard (Keycloak) | ||
|
||
Whether you use AKS, EKS, or your local workstation using KinD, the DevOps Stack provisions the cluster with an Identity and Access Manager. | ||
|
||
For the case of a local KinD cluster, the DevOps Stack spawns a Keycloak instance. | ||
|
||
==== User management with Keycloak | ||
image::explanations/keycloak_users.png[Dashboard for user management] | ||
|
||
==== Client management with Keycloak | ||
image::explanations/keycloak_clients.png[Dashboard for client management] | ||
|
||
=== Continuous Deployment Dashboard (ArgoCD) | ||
|
||
In a xref:ROOT:explanations/continuous_deployment.adoc[GitOps approach], the DevOps Stack uses Argo CD to continuously deploy all applications. | ||
|
||
==== Identity and Access Manager (Keycloak) deployment with Argo CD | ||
|
||
image::explanations/argo_keycloak.png[Keycloak deployment with Argo CD] | ||
|
||
=== Monitoring Dashboards (Grafana) | ||
|
||
The DevOps Stack ships with some pre-configured dashboards. | ||
|
||
==== Grafana dashboard for monitoring the K8s cluster | ||
|
||
image::explanations/grafana_k8s_cluster.png[Dashboard for monitoring the K8s cluster] | ||
|
||
==== Grafana dashboard for monitoring cert-manager | ||
|
||
image::explanations/grafana_cert_manager.png[Dashboard for monitoring cert-manager] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,125 @@ | ||
= Purpose of the DevOps Stack | ||
|
||
|
||
== Introduction | ||
|
||
Since about 2006, the IT industry has been totally disrupted by the | ||
conjunction of several factors, including the virtualization of | ||
resources and the possibility of obtaining them on-demand. | ||
This new way of managing IT resources is widely known as "Cloud Computing". | ||
It has become a kind of standard, or at least an objective | ||
pursued also by people wishing to maintain on-premises infrastructures. | ||
The long-awaited dream of finally being able to consider IT as a simple | ||
commodity. Well that's what many believed. | ||
|
||
The concrete consequences of this revolution for IT companies like | ||
Camptocamp have mainly been the incredible growth in the number of | ||
resources to be provisioned and managed. Extensive process automation | ||
engaged by our teams allowed us to cope with such a growth without lowering | ||
our quality standards, therefore remaining competitive. | ||
|
||
We had no respite in this automation journey and made the best out of a thriving | ||
ecosystem of open source projects that address all the needs related | ||
to this paradigm shift. | ||
We have learned a lot during all these years and the | ||
DevOps Stack is our answer to implement our best practices in a | ||
containerized world where Kubernetes is the _de facto_ standard for | ||
application platforms. | ||
|
||
|
||
== Cloud Agnostic Kubernetes Distribution | ||
|
||
Nowadays, the question is not so much whether there is software to solve | ||
a problem, but which one to use. Like a Linux distribution, we have | ||
container orchestrators (Kubernetes as a kernel) and many other projects | ||
revolving around it and offering different features. These features are | ||
sometimes oriented towards developers, operators or both (e.g. observability | ||
tools). | ||
|
||
As for a Linux distribution, what we need is consistency, components | ||
that work together and that are easy to install and maintain. But unlike a | ||
Linux distribution which is by definition server-centric, the promise of | ||
containers and Kubernetes is portability. The possibility of moving | ||
from one cloud (or on-prem infrastructure) to another, of distributing | ||
the workloads on different platforms. At the end, this DevOps Stack is | ||
simply a "Cloud Agnostic Kubernetes Distribution". | ||
|
||
|
||
== Without the DevOps Stack | ||
|
||
image::explanations/without_devops-stack.png[Without the DevOps Stack] | ||
|
||
The illustration above tries to show that while it is quite | ||
simple to manage one Kubernetes project, the complexity increases | ||
dramatically when you add more and more projects and you want to keep a | ||
certain homogeneity between them. The main problems observed | ||
are: | ||
|
||
* It does not scale | ||
* There is a lot of code duplications | ||
* There is no implementation reference | ||
* Improvements are not always backported to other projects | ||
* It is very hard to maintain | ||
* All projects will eventually diverge | ||
* Each project points to different versions of dependencies | ||
|
||
This is not a new IT problem. Two decades ago, it was not really easy to | ||
manage thousands, or even hundreds of hosts in a homogeneous way. | ||
This problem has been successfully addressed by solutions like CFEngine, | ||
Puppet or more recently by Ansible. Today, we must adapt these | ||
principles for Kubernetes. There are many open source components out | ||
there, we just need a smart way to interconnect them. | ||
|
||
|
||
== With the DevOps Stack | ||
|
||
image::explanations/with_devops-stack.png[With the DevOps Stack] | ||
|
||
By creating the DevOps Stack (a kind of abstraction layer), it becomes | ||
possible to instantiate all the components through a logical and | ||
coherent process. The main advantages are: | ||
|
||
* It scales | ||
* Minimal code duplication | ||
* A reference implementation | ||
* Improvements can be easily backported to other projects | ||
* Dependency pining is done on the DevOps Stack to ensure reproducibility | ||
|
||
== Summary | ||
|
||
In the end, the DevOps Stack provides: | ||
|
||
* A Playground to test/prototype new stuff related to k8s world | ||
* An opinionated abstraction layer to deploy and manage our Kubernetes | ||
deployments | ||
* Enforce DevOps best practices: | ||
** Infrastructure as Code (IaC) | ||
** Configuration Management (CM) | ||
** GitOps | ||
** Continuous Integration (CI) | ||
** Continuous Delivery (CD) | ||
** Documentation as Code | ||
* A consistent toolset | ||
** Observability (monitoring, logging, alerting, trending) | ||
** Continuous Delivery (change management + Git features) | ||
** Dynamic Web Routing | ||
** Dynamic Storage Provisioning | ||
** Dynamic Certificate Management | ||
** Authentication and Authorization | ||
|
||
With the following main features: | ||
|
||
* Supports multiple Kubernetes distributions (AWS EKS, Azure AKS, Local | ||
k3s for development and testing purposes), | ||
* Supports blue/green or prod/qa/int/dev patterns, | ||
* Fully automated with GitLab CI pipelines, | ||
* Infrastructure as Code with Terraform, | ||
* Automatic deployment of "core apps" (cert-manager, prometheus, | ||
alertmanager, grafana, loki, ingress controller…) in GitOps spirit with | ||
ArgoCD, | ||
* Central point of documentations with Antora. | ||
|
||
A summary of good practices used daily and experiences acquired over | ||
the time by the DevOps experts at Camptocamp. | ||
|
||
xref:ROOT:getting_started.adoc[Get Started] |