Skip to content

Commit

Permalink
docs: port some of the legacy documentation (#987)
Browse files Browse the repository at this point in the history
Release-As: v1.0.3
  • Loading branch information
lentidas authored Apr 21, 2023
1 parent 0bfcdcd commit 947260d
Show file tree
Hide file tree
Showing 13 changed files with 226 additions and 0 deletions.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
6 changes: 6 additions & 0 deletions docs/modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
.*Main Documentation*
* xref:ROOT:index.adoc[Home]
* Explanations
** xref:ROOT:explanations/purpose.adoc[Purpose]
** xref:ROOT:explanations/cloud_agnostic.adoc[Cloud Agnostic]
** xref:ROOT:explanations/infrastructure_as_code.adoc[Infrastructure as Code]
** xref:ROOT:explanations/continuous_deployment.adoc[Continuous Deployment]
** xref:ROOT:explanations/lifecycle_operations.adoc[Lifecycle Operations]
* Tutorials
** xref:ROOT:tutorials/deploy_kind.adoc[Deployment on KinD]
** xref:ROOT:tutorials/deploy_aks.adoc[Deployment on AKS]
Expand Down
12 changes: 12 additions & 0 deletions docs/modules/ROOT/pages/explanations/cloud_agnostic.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
= Cloud Agnostic

Nobody likes to be locked-in. That's why the DevOps Stack provides a cloud abstraction so you can deploy it on top of multiple Cloud Platforms, in an agnostic way.

Currently, the DevOps Stack supports the following Cloud Platforms, besides a local test deployment with KinD:

- Amazon Web Services (AWS)
- Azure
- Exoscale
- Scaleway
Good "agnostic" automation practices offer many benefits.
17 changes: 17 additions & 0 deletions docs/modules/ROOT/pages/explanations/continuous_deployment.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
= Continuous Delivery & GitOps

Change management is at the heart of Infrastructure Operations. The rise of Continuous Integration and Continuous Delivery practices in the Infrastructure world has been so important that the expression DevOps Engineer is often used to describe CI/CD Engineering.

The DevOps Stack takes advantage of the CI/CD approach to make changes seamless, both at the provisioning and deployment levels.

== Terraform

The underlying infrastructure which sets up the Kubernetes cluster underneath the DevOps Stack is managed with Terraform.

It is strongly encouraged to use a CI/CD approach to manage this Terraform deployment, and the DevOps Stack provides a set of scripts to integrate pair-reviewed change process on your CI/CD platform of choice.

== Argo CD

On top of Kubernetes, the DevOps Stack sets up Argo CD to continuously deploy all applications to the cluster.

Argo CD monitors Kubernetes resources and can keep them synchronized with their target state.
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
= Infrastructure as Code

Deploying Kubernetes clusters from scratch has become an easy task with IaaS interfaces. However, the proper management of all components that make up the cluster and its ecosystem can quickly become a nightmare if everything was done manually.

The Infrastructure as Code approach has long proved its worth to deploy and maintain complex environments on the long run. In particular, in the context of cloud resources, it allows to make sure all resources are managed centrally and destroyed in their proper context. This is important to control the financial consequences of numerous deployments.

The DevOps Stack is strongly founded on the idea that every one of its components should be managed as code. *As such, the underlying Kubernetes cluster and its components are managed with Terraform and the cluster applications are deployed using Helm charts orchestrated by Argo CD.*

This allows you to keep track of all changes made at any level of the stack, using xref:ROOT:explanations/continuous_deployment.adoc[Continuous Deployment] in a GitOps approach.
57 changes: 57 additions & 0 deletions docs/modules/ROOT/pages/explanations/lifecycle_operations.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
= Lifecycle Operations

It is not easy to describe the different phases of an IT project, especially when you also want to talk about operational aspects. One way to do this, which seems to be used more and more, is to talk about "days".

== What "days" are all about

To make it very simple, here are some explanations:

- *Day 0* is related to software development, questions about requirements, architecture and design.
- *Day 1* is more about deploying software developed in the *Day 0* and starting to build and configure the hosting infrastructure.
- *Day 2* is the time to go live and hope that the efforts made in *Day 1* are adequate. This is a crucial moment, when it is imperative to have metrics, monitoring, tools and best practices to do efficient troubleshooting.

For now, the DevOps Stack is very focused on the points relating to days 1 and 2, in particular, the aspects related to the provisioning of a Kubernetes platform as well as the mandatory services related to its operational management.

We must also take into account that using the Cloud and Cloud Native development practices make it much easier to handle changes related to software life cycle. The boundaries and responsibilities between these different phases have become more blurred, less strict.

== Developers need love

In the near future, we really want to invest significantly in aspects related to day 0 by improving the DevOps Stack in that direction, especially the software factory and development best practices related to Kubernetes integration.

This is closely linked to the generalization of the https://12factor.net/[Twelve-Factor App] methodology, as well as breaking silos and embracing DevOps principles. These good practices should be applied as early as possible in projects. It is absolutely necessary.

== Sysadmins want control

The DevOps Stack provisions our Kubernetes clusters with a set of administration and monitoring tools out-of-the-box. Below you will find some of the services' dashboards.

=== Identity Management Dashboard (Keycloak)

Whether you use AKS, EKS, or your local workstation using KinD, the DevOps Stack provisions the cluster with an Identity and Access Manager.

For the case of a local KinD cluster, the DevOps Stack spawns a Keycloak instance.

==== User management with Keycloak
image::explanations/keycloak_users.png[Dashboard for user management]

==== Client management with Keycloak
image::explanations/keycloak_clients.png[Dashboard for client management]

=== Continuous Deployment Dashboard (ArgoCD)

In a xref:ROOT:explanations/continuous_deployment.adoc[GitOps approach], the DevOps Stack uses Argo CD to continuously deploy all applications.

==== Identity and Access Manager (Keycloak) deployment with Argo CD

image::explanations/argo_keycloak.png[Keycloak deployment with Argo CD]

=== Monitoring Dashboards (Grafana)

The DevOps Stack ships with some pre-configured dashboards.

==== Grafana dashboard for monitoring the K8s cluster

image::explanations/grafana_k8s_cluster.png[Dashboard for monitoring the K8s cluster]

==== Grafana dashboard for monitoring cert-manager

image::explanations/grafana_cert_manager.png[Dashboard for monitoring cert-manager]
125 changes: 125 additions & 0 deletions docs/modules/ROOT/pages/explanations/purpose.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
= Purpose of the DevOps Stack


== Introduction

Since about 2006, the IT industry has been totally disrupted by the
conjunction of several factors, including the virtualization of
resources and the possibility of obtaining them on-demand.
This new way of managing IT resources is widely known as "Cloud Computing".
It has become a kind of standard, or at least an objective
pursued also by people wishing to maintain on-premises infrastructures.
The long-awaited dream of finally being able to consider IT as a simple
commodity. Well that's what many believed.

The concrete consequences of this revolution for IT companies like
Camptocamp have mainly been the incredible growth in the number of
resources to be provisioned and managed. Extensive process automation
engaged by our teams allowed us to cope with such a growth without lowering
our quality standards, therefore remaining competitive.

We had no respite in this automation journey and made the best out of a thriving
ecosystem of open source projects that address all the needs related
to this paradigm shift.
We have learned a lot during all these years and the
DevOps Stack is our answer to implement our best practices in a
containerized world where Kubernetes is the _de facto_ standard for
application platforms.


== Cloud Agnostic Kubernetes Distribution

Nowadays, the question is not so much whether there is software to solve
a problem, but which one to use. Like a Linux distribution, we have
container orchestrators (Kubernetes as a kernel) and many other projects
revolving around it and offering different features. These features are
sometimes oriented towards developers, operators or both (e.g. observability
tools). 

As for a Linux distribution, what we need is consistency, components
that work together and that are easy to install and maintain. But unlike a
Linux distribution which is by definition server-centric, the promise of
containers and Kubernetes is portability. The possibility of moving
from one cloud (or on-prem infrastructure) to another, of distributing
the workloads on different platforms. At the end, this DevOps Stack is
simply a "Cloud Agnostic Kubernetes Distribution".


== Without the DevOps Stack

image::explanations/without_devops-stack.png[Without the DevOps Stack]

The illustration above tries to show that while it is quite
simple to manage one Kubernetes project, the complexity increases
dramatically when you add more and more projects and you want to keep a
certain homogeneity between them. The main problems observed
are:

* It does not scale
* There is a lot of code duplications
* There is no implementation reference
* Improvements are not always backported to other projects
* It is very hard to maintain
* All projects will eventually diverge
* Each project points to different versions of dependencies

This is not a new IT problem. Two decades ago, it was not really easy to
manage thousands, or even hundreds of hosts in a homogeneous way.
This problem has been successfully addressed by solutions like CFEngine,
Puppet or more recently by Ansible. Today, we must adapt these
principles for Kubernetes. There are many open source components out
there, we just need a smart way to interconnect them.


== With the DevOps Stack

image::explanations/with_devops-stack.png[With the DevOps Stack]

By creating the DevOps Stack (a kind of abstraction layer), it becomes
possible to instantiate all the components through a logical and
coherent process. The main advantages are:

* It scales
* Minimal code duplication
* A reference implementation
* Improvements can be easily backported to other projects
* Dependency pining is done on the DevOps Stack to ensure reproducibility

== Summary

In the end, the DevOps Stack provides:

* A Playground to test/prototype new stuff related to k8s world
* An opinionated abstraction layer to deploy and manage our Kubernetes
deployments
* Enforce DevOps best practices:
** Infrastructure as Code (IaC)
** Configuration Management (CM)
** GitOps
** Continuous Integration (CI)
** Continuous Delivery (CD)
** Documentation as Code
* A consistent toolset
** Observability (monitoring, logging, alerting, trending)
** Continuous Delivery (change management + Git features)
** Dynamic Web Routing
** Dynamic Storage Provisioning
** Dynamic Certificate Management
** Authentication and Authorization

With the following main features:

* Supports multiple Kubernetes distributions (AWS EKS, Azure AKS, Local
k3s for development and testing purposes),
* Supports blue/green or prod/qa/int/dev patterns,
* Fully automated with GitLab CI pipelines,
* Infrastructure as Code with Terraform,
* Automatic deployment of "core apps" (cert-manager, prometheus,
alertmanager, grafana, loki, ingress controller…) in GitOps spirit with
ArgoCD,
* Central point of documentations with Antora.

A summary of good practices used daily and experiences acquired over
the time by the DevOps experts at Camptocamp.

xref:ROOT:getting_started.adoc[Get Started]

0 comments on commit 947260d

Please sign in to comment.