- ingress-controller was installed using helm chart
- prometheus was downloaded locally and installed with our custom values to gke k8s cluster
- kubeStateMetrics and nodeExporter were turned on and prometheus config was updated to get metrics from our reddit app with different circumstances
- grafana was installed using helm, and our templates were added and modified to add choice of env
- * alertmanager was enabled and alerting to slack was added in case of nodes down
- * prometehus operator was installed to different namespace and updated to get metrics from comment app
- EFK stack was installed to log our reddit app and * helm-chart was created for it
- prometheus operator: how to add new scraping targets, prometheus operator - good how-to, how to monitor external services (important info about adding your service using "prometheus" service type), debugging usage of external services, some examples of ServiceMonitor usage, common example of prometeus operator usage
- prometheus operator - important tech info: chart, serviceMonitor spec, homesite, defaul port allocations for prometheus services
- Kibana helm chart: good description, helm chart
- how to monitor k8s cluster using prometheus: Monitoring with Prometheus, Monitoring with Prometheus (RUS)
- helm chart with helpers, dependencies and templates was created for our reddit microservices
- helm3 and different tiller modes were tested
- gitlab was installed to gke and our projects were created in it
- CI pipeline was created for our services
- pipelines were developed with different deploy modes (helm2, helm3, tiller-as-plugin)
- pipeline for master branch was modified to deploy to prod after commit to master
- kube-dns service was tested
- different types of network services were tested (NodePort, LoadBalancer, Ingress)
- tls termination with tls secret was added and yaml-manuifest was created for secret (*)
- network policy with yaml-manifest was created and tested
- different types of volumes were created and tested with appropriate yaml-manifests
- tls secrets in k8s: ssl certificates in ingress, tls secrets with ingress/nginx, how to set secret files to k8s, k8s - concepts of secrets, helm - temlpates for k8s tls secrets, MS - ingress own tls
- yaml-manifests for reddit app were created and tested with minikube and gke
- yaml-manifests for gke dashboard were created
- terraform config was created to deploy kubernetes cluster in gcp
- minikube: Getting started with Kubernetes and Docker with minikube, https://kubernetes.io/docs/tutorials/kubernetes-basics/create-cluster/cluster-intro/
- kind (alternative way to test local k8s cluster), kind in GitHub
- GKE and terraform: google_container_cluster, Kubernetes Cluster in Google Kubernetes Engine (GKE), How to create k8s cluster in GKE with namespaces
- comment-deployment.yml, mongo-deployment.yml, post-deployment.yml and ui-deployment.yml manifests were created for our reddit microservice
- kubernetes cluster was created using "Kubernetes The hard way" manual and our manifests were checked on it
- ansible configs were created to deploy KTHW
- info about regions and zones of gcp
- tmux tips and tricks
- Kubernetes the not so hard way with Ansible
- one more easy way to deploy kubernetes cluster: manual and video
- docker-compose-logging.yml was created to use elastic stack
- fluentd was set up, code of our app was updated to send logs and fluentd was added to services configs as logging driver
- logging in kibana was checked
- some different filters were added in fluentd config and checked in kibana
- logging for ui and parsing rules were added and checked in kibana
- grok-templates were added and checked
- zipkin was added to all our services and some traces were checked
- * another grok-template was added to fluentd config
- * bugged app was traced by zipkin and error was found
- grok-templates for fluentd: readme and examples of templates
- docker-compose.yml was splitted to tow different parts - for apps and for monitoring services
- grafana was installed with some dashboards and different requests
- business logic monitoring dashboard was added
- alertmanager was added and integration set up with slack
- Makefile was updated for new services
- * docker was configured to send metrics to prometheus and dashboard was added in grafana for it
- * telegraf was added to get metrics from docker host and appropriate dashboard was added to grafana
- * some metrics with using procentile were added
- * alertmanager was set up to send alerts through email too
- ** grafana config was recreated to install and run our dashboards automatically during deploy
- ** metrics to check ui time response and timeout of any our services were added
- ** metrics from stackdriver were added
- *** trickster was added and grafana was reconfigured successfully to use trickster as data source
- cAdvisor
- Docker and system monitoring Dashboard by Thibaut Mottet
- useful slack info: how to check webhook
- docker experimental mode: how to setup using daemon.josn and what to do if daemon.json not works
- Docker overview dashboard
- telegraf: docker hub, how to configure, how to set up, client to send to prometheus, list of metrics, simple example of set up, good example of set telegraf-influxdb-grafana w/o prometheus
- prometheus: some useful examples of requests, examples of checks alertmanager: how to set up alerts to be send through email and examples
- grafana - import dashboards automatically: how to inport provider, how to provision boards and providers, default paths and setting for docker image of grafana, samples of provisioning config: one, two, three; datasources
- stackdriver: stackoverflow tips, frodenas exporter, how to create account with appropriate rights for stacdriver
- trickster: official repo, how to cinfigure, example of cinfig file
- autoheal + awx: role from geerlingguy, interesting post from habr, autoheal repo, small sample of using autoheal
- prometheus+nodexport+grafana: sample 1, sample 2
- docker host was created and prometheus installed and explored onto it
- new structure was created in repo and docker image was created with prometheus configuration to monitor our services
- our service with prometheus were set up and explored with prometheus monitoring
- habitation of prometheus was checked with diffrenet stopped services
- node exporter was added to monitor our docker host
- * percona mongodb exporter was added to monitor mongodb state
- * google cloudprober was added to check availability of our services
- * Makefile was created to build and push images, and to stop/run our services
- node_exporter
- precona mongodb exporter and how to use it in different ways
- cloudprober on git, cloudprober - getting started, cloudprober images on docker hub
- make/Makefile: include, variables in Makefile, :makefile:4: *** missing separator. Stop. error
- gitlab ci was installed and prepared to use
- ci/cd pipeline was created to build, test and deploy services with different envs, include dynamic ones
- build of docker containers in dind mode was tested
- * docker container creation was added to build step for reddit app
- * deploy of created container was tested to server with gitlab ci using gitlab runner in shell executor mode
- * deploy of created container was tested to external instance using terraform + ansible with dynamic inventory
- * integration for our gitlab ci with our personal slack channel was added: https://devops-team-otus.slack.com/archives/CBJESDZPH
- * automation of creation many instances with runner in shell executor mode using terraform and ansible was created and tested
- * automation of creation many instances with runner in docker mode in docker container using terraform and ansible was created and tested
- gitlab ci pipeline yaml structure
- gitlab ci predefined variables to use in pipelines
- how to authorize in gcp with terraform (getting started + manual from google) and ansible (in pipelines and ansible.cfg) using gcp service accoung
- how to run and register gitlab runner using docker
- how to build docker images with gitlab runner usign different runner modes
- how to pass variables to ansible playbook
- modules for ansible to work with docker: docker_container_module, docker_image_module
- module for ansible to work with gitlub-runner
- ansible galaxy role to work with gitlub runner: riemers.gitlab-runner
- ansible useful special variables to use it in playbooks during privisioning
- docker-in-docker - pro and contra
- different network drivers were tested and compared using joffotron/docker-net-tools image
- usage of different bridge networks was implemented and tested for the same and different containers
- docker-compose was installed and tested using simple docker-compose.yml
- docker-compose.yml was updated to use few networks and network aliases and parametrized variables using .env file
- base name, which uses as prefix for container names by docker compose, usually corresponds to folder name where docker-compose runs; it can be overriden by COMPOSE_PROJECT_NAME environment variable in .env file or by '-p / --project-name' command line option
- docker-compose.overriden.yml was created to add abilities to change app code without rebuilding images and to run puma in debug mode
- Docker network creation
- docker-compose installation
- docker-compose commands
- how to parametrize variables in docker-compose
- how to parametrize network parameters and explanation of network options in compose file
- how to change prexfix in container names: docker env file and more variants from stackoverflow
- docker-compose.override.yml from docker-compose documentation and habr article in Otus blog
- downloaded, installed and builded application with microservices architecture
- network was created and application was run and checked on the docker-host
- containers were restarted with new network aliases using --env commandline parameter
- Dockerfiles were optimized for ui and comment using ruby:alpine3.10 images
- docker volume was created and checked with destroy and recreate containers that posts and comments still alive
- Docker images optimization
- Dockerfile best practices
- ruby alpine images from docker hub
- --env command line parameter
- new project was created, integration with travis and slack was added
- docker was installed and tested with base image "hello world" and base commands like
run,ps,start,attachetc with options - test container was created from ubuntu image, then images was created and log was written to docker-1.log was created from it
docker inspectfrom image and from container was compared and results of comparison were added to docker-1.log- new project was created in gce for microservice homeworks
- docker-machine was installed to use gce for VMs
- new instance was created in gce with docker by docker-machine and tested for creation of container with reddit app
- new account was created in Docker Hub and our new image with reddit app was uploaded into it and checked for run from it in another console
-
- small sample of infrastructure was developed to create some instances with docker in gcp to run reddit app containers in it
- to install and set up docker: docs.docker.com, tutorial from DO, post-install settings
- signals in unix
- install docker-machine
- ansible - docker_container_module