Skip to content

Commit

Permalink
Merge develop to master branch after 0.7.1 release. (#1557)
Browse files Browse the repository at this point in the history
* Initialized test status table

* Added next sections of test status

Refactored status table a bit, added next lines, added next section with descriptions.

* Upgrade cluster section filled

* All sections filled

* Add missing tests

* Move CNS proposition design doc to GH.

* fixed formatting

* Etcd encryption feature refactor for deployment and upgrades (#1427)

* kubernetes_master: etcd encryption simplification and refactor

* upgrade: refactor of upgrade-kubeadm-config.yml (proper yaml parsing)

* upgrade: adding etcd encryption patching procedure

* upgrade-master.yml: small coding style improvement (highlight fix)

* upgrade: enabling patching of the kubeadm config

* fact naming improvements

Co-authored-by: to-bar <46519524+to-bar@users.noreply.github.com>

* patch-kubeadm-config.yml: skipping unnecessary kubectl apply

Co-authored-by: to-bar <46519524+to-bar@users.noreply.github.com>

* Bumping AzureCLI to fix SP secrets with special characters.

* Added Changelog entry.

* Change move to copy build dir during an upgrade (#1429)

* Change move to copy build dir during an upgrade
* Got rid of unused backup_temp_dir

* Update to logging

- log piping for stderr.
- custom colors for different log levels
- mapping some cases of log warnings and errors from Terraform and Ansible

* helm documentation #896

* Progress:

- simplified piping

* Fix K8s upgrade: 'kubeadm upgrade apply' hangs (#1431)

* Clean up and optimize K8s upgrades

* Patch only kubeadm-config ConfigMap

* Downgrade CoreDNS to K8s built-in version before 'kubeadm upgrade apply'

* Deploy customized CoreDNS after K8s is upgraded to the latest version

* Update changelog

* Wait for API resources to propagate

* Rename vendor in VSCode recommendations (#1438)

Vendor moved owner of mauve.terraform repository to HashiCorp (https://marketplace.visualstudio.com/items?itemName=HashiCorp.terraform)

* Fix issue with Vault and Kubernetes Calico/Canal communication (#1434)

* Add vault namespace and fixes related to connection issue

* Add default policy for default namespace

* Remove service endpoint, execute certificate part if enabled, setting protocol correctly in Vault Helm chart

* Add possibility to configure manually Vault endpoint

* Added changelog.

* add howto links for helm doc

* Update Changelog for #1438 (#1460)

* Update Changelog

* Update Changelog - add PR number

* bump rabbitmq version from 3.7.10 to 3.8.3 #1395

* Changes in documentation after creating fix for calico and canal (#1459)

* Changes after creating fix for calico and canal

* Update changelog

* Got rid of pipe and grep (#1472)

* Assert that current version is upgradeable #1474 (#1476)

* Assert that upgrade from current version is supported #1474

* Update core/src/epicli/data/common/ansible/playbooks/roles/upgrade/tasks/kubernetes.yml

Co-authored-by: to-bar <46519524+to-bar@users.noreply.github.com>

* Add docker_version variable support (#1477)

* add docker_version variable support
* Docker installation - 2 tasks merged into 1 to speed up the deployment
* Remove two useless packages from docker installation

Co-authored-by: Grzegorz Dajuk <grzegorz.dajuk@zipzero.com>

* Kubernetes HA upgrades (#1456)

* epicli/upgrade: reusing existing shared-config + cleanups

* upgrade: k8s HA upgrades minimal implementation

* upgrade: kubernetes cleanup and refactor

* Apply suggestions from code review

Co-authored-by: to-bar <46519524+to-bar@users.noreply.github.com>

* upgrade: removing unneeded kubeconfig from k8s nodes (security fix)

* upgrade: statefulset patching refactor

* upgrade: cleanups and refactor for logs

* Make deployment manifest tasks more generic

* Improve detecting CNI plugin

* AnsibleVarsGenerator.py: fixing regression issue introducted during upgrade refactor

* Apply suggestions from code review

Co-authored-by: to-bar <46519524+to-bar@users.noreply.github.com>

* upgrade: statefulset patching refactor

- patching all containers (fix)
- patching init containers also (fix)
- removing include_tasks statements (speedup)

* Ensure settings for backward compatibility

* Revert "Ensure settings for backward compatibility"

This reverts commit 5c9cdb6.

* AnsibleInventoryUpgrade.py: merging shared-config with defaults

* Adding changelog entry

* Revert "AnsibleVarsGenerator.py: fixing regression issue introducted during upgrade refactor"

This reverts commit c38eb9d.

* Revert "epicli/upgrade: reusing existing shared-config + cleanups"

This reverts commit e5957c5.

* AnsibleVarsGenerator.py: adding nicer way to handle shared config

Co-authored-by: to-bar <46519524+to-bar@users.noreply.github.com>

* Fix upgrade of flannel to v0.12.0 (#1484)

* Readme and changelog update (#1493)

Readme and changelog update

* Fixing broken offline CentOS 7.8 installation (#1498)

* repository: adding the missing centos-logos package

* updating 0.7.1 changelog

* repository/centos-7: restoring alphabetical order

* Add modularization-approaches.md design document

* Kibana config always points its elasticsearch.hosts to a "logging" VM (#1347) (#1483)

* Bump elliptic from 6.5.0 to 6.5.3 in /examples/keycloak/implicit/react

Bumps [elliptic](https://github.com/indutny/elliptic) from 6.5.0 to 6.5.3.
- [Release notes](https://github.com/indutny/elliptic/releases)
- [Commits](indutny/elliptic@v6.5.0...v6.5.3)

Signed-off-by: dependabot[bot] <support@github.com>

* Bump elliptic in /examples/keycloak/authorization/react

Bumps [elliptic](https://github.com/indutny/elliptic) from 6.5.0 to 6.5.3.
- [Release notes](https://github.com/indutny/elliptic/releases)
- [Commits](indutny/elliptic@v6.5.0...v6.5.3)

Signed-off-by: dependabot[bot] <support@github.com>

* Always setting hostname on all nodes of the cluster (on-prem fix) (#1509)

* common: always setting hostname on all nodes of the cluster (on-prem fix)

* updating 0.7.1 changelog

* Workarund restart rabbitmq pods during patching #1395

* add missing changelog entry

* Upgrade Kubernetes to v1.18.6 (#1501)

* Upgrade k8s-dashboard to v2.0.3 (#1516)

* fix due to review

* Dashboard unavailability, network fix for Flannel and Canal #1394 (#1519)

* additional defaults for kafka config

* fixes after review, remove redundant code

* Named demo configuration the same as generated one

* Added deletion step description

* Added a note related to versions for upgrades

* Fixed syntax errors

* Added prerequisites section in upgrade doc

* Added key encoding troubleshooting info

* Test fixes for RabbitMQ 3.8.3 (#1533)

* fix missing variable image rabbitmq

* Add Kubernetes Dashboard to COMPONENTS.md (#1546)

* Update CHANGELOG-0.7.md

Minor changes to changelog before release.

* CHANGELOG-0.7.md update v0.7.1 release date (#1552)

* Increment version string to 0.7.1 (#1554)

Co-authored-by: Mateusz Kyc <mateusz.kyc@gmail.com>
Co-authored-by: Mateusz Kyc <mkyc@users.noreply.github.com>
Co-authored-by: Michał Opala <sk4zuzu@gmail.com>
Co-authored-by: to-bar <46519524+to-bar@users.noreply.github.com>
Co-authored-by: Luuk van Venrooij <luukvanvenrooij84@gmail.com>
Co-authored-by: Tomasz Arendt <tomasz.arendt@pl.abb.com>
Co-authored-by: Marcin Pyrka <pyrka.marcin@gmail.com>
Co-authored-by: erzetpe <erzetpe@gmail.com>
Co-authored-by: Luuk van Venrooij <11056665+seriva@users.noreply.github.com>
Co-authored-by: ar3ndt <tomasz.arendt@gmail.com>
Co-authored-by: Grzegorz Dajuk <grzegorz@dajuk.net>
Co-authored-by: Grzegorz Dajuk <grzegorz.dajuk@zipzero.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: TolikT <tolikt@users.noreply.github.com>
Co-authored-by: przemyslavic <43173646+przemyslavic@users.noreply.github.com>
  • Loading branch information
16 people committed Aug 13, 2020
1 parent 862dec1 commit 9960dea
Show file tree
Hide file tree
Showing 142 changed files with 2,824 additions and 3,268 deletions.
34 changes: 32 additions & 2 deletions CHANGELOG-0.7.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,35 @@
# Changelog 0.7

## [0.7.1] 2020-08-12

### Added

- Minor logging improvements added while fixing issue [#1424](https://github.com/epiphany-platform/epiphany/issues/1424)
- [#1438](https://github.com/epiphany-platform/epiphany/pull/1438) - Rename Terraform plugin vendor in VSCode recommendations
- [#1413](https://github.com/epiphany-platform/epiphany/issues/1413) - Set protocol for Vault only in one place in configuration
- [#1423](https://github.com/epiphany-platform/epiphany/issues/1423) - Error reading generated service principal

### Updated

- [#1479](https://github.com/epiphany-platform/epiphany/issues/1479) - Upgrade K8s to v1.18.6
- [#1510](https://github.com/epiphany-platform/epiphany/issues/1510) - Upgrade Kubernetes Dashboard to v2.0.3

### Fixed

- [#1424](https://github.com/epiphany-platform/epiphany/issues/1424) - Terraform returning an error during deployments on Azure ("A retryable error occurred.")
- [#1399](https://github.com/epiphany-platform/epiphany/issues/1399) - Epicli upgrade: Kubernetes upgrade may hang
- [#1398](https://github.com/epiphany-platform/epiphany/issues/1398) - Vault installation fails when using canal/calico network plugin
- [#1412](https://github.com/epiphany-platform/epiphany/issues/1412) - Certificate in Vault is also generated or copied even if flag in configuration tls_disable is set to true
- [#1408](https://github.com/epiphany-platform/epiphany/issues/1408) - Epicli upgrade: Epiphany does not support upgrades for Kubernetes in HA mode
- [#1482](https://github.com/epiphany-platform/epiphany/issues/1482) - Epicli upgrade: flannel CNI plugin is not upgraded to v0.12.0
- [#1462](https://github.com/epiphany-platform/epiphany/issues/1461) - Epicli upgrade: [AWS/RHEL/calico] Upgrading cluster from 0.6 to 0.7 fails
- [#1072](https://github.com/epiphany-platform/epiphany/issues/1072) - [AWS/RHEL] Cluster networking issues/lags when using flannel/canal plugin
- [#802](https://github.com/epiphany-platform/epiphany/issues/802) - Docker version is hard-coded in installation tasks
- [#1495](https://github.com/epiphany-platform/epiphany/issues/1495) - Offline installation is broken for CentOS 7.8 environments
- [#1347](https://github.com/epiphany-platform/epiphany/issues/1347) - Kibana config always points its elasticsearch.hosts to a "logging" VM
- [#1336](https://github.com/epiphany-platform/epiphany/issues/1336) - Deployment of version 0.7.0 failed on-prem (spec.hostname)
- [#1394](https://github.com/epiphany-platform/epiphany/issues/1394) - Cannot access Kubernetes dashboard after upgrading

## [0.7.0] 2020-06-30

### Added
Expand Down Expand Up @@ -33,7 +63,8 @@
- [#1180](https://github.com/epiphany-platform/epiphany/issues/1180) - Update list of ports used by Epiphany components
- [#1310](https://github.com/epiphany-platform/epiphany/issues/1310) - Updated Azure-cli from 2.0.67 to 2.6.0
- [#1330](https://github.com/epiphany-platform/epiphany/issues/1330) - Update cloud based OS images
- [#1138](https://github.com/epiphany-platform/epiphany/issues/1138) - Upgrade Kubernetes to 1.17.7
- [#1138](https://github.com/epiphany-platform/epiphany/issues/1138) - Upgrade Kubernetes to 1.17.70
- [#1395](https://github.com/epiphany-platform/epiphany/issues/1395) - Upgrade RabbitMQ from 3.7.10 to 3.8.3

### Fixed

Expand All @@ -48,7 +79,6 @@
- [#1372](https://github.com/epiphany-platform/epiphany/issues/1372) - [BUG] Epicli does not create Postgresql SET\_BY\_AUTOMATION values correctly
- [#1373](https://github.com/epiphany-platform/epiphany/issues/1373) - [BUG] permission denied for shared directory in the container when no volume was mounted
- [#1385](https://github.com/epiphany-platform/epiphany/issues/1385) - [BUG] Regression issue with disabling etcd encryption
- [#1399](https://github.com/epiphany-platform/epiphany/issues/1399) - [BUG] Epicli upgrade issue - the process hangs for several hours on the task kubeadm upgrade apply

### Known Issues

Expand Down
39 changes: 30 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,26 @@
# Documentation
# Epiphany Platform
[![GitHub release](https://img.shields.io/github/v/release/epiphany-platform/epiphany.svg)](https://github.com/epiphany-platform/epiphany/releases)
[![Github license](https://img.shields.io/github/license/epiphany-platform/epiphany)](https://github.com/epiphany-platform/epiphany/releases)

## Overview

Epiphany at its core is a full automation of Kubernetes and Docker plus additional builtin services/components like:

- Kafka or RabbitMQ for high speed messaging/events
- Prometheus and Alertmanager for monitoring with Graphana for visualization
- Elasticsearch and Kibana for centralized logging
- Elasticsearch and Kibana for centralized logging (OpenDistro)
- HAProxy for loadbalancing
- Postgress for storage
- Postgres and Elasticsearch for data storage
- KeyCloak for authentication
- Vault (MVP) for protecting secrets and other sensitive data
- Helm as package manager for Kubernetes

Epiphany can run on as few as one node (laptop, desktop, server) but the real value comes from running 3 or more nodes for scale and HA. Nodes can be added or removed at will depending on data in the manifest. Everything is data driven so simply changing the manifest data and running the automation will modify the environment.
The following target platforms are available: AWS, Azure and on-prem installation.

Epiphany can run on as few as one node (laptop, desktop, server) but the real value comes from running 3 or more nodes for scale and HA. Everything is data driven so simply changing the manifest data and running the automation will modify the environment.
Kubernetes hosts (masters, nodes) and component VMs can be added depending on data in the initial manifest. More information [here](https://github.com/epiphany-platform/epiphany/blob/develop/docs/home/howto/CLUSTER.md#how-to-scale-or-cluster-components).

Please note that currently Epiphany supports only creating new masters and nodes and adding them to the Kubernetes cluster. It doesn't support downscale. To remove them from Kubernetes cluster you have to do it manually.

We currently use Terraform and Ansible for our automation orchestration. All automation is idempotent so you can run it as many times as you wish and it will maintain the same state unless you change the data. If someone makes a "snow flake" change to the environment (you should never do this) then simply running the automation again will put the environment back to the desired state.

Expand Down Expand Up @@ -41,13 +50,25 @@ This minimum file definition is fine to start with, if you need more control ove
epicli init -p aws -n demo --full
```

You will need to modify a few values (like your AWS secrets, directory path for SSH keys). Once you are done with `demo.yaml` you can start cluster deployment by executing:
You will need to modify a few values (like your AWS secrets, directory path for SSH keys). Once you are done with `demo.yml` you can start cluster deployment by executing:

```shell
epicli apply -f demo.yaml
epicli apply -f demo.yml
```
You will be asked for a password that will be used for encryption of some of build artifacts. More information [here](docs/home/howto/SECURITY.md#how-to-run-epicli-with-password)

Since version 0.7 epicli has an option to backup/recovery some of its components. More information [here](https://github.com/epiphany-platform/epiphany/blob/develop/docs/home/howto/BACKUP.md)
```shell
epicli backup -f <file.yml> -b <build_folder>
epicli recovery -f <file.yml> -b <build_folder>
```

To delete all deployed components following command should be used

```shell
epicli delete -b <build_folder>
```

Find more information using table of contents below - especially the [How-to guides](docs/home/HOWTO.md).

## Documentation
Expand All @@ -59,8 +80,8 @@ Find more information using table of contents below - especially the [How-to gui
- [How-to guides](docs/home/HOWTO.md)
- [Components](docs/home/COMPONENTS.md)
- [Security](docs/home/SECURITY.md)
- [Troubleshooting](docs/home/TROUBLESHOOTING.md)
- [Changelog](CHANGELOG.md)
- [Troubleshooting](docs/home/TROUBLESHOOTING.md)
- [Changelog](CHANGELOG.md)
- Architecture
- [Logical View](docs/architecture/logical-view.md)
- [Process View](docs/architecture/process-view.md)
Expand All @@ -69,5 +90,5 @@ Find more information using table of contents below - especially the [How-to gui
- [Governance model](docs/home/GOVERNANCE.md)
- [Development environment](docs/home/DEVELOPMENT.md)
- [GIT Workflow](docs/home/GITWORKFLOW.md)

<!-- TOC -->
2 changes: 1 addition & 1 deletion core/src/epicli/.devcontainer/Pipfile
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jsonschema = "*"
python-json-logger = "*"
ansible = "==2.8.8"
terraform-bin = "*"
azure-cli = "==2.6.0"
azure-cli = "==2.8.0"
skopeo-bin = "*"
"ruamel.yaml" = "*"

Expand Down
Loading

0 comments on commit 9960dea

Please sign in to comment.