Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

draft: docs for kubefirst local #823

Merged
merged 18 commits into from
Dec 12, 2022
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 37 additions & 16 deletions cmd/destroyAwsGithub.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,9 @@ import (
"time"

"github.com/kubefirst/kubefirst/configs"
"github.com/kubefirst/kubefirst/internal/argocd"
"github.com/kubefirst/kubefirst/internal/flagset"
"github.com/kubefirst/kubefirst/internal/gitClient"
"github.com/kubefirst/kubefirst/internal/handlers"
"github.com/kubefirst/kubefirst/internal/k8s"
"github.com/kubefirst/kubefirst/internal/progressPrinter"
Expand Down Expand Up @@ -85,25 +87,9 @@ var destroyAwsGithubCmd = &cobra.Command{
}
progressPrinter.AddTracker("step-prepare", "Open Ports", 3)

informUser("Open argocd port-forward", globalFlags.SilentMode)
progressPrinter.IncrementTracker("step-prepare", 1)

progressPrinter.AddTracker("step-destroy", "Destroy Cloud", 4)
progressPrinter.IncrementTracker("step-destroy", 1)

if !destroyFlags.SkipGithubTerraform {
githubTfApplied := viper.GetBool("terraform.github.apply.complete")
if githubTfApplied {
informUser("terraform destroying github resources", globalFlags.SilentMode)
tfEntrypoint := config.GitOpsRepoPath + "/terraform/github"
terraform.InitDestroyAutoApprove(globalFlags.DryRun, tfEntrypoint)
informUser("successfully destroyed github resources", globalFlags.SilentMode)
}
log.Println("github terraform destruction complete")
}

progressPrinter.IncrementTracker("step-destroy", 1)

//This should wrapped into a function, maybe to move to: k8s.DeleteRegistryApplication
if !destroyFlags.SkipDeleteRegistryApplication {
kPortForwardArgocd, _ := k8s.PortForward(globalFlags.DryRun, "argocd", "svc/argocd-server", "8080:80")
Expand All @@ -116,13 +102,48 @@ var destroyAwsGithubCmd = &cobra.Command{
informUser("Open argocd port-forward", globalFlags.SilentMode)
progressPrinter.IncrementTracker("step-prepare", 1)

informUser("Refreshing local gitops repository", globalFlags.SilentMode)
log.Println("removing local gitops directory")
os.RemoveAll(config.GitOpsRepoPath)

log.Println("cloning fresh gitops directory from github owner's private gitops")
gitClient.ClonePrivateRepo(fmt.Sprintf("https://github.com/%s/gitops", viper.GetString("github.owner")), config.GitOpsRepoPath)

informUser("Removing ingress-nginx load balancer", globalFlags.SilentMode)

log.Println("removing ingress-nginx.yaml from local gitops repo registry")
os.Remove(fmt.Sprintf("%s/registry/ingress-nginx.yaml", config.GitOpsRepoPath))

gitClient.PushLocalRepoUpdates("github.com", viper.GetString("github.owner"), "gitops", "origin")
token, err := argocd.GetArgoCDToken("admin", viper.GetString("argocd.admin.password"))
if err != nil {
log.Fatal("could not collect argocd token", err)
}

log.Println("syncing argocd registry application")
argocd.SyncArgocdApplication(false, "registry", token)

log.Println("waiting for nginx to deprovision load balancer")
time.Sleep(time.Second * 15)

log.Println("deleting registry application in argocd")
// delete argocd registry
informUser("Destroying Registry Application", globalFlags.SilentMode)
k8s.DeleteRegistryApplication(destroyFlags.SkipDeleteRegistryApplication)
log.Println("registry application deleted")
}
progressPrinter.IncrementTracker("step-destroy", 1)

if !destroyFlags.SkipGithubTerraform {
githubTfApplied := viper.GetBool("terraform.github.apply.complete")
if githubTfApplied {
informUser("terraform destroying github resources", globalFlags.SilentMode)
tfEntrypoint := config.GitOpsRepoPath + "/terraform/github"
terraform.InitDestroyAutoApprove(globalFlags.DryRun, tfEntrypoint)
informUser("successfully destroyed github resources", globalFlags.SilentMode)
}
log.Println("github terraform destruction complete")
}
progressPrinter.IncrementTracker("step-destroy", 1)

log.Println("terraform destroy base")
Expand Down
20 changes: 5 additions & 15 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,21 +12,11 @@ docker run --rm -it -p 8000:8000 -v ${PWD}:/docs squidfunk/mkdocs-material

### publishing to preprod (temp docs)

```
# confirm aws config pointed at mgmt account
merging docs changes to the main branch will automatically kick off a publish to preprod using the [Publish Docs](https://github.com/kubefirst/kubefirst/actions/workflows/publish-docs.yaml) action.
[https://docs.kubefirst.com/preprod/index.html](https://docs.kubefirst.com/preprod/index.html)

pip3 install mkdocs mkdocs-material
rm -rf ./dist
mkdocs build --no-directory-urls -d dist
aws s3 sync dist s3://docs.kubefirst.com/preprod --delete
aws cloudfront create-invalidation --distribution-id E1DXWJ8ITAKV61 --paths "/preprod/*"
# https://docs.kubefirst.com/preprod/index.html
```

### promote to prod
```
aws s3 sync dist s3://docs.kubefirst.com --delete
aws s3 sync dist s3://docs.kubefirst.com/preprod --delete # prod deploy blows away preprod
aws cloudfront create-invalidation --distribution-id E1DXWJ8ITAKV61 --paths "/*"
# https://docs.kubefirst.com/index.html
```

after confirming there are no rendering issues in preprod, run the github action [Promote Docs To Prod](https://github.com/kubefirst/kubefirst/actions/workflows/promote-docs-to-prod.yaml) to update the live site.
[https://docs.kubefirst.com/index.html](https://docs.kubefirst.com/index.html)
File renamed without changes.
Binary file modified docs/img/kubefirst/gitops/gitops-cicd.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/kubefirst/icons/k-ray.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/kubefirst/local/atlantis.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/kubefirst/local/console.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/kubefirst/local/gitops-registry.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/kubefirst/local/handoff-screen.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/kubefirst/local/repos-list.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
23 changes: 13 additions & 10 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,23 @@
# Choose your adventure

To try kubefirst locally with no costs, check out our new `kubefirst local` command here:
[https://kubefirst.io/blog/kubefirst-v110-release-notes](https://kubefirst.io/blog/kubefirst-v110-release-notes)
The Kubefirst Platform now supports 3 installation types

The kubefirst platform now supports both the [GitHub](https://www.github.com) and [GitLab](https://www.gitlab.com) Git providers.
The Git provider you choose will host the code for you applications, your new gitops repository, and will be fully configured through that gitops repository with some new Terraform that will manage your team and user access to the system.
## Kubefirst Local (GitHub Only)

If you choose **GitLab**, your Git Provider will be self-hosted, meaning Kubefirst will install **GitLab** into your newly created kubernetes
cluster. If you choose **GitHub**, the Kubefirst platform will leverage the free **GitHub** system at github.com.
This is the fastest way to check otu the kubefirst platform. This installation type will automatically create a local k3d cluster on your laptop, put a gitops repository in your personal GitHub account, and bootstrap the local cluster against that new repository. You will be able to run gitops deployments, build images, publish helm charts, and even run automated infrastructure as code, all without a cloud account or a domain requirement.
johndietz marked this conversation as resolved.
Show resolved Hide resolved

## Kubefirston AWS (GitHub or GitLab)

Our AWS cloud platform supports both [GitHub](https://www.github.com) and [GitLab](https://www.gitlab.com) as git providers.

If you choose **GitLab**, your Git Provider will be self-hosted, meaning Kubefirst will install **GitLab** into your newly created kubernetes cluster. If you choose **GitHub**, the Kubefirst platform will leverage the free **GitHub** system at github.com.

<center>

| Kubefirst with GitHub | Kubefirst with Self Hosted GitLab |
|:------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------:|
| [![GitHub](./img/kubefirst/icons/github-200x200.png)](./kubefirst/github/install.md) | [![GitLab](./img/kubefirst/icons/gitlab-200x200.png)](./kubefirst/gitlab/install.md) |
| [Team Octocat!](./kubefirst/github/install.md) | [Team Tanuki!](./kubefirst/gitlab/install.md) |
| Kubefirst Local | Kubefirst with GitHub | Kubefirst with Self Hosted GitLab |
|:------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------:|
| [![Kubefirst](./img/kubefirst/icons/k-ray.png)](./kubefirst/local/install.md) | [![GitHub](./img/kubefirst/icons/github-200x200.png)](./kubefirst/github/install.md) | [![GitLab](./img/kubefirst/icons/gitlab-200x200.png)](./kubefirst/gitlab/install.md) |
| [Team K-Ray!](./kubefirst/local/install.md) | [Team Octocat!](./kubefirst/github/install.md) | [Team Tanuki!](./kubefirst/gitlab/install.md) |

</center>

Expand Down
55 changes: 35 additions & 20 deletions docs/kubefirst/github/after-install.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@

[//]: # (`todo: need new getting started video for github`)

[//]: # (<iframe width="784" height="441" src="https://www.youtube.com/embed/KEUOaNMUqOM" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>)
<div class="video-wrapper">
<iframe width="1280" height="400" src="https://www.youtube.com/embed/KEUOaNMUqOM" frameborder="0" allowfullscreen></iframe>
</div>

**psssst** *- if you plan to destroy your kubefirst platform and recreate it again we recommend running `kubefirst backupSSL --include-metaphor` to re-use your ssl certs from Let's Encrypt. See the [docs](https://docs.kubefirst.io/common/certificates.html#backup-and-restore-certificates).*

Expand Down Expand Up @@ -36,7 +38,7 @@ You now have an EKS cluster with the following content installed in it:
Once you run the `cluster create` command at the end of the installation will open a new browser tab with the Console UI at
`http://localhost:9094` to provide you a dashboard to navigate through the different services that were provisioned.

![console ui](../../img/kubefirst/console-ui.png)
![console ui](../../img/kubefirst/github/console.png)

![terminal handoff](../../img/kubefirst/getting-started/cluster-create-result.png)

Expand All @@ -46,32 +48,45 @@ store them in a safe place.

## Step 2: Add Your Team

Log into GitLab using the root credentials that were provided to you in your terminal.
Navigate to the `gitops` repo in your GitHub organizaiton and edit the file `terraform/users/admin.tf`. In this file, you'll see a commented block that represents another admin user:

Once logged in, navigate to the `gitops` project and edit the file `terraform/users/admin.tf`. In this file, you'll see some blocks that represent admin users:
```
# module "admin_one" {
# source = "./modules/user/github"

# acl_policies = ["admin"]
# email = "admin@your-company-io.com"
# first_name = "Admin"
# github_username = "admin_one_github_username"
# last_name = "One"
# username = "aone"
# user_disabled = false
# userpass_accessor = data.vault_auth_backend.userpass.accessor
# }
```

Edit this code to replace the values for the `module name`, `email`, `first_name`, `github_username`, `last_name`, and `user_name`. There is also a file for your developers at `terraform/users/developers.tf`. You can duplicate those snippets of code in these files to create as many developers and admins as you need.

There's also a list at the top of this file, which you should add your new admin to
```
module "admin_one" {
source = "./templates/oidc-user"
admins_group_id = gitlab_group.admins.id
developer_group_id = gitlab_group.developer.id
username = "admin1"
fullname = "Admin One"
email = "admin1@yourcompany.com"
is_admin = true
resource "vault_identity_group_member_entity_ids" "admins_membership" {
member_entity_ids = [
module.kubefirst_bot.vault_identity_entity_id,
#### PUT YOUR NEW ADMIN HERE
]

group_id = data.vault_identity_group.admins.group_id
}
```

Edit this code to replace the values for the `module name`, `username`, `fullname`, and `email`. There is also a file for your developers at `terraform/users/developers.tf`. You can duplicate those snippets of code in these files to create as many developers and admins as you need.

Commit this change to a **new branch** and create a merge request. This will kick off the Atlantis workflow. Within a minute or so of submitting the merge request, a comment will appear on the merge request that shows the terraform plan with the changes it will be making to your infrastructure.
Commit this change to a **new branch** and create a merge request. This will kick off the Atlantis workflow. On the local story, this is a little interesting. We've started an ngrok tunnel on your localhost that's kept open through the `kubefirst local` command. So when the pull request is submitted, the gitops repo has a webhook that invokes a call to your ngrok tunnel, in order to invoke Atlantis terraform execution within your local cluster. Within a minute or so of submitting the merge request, a comment will appear on the pull request that shows the terraform plan with the changes it will be making to your infrastructure.

To apply these changes, submit a comment on that Merge Request with the following comment text:
```
atlantis apply
```

Doing so will instruct Atlantis to apply the plan. It will report back with the results of the apply within a minute or so.
Doing so will instruct your local Atlantis to apply the plan. It will report back with the results of the apply within a minute or so.

NOTE: Atlantis merges your Pull Request automatically once an apply is successfully executed. Don't merge Terraform merge requests yourself.

Expand All @@ -83,19 +98,19 @@ Any new users you have created through this process will have their temporary in

## Step 3: Deliver Metaphor to Development, Staging, and Production

Metaphor is our sample application that we use to demonstrate parts of the platform and to test CI changes. It's the other project in the Kubefirst group in GitLab.
Metaphor is our sample application that we use to demonstrate parts of the platform and to test CI changes.

If you visit its `.gitlab-ci.yml` in the metaphor repo root, you'll see it's sending some workflows to argo. Those workflows are also in the `metaphor` repo in the `.argo` directory.
If you visit its `/.github/workflows/main.yaml` in the metaphor repo, you'll see it's just sending some workflows to argo in your local k3d cluster. Those workflows are also in the `metaphor` repo in the `.argo` directory.

The metaphor pipeline will:

- Publish the metaphor container to your private ECR.
- add the metaphor image to a release candidate helm chart and publish it to chartmuseum
- set the metaphor with the desired Helm chart version in the GitOps repo for development. Staging
- set the metaphor with the desired Helm chart version in the GitOps repo for development and staging
- the release stage of the pipeline will republish the chart, this time without the release candidate notation making it an officially released version and prepare the metaphor application chart for the next release version
- the officially released chart will be set as the desired Helm chart for production.

To watch this pipeline occur, make any change to the `main` branch of the `metaphor` repo. If you're not feeling creative, we put a file at `.argo/ci-files/trigger.txt` that you can use. Once a file in `main` is changed, navigate to metaphor's CI/CD in GitLab to see the workflows get submitted to Argo workflows.
To watch this pipeline occur, make any change to the `main` branch of the `metaphor` repo. If you're not feeling creative, you can just add a newline to the README.md. Once a file in `main` is changed, navigate to metaphor's CI/CD in the github Actions tab to see the workflows get submitted to Argo workflows.

You can visit the metaphor development, staging, and production apps in your browser to see the versions change as you complete resources and ArgoCD syncs the apps. The metaphor URLs can be found in your GitOps and metaphor project `README.md` files.

Expand Down
9 changes: 1 addition & 8 deletions docs/kubefirst/github/credit.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,9 @@ site: [https://external-secrets.io/](https://external-secrets.io/)
site: [https://docs.github.com/en/actions/learn-github-actions/understanding-github-actions](https://docs.github.com/en/actions/learn-github-actions/understanding-github-actions)

### summerwind/actions-runner

site: [https://hub.docker.com/r/summerwind/actions-runner](https://hub.docker.com/r/summerwind/actions-runner)

### Ingress-NGINX
### Ingress-Nginx
site: [https://kubernetes.github.io/ingress-nginx/](https://kubernetes.github.io/ingress-nginx/)

available annotations: [https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/)
Expand All @@ -39,12 +38,6 @@ site: [https://kubernetes.io/](https://kubernetes.io/)

training: [https://kubernetes.io/training/](https://kubernetes.io/training/)

### Material for MkDocs
site: [https://squidfunk.github.io/mkdocs-material](https://squidfunk.github.io/mkdocs-material)

### Soft Serve
code: [https://github.com/charmbracelet/soft-serve](https://github.com/charmbracelet/soft-serve)

### Terraform
site: [https://www.terraform.io/](https://www.terraform.io/)

Expand Down
2 changes: 1 addition & 1 deletion docs/kubefirst/github/github-repositories.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ using cert-manager and the Let's Encrypt cluster-issuer.


The available Metaphor applications are, **Metaphor (NodeJS API)**, **Metaphor (Go API)** and **Metaphor Frontend**.
The Metaphor applications are describe in more details [here](../../common/metaphors.md).
The Metaphor applications are describe in more details [here](../../common/metaphor.md).

## GitHub Repository Management

Expand Down
5 changes: 0 additions & 5 deletions docs/kubefirst/github/gitops.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,6 @@ The benefit of defining your desired state in a declarative way in Git is enormo
- Creating new environments is as straight-forward as duplicating directory structures.
- Having everything declared in the GitOps repo means everyone can agree to the source of truth for every version of every app.

## See How Rollbacks Work
<div class="video-wrapper">
<iframe width="1280" height="500" src="https://www.youtube.com/embed/iqZA9Eycvgo" frameborder="0" allowfullscreen></iframe>
</div>

## Pipelines

![](../../img/kubefirst/gitops/gitops-cicd.png)
Expand Down
2 changes: 1 addition & 1 deletion docs/kubefirst/gitlab/after-install.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ You now have an EKS cluster with the following content installed in it:
Once you run the `cluster create` command at the end of the installation, a new browser tab will open with the Console UI at
`http://localhost:9094` to provide you a dashboard to navigate through the different services that were previsioned.

![console ui](../../img/kubefirst/console-ui.png)
![console ui](../../img/kubefirst/github/console.png)

![terminal handoff](../../img/kubefirst/getting-started/cluster-create-result.png)

Expand Down
Loading