Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add proposal for k1-for-devs #1757

Closed
wants to merge 3 commits into from
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 76 additions & 0 deletions docs/proposals/20220126_k1-for-devs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# KubeOne for Developers
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This proposal is missing some key sections:

  • Values and alternatives -- the key question I'd like to see answered is why would someone want to use this over GitPod and GitHub Codespaces? Those two products are also cheaper than a Kubernetes cluster created by KubeOne on any cloud provider, so there's that too.
  • Ownership -- who is going to own and maintain the project?
  • Location -- where is this addon going to live? This depends on the ownership question as well. If this is to be maintained by a dedicated team then it should live in another repo.

In my opinion, everyone should be able to use KubeOne as a building block. In that aspect, I'd like this proposal to focus on what we can do to make implementing this possible. I see two key aspects:

  • Improving usability and the infrastructure management experience
  • Improving the addons mechanism

If that's the case, I'd like to see what we can do to improve those two aspects.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Values and alternatives Yes, I think we should spent some time looking for alternatives on the market before starting implementation. Concerning GitPod and CodeSpaces: I am not sure if those are usable on private clusters. Those tools are running on some GitHub infra and not in a dedicated cluster. IMO the idea is to add functionality to clusters created via KubeOne. Which you may can do with GitPod and CodeSpaces via connecting them to your Kubernetes Cluster but this will not be doable all the time (private clusters) and also has some security implications.

Ownership & Location Good question, we should discuss this, maybe @scheeles has some input here


## Goals
* "Developer Batteries included"
* Developers should get enabled to start working on the Kubernetes Cluster without having to deal with infrastructure setup problems
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is unclear and should be clarified.

It's just impossible to get a cluster on any cloud provider without infrastructure setup problems. You at least have to create an account, and in many cases, set up your account, configure billing, configure initial subnets and network gateways (especially on AWS), etc...

Therefore, what are the problems that we should solve?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a different problem you are talking. Yes, you are right you need that stuff to get your cluster via KubeOne up and running.

The problem which we want to solve with this proposal is more about: OK, now I have my running cluster via KubeOne, what do I need to do to have a cool developer experience on it. By developer I mean the "enduser-developer", the one writing apps on the already existing Kubernetes Cluster.

* Independent KubeOne AddOns which can be enabled/disabled depending on the needs

## Non Goals
* Having the plugins available in KKP.
* Setting up Development Environments for specific programming languages.
* Being to opiniated about tooling. Eg zsh or fish should not be included.
* Installing opiniated IDE AddOns. Only the minimal set should be delivered.

## Motivation and Background
For Developers it is still hard to get started with Kubernetes. For getting into production mode they have to care about some infrastructure tasks.

For example:
* Getting an IDE up and running
* Connect to the Kubernetes Cluster
* Installing the Kubernetes Dashboard
* Having a Container Registry in Place
* Having a proper Git setup
* Having GitOps tooling up and running
* Having kubectl and other useful tools (kubectx, bash completion, ) installed
* Having a tool in place for handling sensitive data

## Implementation

### IDE
* Using [Eclipse Che](https://github.com/eclipse/che) as Cloud IDE. Eclipse Che is a workspace server integrating Theia as a default.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let try the devworkspace https://www.eclipse.org/che/docs/che-7/administration-guide/architecture-overview-with-devworkspace/ as it uses more the K8s API and is the future of Che eclipse-che/che#20830

* Theia is a nodejs application which provides the IDE. Workspace management has to be provided by Eclipse Che.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the future, I think we should consider VS Code as the default but still need some development eclipse-che/che#20500

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes we should definitely consider something different than Theia. I had bad experiences with it due to nodejs dependency hell.

* Eclipse Che is holding all the workspace data and all user-specific file-based configurations, eg the .gitconfig

### Terminal integrations
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add tilt https://github.com/tilt-dev/tilt so that you constantly deploy the code to the cluster

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@archups @nerdeveloper can you please take a look at it and give some info about your experiences

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have heard about it, I am not entirely sure if tilt and entando do the same thing. I could take tilt for a spin and check the developer experience.

* Theia provides a terminal which can be configured eg .bashrc
* We should ensure
* The terminal is connected to the Kubernetes Cluster the IDE is running in. The kubeconfig should restrict permissions to developer matters.
* kubectl has to be installed.
* Either docker or crictl has to be installed.
* [Helm](https://github.com/helm/helm) has to be installed
* Useful tools like [kubectx](https://github.com/ahmetb/kubectx) have to be installed.
* [Stern](https://github.com/wercker/stern) should be installed
* Questionable candidates: Istioctl
* Git has to be installed. May we can support developers here also on credential management.

### Security
* The IDE has to be secured. Everything should be done via HTTPS (LetsEncrypt?)
* Password Protection: Otherwise we create a huge security hole for the Kubernetes Cluster.
* Dex as IDP with Github

### Visualization of the Kubernetes Cluster
* Either via enabling the Kubernetes Dashboard or via the [VSCode Kubernetes AddOn](https://marketplace.visualstudio.com/items?itemName=ms-kubernetes-tools.vscode-kubernetes-tools) (if that is doable in Theia)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Preferred the K8s Dashboard


### Container Registry
* Open Source Container Registry should become its own KubeOne AddOn. Harbor should be supported, may other Open Source Container Registries can be considered.

### GitOps
* FluxCD and/or ArgoCD should get supported. They should become its own KubeOne AddOn. Their controllers should get installed into the KubeOne cluster. We should take care that the Git Repo holding the yaml files for the Kubernetes Cluster can be configured easily.

### Vault
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would consider Vault for a later stage.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH, I wouldn't consider Vault at all. Vault is in fact a very complex component if you want to set it up in a proper way. You mostly like want to integrate with your cloud provider KMS solution, secure your keys, etc... Abstracting that away from the user can create a huge security hole.

* Vault should become its own KubeOne AddOn. Developers should have the possibility to store sensitive data. Attention has to be paid to the persistence of this data.

## Alternatives Considered

## Task & Effort

Rough Estimate:
* [3] IDE
* [1] Terminal integrations
* [3] Security
* [1] Visualization of the Kubernetes Cluster
* [3] Container Registry
* [3] GitOps
* [2] Vault
-----------
16 MD