Add Helm chart for installing Kubeapps #349

prydonius · 2018-06-07T01:31:11Z

fixes #325

Todos:

values for configuring TLS with tiller-proxy

prydonius · 2018-07-21T19:15:52Z

@arapulido the latest changes should be testable, you'll need to install Tiller using helm init and build your own images for the dashboard and apprepository-controller

arapulido · 2018-07-23T10:16:49Z

@prydonius I have checked this and I am getting a CRD validation error:

helm install --set apprepository.image.repository=arapulido/apprepository-controller --set dashboard.image.repository=arapulido/dashboard ./kubeapps
Error: apiVersion "kubeapps.com/v1alpha1" in kubeapps/templates/apprepositories.yaml is not available

I have been checking around but it seems that there is not yet a solution for this. Did you encounter this error? How did you fix it?

prydonius · 2018-07-24T02:09:00Z

Oh woops, I forgot to mention you also need the RC Helm version: https://github.com/helm/helm/releases/tag/v2.10.0-rc.1, it adds support for a CRD hook. I was hoping for this to be released this week, but if not it will be a blocker for our release or we will need a workaround.

arapulido · 2018-07-24T10:18:53Z

chart/kubeapps/templates/NOTES.txt

+  export SERVICE_IP=$(kubectl get svc --namespace {{ .Release.Namespace }} {{ template "kubeapps.fullname" . }} -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
+  echo http://$SERVICE_IP:{{ .Values.frontend.service.port }}
+{{- else if contains "ClusterIP" .Values.frontend.service.type }}
+  export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app={{ template "kubeapps.name" . }},release={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}")


The pod get the app label as .Release.Name-.kubeapps.name, instead of just kubeapps, so the command shown here will get 0 pods

fixed, thanks

arapulido · 2018-07-24T10:21:32Z

chart/kubeapps/templates/NOTES.txt

+{{- else if contains "ClusterIP" .Values.frontend.service.type }}
+  export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app={{ template "kubeapps.name" . }},release={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}")
+  echo "Visit http://127.0.0.1:8080 to use your application"
+  kubectl port-forward $POD_NAME 8080:80


The pod is listening in the port 8080, so the command above should be kubectl port-forward $POD_NAME 8080

fixed, thanks

arapulido · 2018-07-24T10:32:59Z

App Repositories view is empty, even though the charts are loading fine:

There are no errors in the console or the logs.

Adding a new repo fails, this shows in the console:

TypeError: "e.metadata is undefined"
renderhttp://localhost:8080/static/js/main.6090e182.js:1:3562936renderhttp://localhost:8080/static/js/main.6090e182.js:1:3562890lhttp://localhost:8080/static/js/main.6090e182.js:1:1056655beginWorkhttp://localhost:8080/static/js/main.6090e182.js:1:1058377ihttp://localhost:8080/static/js/main.6090e182.js:1:1070502ahttp://localhost:8080/static/js/main.6090e182.js:1:1070823shttp://localhost:8080/static/js/main.6090e182.js:1:1071061whttp://localhost:8080/static/js/main.6090e182.js:1:1074605yhttp://localhost:8080/static/js/main.6090e182.js:1:1074342hhttp://localhost:8080/static/js/main.6090e182.js:1:1073470phttp://localhost:8080/static/js/main.6090e182.js:1:1072860enqueueSetStatehttp://localhost:8080/static/js/main.6090e182.js:1:1047385setStatehttp://localhost:8080/static/js/main.6090e182.js:1:1026654onStateChangehttp://localhost:8080/static/js/main.6090e182.js:1:241229notifyhttp://localhost:8080/static/js/main.6090e182.js:1:1125085notifyNestedSubshttp://localhost:8080/static/js/main.6090e182.js:1:1125602onStateChangehttp://localhost:8080/static/js/main.6090e182.js:1:241247uhttp://localhost:8080/static/js/main.6090e182.js:1:243585nhttp://localhost:8080/static/js/main.6090e182.js:1:1142948nhttp://localhost:8080/static/js/main.6090e182.js:1:1738533dispatchhttp://localhost:8080/static/js/main.6090e182.js:1:1132779Phttp://localhost:8080/static/js/main.6090e182.js:1:1199752nhttp://localhost:8080/static/js/main.6090e182.js:1:1197258rhttp://localhost:8080/static/js/main.6090e182.js:1:1196595ahttp://localhost:8080/static/js/main.6090e182.js:1:1196335

arapulido · 2018-07-24T10:38:05Z

Installing charts doesn't work for me either

arapulido · 2018-07-24T14:56:01Z

@tompizmor @javsalgar It would be very useful if you start reviewing this as well, thanks!

prydonius · 2018-07-24T23:42:24Z

Hmm, thanks @arapulido. Seems to be an issue when building the docker image for the Dashboard, because with Telepresence the AppRepositories work for me.

With the image you built, I can see that the namespace it retrieves from the config is undefined: http://localhost:8080/api/kube/apis/kubeapps.com/v1alpha1/namespaces/undefined/apprepositories. This is also why installing charts doesn't work, because it tries to fetch the AppRepository but the namespace is not being found. Will continue looking into it.

also renames API URL method to getResourceLink

prydonius · 2018-07-25T00:13:17Z

The issue was the ConfigMap was incorrect JSON (oops!), when using telepresence it uses the local config.json which was correct.

Also the creation error for AppRepositories was due to using the wrong resource link, fixed that too.

Note, when doing a Helm upgrade, please make sure you set the same password for MongoDB, otherwise it will get regenerated (known Helm issue - helm/helm#3053) and break chart syncs and the chartsvc:

$ kubectl get secret $RELEASE_NAME-mongodb -o jsonpath="{.data.mongodb-root-password}" | base64 --decode
$ helm upgrade $RELEASE_NAME --set apprepository.image.repository=arapulido/apprepository-controller --set dashboard.image.repository=arapulido/dashboard --set mongodb.mongodbRootPassword=$MONGO_PASSWORD ./kubeapps

arapulido · 2018-07-25T09:00:14Z

@prydonius I have rebuilt and deployed the chart in a new cluster, and now the Dashboard doesn't get the login form, but tries directly to get to the App list (getting a 401 in response)

arapulido · 2018-07-26T14:46:52Z

@prydonius I have tested the chart and with the cluster-admin role things seem to work, but as soon as I create a serviceaccount with less permissions, it seems that we need to create the roles manually (when with kubeapps up they are created automatically). It looks like all RBAC roles are missing: https://github.com/kubeapps/kubeapps/blob/master/manifests/user-roles.jsonnet

I get this error when deploying to the namespace my serviceaccount has write access to:

that also talks about the kubeapps namespace, which is wrong in this case.

arapulido · 2018-07-26T14:53:40Z

Once I created a role to read apprepositories in the default namespace and applied it in a binding to the serviceaccount I was using, I was able to deploy the chart to my namespace

tompizmor · 2018-07-27T13:36:47Z

chart/kubeapps/Chart.yaml

@@ -0,0 +1,5 @@
+apiVersion: v1


Should you also add the maintainers and other metadata like home, sources and icon?

yes, I'll do that, thanks!

tompizmor · 2018-07-27T13:38:18Z

chart/kubeapps/templates/apprepository-deployment.yaml

+metadata:
+  name: {{ template "kubeapps.apprepository.fullname" . }}
+  labels:
+    app: {{ template "kubeapps.apprepository.fullname" . }}


I see that when there is an issue deploying a chart from kubeapps the following message is show:

Check the health of Kubeapps components 'kubectl get po --all-namespaces -l created-by=kubeapps'.

However, it seems any pod created by the chart has the label created-by=kubeapps

Where are you seeing that? I thought that was from the kubeapps CLI, it's not in the NOTES.txt?

Oh sorry, it's actually shown in the UI, will fix that thanks. Note you'll need to build images for the apprepository-controller and dashboard to make this actually work.

yes, I saw that Ara already built the images so I used the same ones

tompizmor · 2018-07-27T13:45:21Z

chart/kubeapps/values.yaml

+    repository: kubeapps/chart-repo
+    tag: latest
+  repos:
+  - name: stable


Have you thought about also adding the bitnami charts repo here?

tompizmor · 2018-07-27T13:47:49Z

chart/kubeapps/templates/apprepository-deployment.yaml

+      serviceAccountName: {{ template "kubeapps.apprepository.fullname" . }}
+      containers:
+      - name: controller
+        image: "{{ .Values.apprepository.image.repository }}:{{ .Values.apprepository.image.tag }}"


do you think it makes sense to also allow specifying the registry for the container image?

Maybe we should do it for consistency, but I was just going off the helm create template. TBH I don't really like adding the registry value, because I don't think the value needs to be so fine-grained, setting repository=myregistry.com/apprepository-controller works fine right?

It's also confusing if you're used to the upstream charts that just have the repository value, if we had registry default to docker.io, then the above would actually render docker.io/myregistry.com/apprepository-controller which would be incorrect.

In other charts the image is set using a template. This could be easier down the line if we want to change the way we set the images ("registry, repository, image" or "repository,image")

@javsalgar I couldn't find where the template was used, we seem to be just doing this: https://github.com/helm/charts/blob/master/stable/drupal/templates/deployment.yaml#L27. In any case, since this chart will eventually be going into bitnami/charts I am going to update it for consistency

andresmgot

I gave this a test and it worked as expected. For Kubeless I would suggest that instead of adding it as a subchart I would do the same that we do with the service catalog: Detect that it's missing and point to the kubeless chart that is in the incubator registry. If that gets complicated for any reason we should show a custom error message in the Functions view.

Let me know when you want me to take a look to this again!

andresmgot · 2018-07-30T14:02:07Z

chart/kubeapps/templates/NOTES.txt

+{{- else if contains "ClusterIP" .Values.frontend.service.type }}
+  export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app={{ template "kubeapps.fullname" . }},release={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}")
+  echo "Visit http://127.0.0.1:8080 to use your application"
+  kubectl port-forward $POD_NAME 8080:8080


kubectl port-forward --namespace {{ .Release.Namespace}} $POD_NAME 8080:8080

ah good catch, thanks!

Is that what technically kubeapps dashboard used to do?

migmartri

Does helm support subdirectories under templates/ i.e apprepository/ chart-svc/`

Not for now, but I am curious if it would make sense to make them separated charts and the dashboard imports them as subcharts. As I said, not important since nobody is going to re-use them for now and we are going to package them all at the same time but food for thought.

migmartri · 2018-07-30T22:36:45Z

cmd/apprepository-controller/main.go

@@ -75,4 +78,7 @@ func init() {
 	flag.StringVar(&kubeconfig, "kubeconfig", "", "Path to a kubeconfig. Only required if out-of-cluster.")
 	flag.StringVar(&masterURL, "master", "", "The address of the Kubernetes API server. Overrides any value in kubeconfig. Only required if out-of-cluster.")
 	flag.StringVar(&repoSyncImage, "repo-sync-image", "kubeapps/chart-repo:latest", "container repo/image to use in CronJobs")
+	flag.StringVar(&namespace, "namespace", "kubeapps", "Namespace to discover AppRepository resources")


May I ask why these changes are related to the creation of a chart? Couldn't this have been done in another previous patch?

Just trying to understand the dynamics :)

This was needed because previously apprepository-controller was hardcoded to use the kubeapps namespace which was fine as kubeapps up didn't make that configurable. The chart can be installed in any namespace though (although we could hardcode it, but bad practice), so this is needed for that to work.

prydonius · 2018-07-30T22:49:25Z

@migmartri good points, I don't know if Helm supports directories inside templates, but another option is to just have the subcharts inside the charts/ dir separating them as you suggest. I thought about it, but figured for now it wasn't that important and probably adds more boilerplate. Simply adding directories under templates/ though might be nice to clean it up a bit more.

- update hardcoded references to `kubeapps` namespace to variable Kubeapps installation namespace - remove Kubeless roles as Kubeapps n longer bundles it and so is unable to create the necessary Roles in the `kubeless` namespace. The permission error alerts in Kubeapps should be sufficient for users using Kubeless within Kubeapps

prydonius · 2018-07-30T23:53:25Z

#349 (comment)

that also talks about the kubeapps namespace, which is wrong in this case.

@arapulido I will fix this in a separate PR as I need to rework the way we get the namespace in the dashboard a bit and don't want to add more to this. For now, this PR functionally works and so I think we can merge as-is.

I believe I've addressed all the other review changes.

Things to address in other PRs:

Removing Kubeless from menu (and pointer to incubator Helm chart for kubeless if you navigate to /functions)
Fixing error alerts for Kubeapps namespace and command to check health of Kubeapps components

andresmgot · 2018-07-31T08:28:48Z

docs/user/access-control.md

 ```

 #### Write access to App Repositories

 In order to create and refresh App Repositories in Kubeapps, apply the
-`kubeapps-repositories-write` Role in the kubeapps namespace.
+`kubeapps-repositories-write` Role in the namespace KUbeapps is installed in.


s/KUbeapps/Kubeapps

oops, thanks

andresmgot · 2018-07-31T08:38:02Z

For me I am okay to merge this PR as it is and open different ones to address the items missing to avoid more and a bigger PR.

javsalgar · 2018-07-31T13:22:37Z

chart/kubeapps/templates/NOTES.txt

+  echo http://$NODE_IP:$NODE_PORT
+{{- else if contains "LoadBalancer" .Values.frontend.service.type }}
+     NOTE: It may take a few minutes for the LoadBalancer IP to be available.
+           You can watch the status of by running 'kubectl get svc -w {{ template "kubeapps.fullname" . }}'


Remove "of"

javsalgar · 2018-07-31T13:28:05Z

chart/kubeapps/templates/apprepository-deployment.yaml

+      serviceAccountName: {{ template "kubeapps.apprepository.fullname" . }}
+      containers:
+      - name: controller
+        image: "{{ .Values.apprepository.image.repository }}:{{ .Values.apprepository.image.tag }}"


In other charts the image is set using a template. This could be easier down the line if we want to change the way we set the images ("registry, repository, image" or "repository,image")

javsalgar · 2018-07-31T13:34:55Z

chart/kubeapps/templates/kubeapps-frontend-deployment.yaml

+          httpGet:
+            path: /healthz
+            port: 8080
+          initialDelaySeconds: 60


In our production-ready infrastructure assets, these kind of values go in values.yaml

right! We usually export health checks settings so users can fine tune them

sameersbn · 2018-08-01T09:49:33Z

@prydonius you seem to have added the tarball charts/mongodb-4.0.4.tgz to PR.

prydonius · 2018-08-01T18:42:00Z

@sameersbn I was thinking of vendoring it, but I'll remove it

migmartri · 2018-08-01T20:25:23Z

chart/kubeapps/values.yaml

+    registry: docker.io
+    repository: kubeapps/chart-repo
+    tag: latest
+  jobsImage:


Add comment explaining that this is to perform a workaround.

Do we run it even if you are using the version of helm that does not require the workaround?

We do run it yes, this workaround works with all versions of Helm, and I don't know if we can detect the new version easily enough to change the way we do this (also it's still not yet released).

@migmartri fixed

…sage

migmartri · 2018-08-02T23:00:30Z

chart/kubeapps/templates/apprepository-jobs-bootstrap-rbac.yaml

+  - delete
+---
+apiVersion: rbac.authorization.k8s.io/v1beta1
+kind: RoleBinding


Add hook annotations.

migmartri · 2018-08-02T23:10:14Z

chart/kubeapps/templates/dashboard-config.yaml

+    chart: {{ template "kubeapps.chart" . }}
+    release: {{ .Release.Name }}
+    heritage: {{ .Release.Service }}
+data:


Eventually it might make sense to bake this configuration in the image.

Adnan Abdulhussein added 4 commits July 12, 2018 15:10

apprepo controller changes to support different mongo host

08cd563

add WIP chart

a84af5a

all working components for kubeapps chart

a7c143f

dashboard changes for discovering installation namespace

4712447

prydonius force-pushed the helm-chart branch from 3453a88 to 4712447 Compare July 21, 2018 02:29

arapulido reviewed Jul 24, 2018

View reviewed changes

Adnan Abdulhussein added 2 commits July 24, 2018 16:57

fix dashboard config JSON in chat configmap

4ae5cd8

fix AppRepository.create to use namespace from config

1378a22

also renames API URL method to getResourceLink

fix NOTES.txt port-forward instructions

4b073f5

tompizmor reviewed Jul 27, 2018

View reviewed changes

Adnan Abdulhussein added 2 commits July 27, 2018 07:53

add apprepository bootstrap post-install job

8a293da

add tiller TLS values

c0530e9

andresmgot reviewed Jul 30, 2018

View reviewed changes

migmartri reviewed Jul 30, 2018

View reviewed changes

Adnan Abdulhussein added 2 commits July 30, 2018 15:51

add namespace to port-forward instructions

5cac791

add preset RBAC roles for users

ba30bfd

prydonius changed the title ~~[WIP] Add Helm chart for installing Kubeapps~~ Add Helm chart for installing Kubeapps Jul 30, 2018

add chart metadata

a808efd

prydonius mentioned this pull request Jul 31, 2018

Fix hardcoded references to kubeapps namespace in Kubeapps UI #415

Closed

andresmgot reviewed Jul 31, 2018

View reviewed changes

javsalgar reviewed Jul 31, 2018

View reviewed changes

Adnan Abdulhussein added 5 commits July 31, 2018 09:59

fix typo in access-control doc

469f00a

fix typo in NOTES.txt

bd583b7

add registry option for image

781ae64

add support for modifying liveness and readiness probes with values

8b00899

update mongodb to 4.0.4

a215485

unvendor mongodb chart and update requirements.lock

4784987

migmartri reviewed Aug 2, 2018

View reviewed changes

Adnan Abdulhussein added 2 commits August 2, 2018 15:45

add comment explaining apprepository.jobsImage value

5bffdf6

rename apprepository.repos to apprepository.initialRepos to clarify u…

3f7a80f

…sage

migmartri approved these changes Aug 2, 2018

View reviewed changes

add missing hook annotations to apprepo job rolebinding

d720a31

prydonius merged commit d9102c6 into vmware-tanzu:master Aug 2, 2018

prydonius deleted the helm-chart branch August 2, 2018 23:43

Add Helm chart for installing Kubeapps #349

Add Helm chart for installing Kubeapps #349

Conversation

prydonius commented Jun 7, 2018 • edited Loading

prydonius commented Jul 21, 2018

arapulido commented Jul 23, 2018

prydonius commented Jul 24, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

arapulido commented Jul 24, 2018 • edited Loading

arapulido commented Jul 24, 2018

arapulido commented Jul 24, 2018

prydonius commented Jul 24, 2018

prydonius commented Jul 25, 2018

arapulido commented Jul 25, 2018

arapulido commented Jul 26, 2018

arapulido commented Jul 26, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

prydonius Jul 27, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andresmgot left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

migmartri left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

prydonius commented Jul 30, 2018

prydonius commented Jul 30, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andresmgot commented Jul 31, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sameersbn commented Aug 1, 2018

prydonius commented Aug 1, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

prydonius commented Jun 7, 2018 •

edited

Loading

arapulido commented Jul 24, 2018 •

edited

Loading

prydonius Jul 27, 2018 •

edited

Loading

prydonius commented Jul 30, 2018 •

edited

Loading

andresmgot commented Jul 31, 2018 •

edited

Loading