Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hkatz/dashboard and summary overhaul #180

Merged
Merged
Changes from 62 commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
73f3246
Revert "Make units consistent for memory fixes issue #78 (#103)"
Feb 28, 2020
6ad3a58
Add FormatResourceList and test cases
while1malloc0 Nov 6, 2019
091491f
Use v1 resource memory constants instead of hardcoded strings
while1malloc0 Nov 7, 2019
14442ea
Merge pull request #1 from FairwindsOps/master
hjkatz Mar 6, 2020
5e72ef0
Merge pull request #2 from FairwindsOps/master
hjkatz Mar 16, 2020
85dae07
Add a VPAUpdateMode to the vpa.Reconciler
while1malloc0 Nov 7, 2019
ab9148e
Add ability to specify vpa update mode
while1malloc0 Nov 8, 2019
69f688e
Extract variables instead of hardcoded values
while1malloc0 Nov 11, 2019
dce56fc
VPA update mode label case insensitive
while1malloc0 Nov 12, 2019
5b16cbd
Summary fix for multiple containers in a pod (#123)
Mar 17, 2020
67e89ee
go.mod updated for some reason
Mar 17, 2020
5ebd109
Add support for all VPA UpdateModes
Mar 17, 2020
3d59677
Use ParseBool for deployment enabled label value
Mar 17, 2020
bc50617
Use ParseBool for namespace enabled label value
Mar 17, 2020
701411a
Merge branch 'master' into hkatz/add-vpa-update-mode
Mar 17, 2020
ac94d48
Use lowercase for switch statement
Mar 18, 2020
eb83cb6
Merge pull request #3 from hjkatz/hkatz/add-vpa-update-mode
hjkatz Mar 18, 2020
1e14372
Merge pull request #4 from FairwindsOps/master
hjkatz Mar 24, 2020
65d1340
Add Deployment Exclusion Annotation
Nov 15, 2019
662477b
Change no-vpa annotation to vpa-opt-out (to opt out of vpa in auto mode)
Nov 15, 2019
a7e18b2
Add more logging to VPA reconciler
Nov 20, 2019
c57d004
Fix namespace logs to use Name instead of the full Namespace
Nov 20, 2019
563d78d
Update README
Mar 24, 2020
8035039
Fix Typo aut -> auto
hjkatz Mar 25, 2020
01f6610
Fix grammar matched -> match
hjkatz Mar 25, 2020
ef29113
Use vpa-update-mode as an annotation for Deployment specific VPA control
Mar 30, 2020
114565d
Fix ineffectual assignment
Mar 30, 2020
828c09d
Merge branch 'master' into hkatz/vpa-deployment-exclusion-and-updates
Mar 30, 2020
a68fe5f
Fix test label
Apr 6, 2020
8576ca6
Use index to get correct vpa
Apr 6, 2020
a3b8fdb
Use short-hand if statement for map lookup
Apr 6, 2020
3f04ff6
Merge pull request #5 from hjkatz/hkatz/vpa-deployment-exclusion-and-…
Apr 6, 2020
1fecfc4
Merge pull request #6 from FairwindsOps/master
Apr 13, 2020
f421278
Merge pull request #7 from FairwindsOps/master
May 4, 2020
102aded
Merge pull request #8 from FairwindsOps/master
May 11, 2020
480b0ee
Split templates by kind
Nov 20, 2019
7a1c806
Restructure/Organize pkg/dashboard into sub-packages
Nov 21, 2019
753e725
Add options pattern to pkg/dashboard
Nov 21, 2019
27963da
Move handlers pkg back to dashboard ; Move dashboard.go -> router.go
Nov 22, 2019
3ae5f66
Change Summary format to better represent its logical data structure
Nov 22, 2019
913d1c2
Change templates to work with new Summary structure
Nov 22, 2019
c01df28
Fix container resource recommendation formatting
Nov 22, 2019
e07be78
Move pkg/dashboard -> pkg/web
Nov 26, 2019
2ed086b
Split templates and dashboard into files ; Make template util functio…
Nov 27, 2019
6f78ba4
Add /namespaces ; Make /namespaces the default route ; Style /namespa…
Nov 27, 2019
f774289
Limit namespace-list.go data passed to template
Nov 27, 2019
dc52aa5
Fix dashboard link
Nov 27, 2019
0cc37e9
Add length to All Namespaces
Nov 27, 2019
fe04ae1
Fix summary_test.go
Nov 27, 2019
51d9d15
Fix typo 'reccomendations' -> 'recommendations'
Nov 27, 2019
ce751e0
Fix typo 'NamespceList' -> 'NamespaceList'
Nov 27, 2019
668e492
Add clarifying comment about template data limiting
Nov 27, 2019
004ed52
Add outer structure to Summary
Nov 27, 2019
01d2283
Rename dashboard -> web
Jun 28, 2020
1627acc
Allow goldilocks' web RBAC to list/get namespaces
Jun 28, 2020
9cddf35
Move common labels to utils.go
Jun 28, 2020
855681c
Use correct labels location
Jun 28, 2020
216806a
Attempt to fix e2e
Jun 28, 2020
b885817
Fix incorrect args usage for namespace
Jun 28, 2020
33297c9
Add oidc auth support
Jun 28, 2020
5db0769
Remove redundant return statement from health.go
Jun 28, 2020
7b2f783
Merge branch 'master' into hkatz/dashboard-and-summary-overhaul
Jul 20, 2020
cb0bb73
Update cmd/web.go
Jul 21, 2020
eae73cd
Rename web back to -> dashboard
Jul 22, 2020
5fa8536
Merge branch 'master' into hkatz/dashboard-and-summary-overhaul
Aug 5, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -49,7 +49,7 @@ Prerequisites:
* Install the project with `go get github.com/fairwindsops/goldilocks`
* Change into the goldilocks directory which is installed at `$GOPATH/src/github.com/fairwindsops/goldilocks`
* Use `make tidy` or `make build` to ensure all dependencies are downloaded.
* See the dashboard with `go run main.go dashboard`, then open http://localhost:8080/. This assumes that you have a working KUBECONFIG in place with access to a cluster.
* See the web ui with `go run main.go web`, then open http://localhost:8080/. This assumes that you have a working KUBECONFIG in place with access to a cluster.
sudermanjr marked this conversation as resolved.
Show resolved Hide resolved

### End-To-End Tests

20 changes: 10 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<div align="center">
<img src="/pkg/dashboard/assets/images/goldilocks.svg" height="150" alt="Goldilocks" style="padding-bottom: 20px" />
<img src="/pkg/web/assets/images/goldilocks.svg" height="150" alt="Goldilocks" style="padding-bottom: 20px" />
<br>

<h3>Get your resource requests "Just Right"</h3>
@@ -64,12 +64,12 @@ helm install --name goldilocks --namespace goldilocks fairwinds-stable/goldilock

### Method 2 - Manifests

The [hack/manifests](hack/manifests) directory contains collections of Kubernetes YAML definitions for installing the controller and dashboard components in cluster.
The [hack/manifests](hack/manifests) directory contains collections of Kubernetes YAML definitions for installing the controller and web components in cluster.

```
kubectl create namespace goldilocks
kubectl -n goldilocks apply -f hack/manifests/controller
kubectl -n goldilocks apply -f hack/manifests/dashboard
kubectl -n goldilocks apply -f hack/manifests/web
```

### Enable Namespace
@@ -82,12 +82,12 @@ kubectl label ns goldilocks goldilocks.fairwinds.com/enabled=true

After that you should start to see VPA objects in that namespace.

### Viewing the Dashboard
### Viewing the Web UI

The default installation creates a ClusterIP service for the dashboard. You can access via port forward:

```
kubectl -n goldilocks port-forward svc/goldilocks-dashboard 8080:80
kubectl -n goldilocks port-forward svc/goldilocks-web 8080:80
```

Then open your browser to [http://localhost:8080](http://localhost:8080)
@@ -108,7 +108,7 @@ Usage:
Available Commands:
controller Run goldilocks as a controller inside a kubernetes cluster.
create-vpas Create VPAs
dashboard Run the goldilocks dashboard that will show recommendations.
web Run the goldilocks web that will show recommendations.
delete-vpas Delete VPAs
help Help about any command
summary Genarate a summary of the vpa recommendations in a namespace.
@@ -182,11 +182,11 @@ This will search for any deployments in the given namespace and generate a VPA f

This will delete all vpa objects in a namespace that are labelled for use by this tool.

### dashboard
### web

`goldilocks dashboard`
`goldilocks web`

Runs the goldilocks dashboard web server that will display recommendations. Listens on port `8080` by default.
Runs the goldilocks web server that will display recommendations. Listens on port `8080` by default.

### summary

@@ -196,7 +196,7 @@ Queries all the VPA objects that are labelled for this tool across all namespace

### Container Exclusions

The `dashboard` and `summary` commands can exclude recommendations for a list of comma separated container names using the `--exclude-containers` argument. This option can be useful for hiding recommendations for sidecar containers for things like Linkerd and Istio.
The `web` and `summary` commands can exclude recommendations for a list of comma separated container names using the `--exclude-containers` argument. This option can be useful for hiding recommendations for sidecar containers for things like Linkerd and Istio.

Containers can be excluded for individual deployments by applying a label to any deployment. The label value should be a list of comma separated container names. The label value will be combined with any values provided through the `--exclude-containers` argument.

49 changes: 0 additions & 49 deletions cmd/dashboard.go

This file was deleted.

2 changes: 1 addition & 1 deletion cmd/summary.go
Original file line number Diff line number Diff line change
@@ -49,7 +49,7 @@ By default the summary will be about all VPAs in all namespaces.`,

// limit to a single namespace
if namespace != "" {
opts = append(opts, summary.ForNamespace(args[0]))
opts = append(opts, summary.ForNamespace(namespace))
}

// exclude containers from the summary
53 changes: 53 additions & 0 deletions cmd/web.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
// Copyright 2019 FairwindsOps Inc
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package cmd

import (
"fmt"
"net/http"
"strings"

"github.com/spf13/cobra"
"k8s.io/apimachinery/pkg/util/sets"
"k8s.io/klog"

"github.com/fairwindsops/goldilocks/pkg/web"
)

var serverPort int
var basePath string

func init() {
rootCmd.AddCommand(webCmd)
webCmd.PersistentFlags().IntVarP(&serverPort, "port", "p", 8080, "The port to serve the web ui on.")
webCmd.PersistentFlags().StringVar(&basePath, "base-path", "/", "Path on which the web ui is served")
webCmd.PersistentFlags().StringVarP(&excludeContainers, "exclude-containers", "e", "", "Comma delimited list of containers to exclude from recommendations.")
}

var webCmd = &cobra.Command{
Use: "web",
Short: "Run the goldilocks web ui that will show recommendations.",
Long: `Run the goldilocks web ui that will show recommendations.`,
sudermanjr marked this conversation as resolved.
Show resolved Hide resolved
sudermanjr marked this conversation as resolved.
Show resolved Hide resolved
Run: func(cmd *cobra.Command, args []string) {
router := web.GetRouter(
web.OnPort(serverPort),
web.WithBasePath(basePath),
web.ExcludeContainers(sets.NewString(strings.Split(excludeContainers, ",")...)),
)
http.Handle("/", router)
klog.Infof("Starting goldilocks web ui server on port %d", serverPort)
klog.Fatalf("%v", http.ListenAndServe(fmt.Sprintf(":%d", serverPort), nil))
},
}
6 changes: 3 additions & 3 deletions e2e/pre.sh
Original file line number Diff line number Diff line change
@@ -25,12 +25,12 @@ docker tag "quay.io/fairwinds/goldilocks:latest" "quay.io/fairwinds/goldilocks:$
docker images
kind load docker-image "quay.io/fairwinds/goldilocks:$CI_SHA1" --name "e2e"

yq w -i hack/manifests/dashboard/deployment.yaml spec.template.spec.containers[0].imagePullPolicy "Never"
yq w -i hack/manifests/web/deployment.yaml spec.template.spec.containers[0].imagePullPolicy "Never"
yq w -i hack/manifests/controller/deployment.yaml spec.template.spec.containers[0].imagePullPolicy "Never"
yq w -i hack/manifests/dashboard/deployment.yaml spec.template.spec.containers[0].image "quay.io/fairwinds/goldilocks:$CI_SHA1"
yq w -i hack/manifests/web/deployment.yaml spec.template.spec.containers[0].image "quay.io/fairwinds/goldilocks:$CI_SHA1"
yq w -i hack/manifests/controller/deployment.yaml spec.template.spec.containers[0].image "quay.io/fairwinds/goldilocks:$CI_SHA1"

cat hack/manifests/dashboard/deployment.yaml
cat hack/manifests/web/deployment.yaml
cat hack/manifests/controller/deployment.yaml

docker cp . e2e-command-runner:/goldilocks
4 changes: 2 additions & 2 deletions e2e/tests/00_setup.yaml
Original file line number Diff line number Diff line change
@@ -28,8 +28,8 @@ testcases:
steps:
- script: kubectl create ns goldilocks
- script: |
kubectl -n goldilocks apply -f ../../hack/manifests/dashboard/
kubectl -n goldilocks apply -f ../../hack/manifests/web/
kubectl -n goldilocks apply -f ../../hack/manifests/controller/
- script: |
kubectl -n goldilocks wait deployment --timeout={{.timeout}} --for condition=available -l app.kubernetes.io/name=goldilocks,app.kubernetes.io/component=dashboard
kubectl -n goldilocks wait deployment --timeout={{.timeout}} --for condition=available -l app.kubernetes.io/name=goldilocks,app.kubernetes.io/component=web
kubectl -n goldilocks wait deployment --timeout={{.timeout}} --for condition=available -l app.kubernetes.io/name=goldilocks,app.kubernetes.io/component=controller
2 changes: 1 addition & 1 deletion hack/kind/setup.sh
Original file line number Diff line number Diff line change
@@ -70,7 +70,7 @@ if $install_goldilocks; then
## Install Goldilocks
kubectl get ns goldilocks || kubectl create ns goldilocks
kubectl -n goldilocks apply -f ../manifests/controller
kubectl -n goldilocks apply -f ../manifests/dashboard
kubectl -n goldilocks apply -f ../manifests/web
fi

echo "Your test environment should now be running."
Original file line number Diff line number Diff line change
@@ -2,7 +2,7 @@
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: goldilocks-dashboard
name: goldilocks-web
labels:
app: goldilocks
rules:
@@ -20,3 +20,10 @@ rules:
verbs:
- 'get'
- 'list'
- apiGroups:
- '' # core
resources:
- 'namespaces'
verbs:
- 'get'
- 'list'
Original file line number Diff line number Diff line change
@@ -2,14 +2,14 @@
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: goldilocks-dashboard
name: goldilocks-web
labels:
app: goldilocks
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: goldilocks-dashboard
name: goldilocks-web
subjects:
- kind: ServiceAccount
name: goldilocks-dashboard
name: goldilocks-web
namespace: goldilocks
Original file line number Diff line number Diff line change
@@ -2,30 +2,30 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: goldilocks-dashboard
name: goldilocks-web
labels:
app.kubernetes.io/name: goldilocks
app.kubernetes.io/component: dashboard
app.kubernetes.io/component: web
spec:
replicas: 2
selector:
matchLabels:
app.kubernetes.io/name: goldilocks
app.kubernetes.io/component: dashboard
app.kubernetes.io/component: web
template:
metadata:
labels:
app.kubernetes.io/name: goldilocks
app.kubernetes.io/component: dashboard
app.kubernetes.io/component: web
spec:
serviceAccountName: goldilocks-dashboard
serviceAccountName: goldilocks-web
containers:
- name: goldilocks
image: "quay.io/fairwinds/goldilocks:master"
imagePullPolicy: Always
command:
- /goldilocks
- dashboard
- web
- --exclude-containers=linkerd-proxy,istio-proxy
- -v3
securityContext:
Original file line number Diff line number Diff line change
@@ -2,10 +2,10 @@
apiVersion: v1
kind: Service
metadata:
name: goldilocks-dashboard
name: goldilocks-web
labels:
app.kubernetes.io/name: goldilocks
app.kubernetes.io/component: dashboard
app.kubernetes.io/component: web
spec:
type: ClusterIP
ports:
@@ -15,4 +15,4 @@ spec:
name: http
selector:
app.kubernetes.io/name: goldilocks
app.kubernetes.io/component: dashboard
app.kubernetes.io/component: web
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: goldilocks-dashboard
name: goldilocks-web
labels:
app: goldilocks
Loading