Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hkatz/batch summary deployments api call #194

Merged
Merged
Changes from all commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
73f3246
Revert "Make units consistent for memory fixes issue #78 (#103)"
Feb 28, 2020
6ad3a58
Add FormatResourceList and test cases
while1malloc0 Nov 6, 2019
091491f
Use v1 resource memory constants instead of hardcoded strings
while1malloc0 Nov 7, 2019
14442ea
Merge pull request #1 from FairwindsOps/master
hjkatz Mar 6, 2020
5e72ef0
Merge pull request #2 from FairwindsOps/master
hjkatz Mar 16, 2020
85dae07
Add a VPAUpdateMode to the vpa.Reconciler
while1malloc0 Nov 7, 2019
ab9148e
Add ability to specify vpa update mode
while1malloc0 Nov 8, 2019
69f688e
Extract variables instead of hardcoded values
while1malloc0 Nov 11, 2019
dce56fc
VPA update mode label case insensitive
while1malloc0 Nov 12, 2019
5b16cbd
Summary fix for multiple containers in a pod (#123)
Mar 17, 2020
67e89ee
go.mod updated for some reason
Mar 17, 2020
5ebd109
Add support for all VPA UpdateModes
Mar 17, 2020
3d59677
Use ParseBool for deployment enabled label value
Mar 17, 2020
bc50617
Use ParseBool for namespace enabled label value
Mar 17, 2020
701411a
Merge branch 'master' into hkatz/add-vpa-update-mode
Mar 17, 2020
ac94d48
Use lowercase for switch statement
Mar 18, 2020
eb83cb6
Merge pull request #3 from hjkatz/hkatz/add-vpa-update-mode
hjkatz Mar 18, 2020
1e14372
Merge pull request #4 from FairwindsOps/master
hjkatz Mar 24, 2020
65d1340
Add Deployment Exclusion Annotation
Nov 15, 2019
662477b
Change no-vpa annotation to vpa-opt-out (to opt out of vpa in auto mode)
Nov 15, 2019
a7e18b2
Add more logging to VPA reconciler
Nov 20, 2019
c57d004
Fix namespace logs to use Name instead of the full Namespace
Nov 20, 2019
563d78d
Update README
Mar 24, 2020
8035039
Fix Typo aut -> auto
hjkatz Mar 25, 2020
01f6610
Fix grammar matched -> match
hjkatz Mar 25, 2020
ef29113
Use vpa-update-mode as an annotation for Deployment specific VPA control
Mar 30, 2020
114565d
Fix ineffectual assignment
Mar 30, 2020
828c09d
Merge branch 'master' into hkatz/vpa-deployment-exclusion-and-updates
Mar 30, 2020
a68fe5f
Fix test label
Apr 6, 2020
8576ca6
Use index to get correct vpa
Apr 6, 2020
a3b8fdb
Use short-hand if statement for map lookup
Apr 6, 2020
3f04ff6
Merge pull request #5 from hjkatz/hkatz/vpa-deployment-exclusion-and-…
Apr 6, 2020
1fecfc4
Merge pull request #6 from FairwindsOps/master
Apr 13, 2020
f421278
Merge pull request #7 from FairwindsOps/master
May 4, 2020
102aded
Merge pull request #8 from FairwindsOps/master
May 11, 2020
480b0ee
Split templates by kind
Nov 20, 2019
7a1c806
Restructure/Organize pkg/dashboard into sub-packages
Nov 21, 2019
753e725
Add options pattern to pkg/dashboard
Nov 21, 2019
27963da
Move handlers pkg back to dashboard ; Move dashboard.go -> router.go
Nov 22, 2019
3ae5f66
Change Summary format to better represent its logical data structure
Nov 22, 2019
913d1c2
Change templates to work with new Summary structure
Nov 22, 2019
c01df28
Fix container resource recommendation formatting
Nov 22, 2019
e07be78
Move pkg/dashboard -> pkg/web
Nov 26, 2019
2ed086b
Split templates and dashboard into files ; Make template util functio…
Nov 27, 2019
6f78ba4
Add /namespaces ; Make /namespaces the default route ; Style /namespa…
Nov 27, 2019
f774289
Limit namespace-list.go data passed to template
Nov 27, 2019
dc52aa5
Fix dashboard link
Nov 27, 2019
0cc37e9
Add length to All Namespaces
Nov 27, 2019
fe04ae1
Fix summary_test.go
Nov 27, 2019
51d9d15
Fix typo 'reccomendations' -> 'recommendations'
Nov 27, 2019
ce751e0
Fix typo 'NamespceList' -> 'NamespaceList'
Nov 27, 2019
668e492
Add clarifying comment about template data limiting
Nov 27, 2019
004ed52
Add outer structure to Summary
Nov 27, 2019
01d2283
Rename dashboard -> web
Jun 28, 2020
1627acc
Allow goldilocks' web RBAC to list/get namespaces
Jun 28, 2020
9cddf35
Move common labels to utils.go
Jun 28, 2020
855681c
Use correct labels location
Jun 28, 2020
216806a
Attempt to fix e2e
Jun 28, 2020
b885817
Fix incorrect args usage for namespace
Jun 28, 2020
33297c9
Add oidc auth support
Jun 28, 2020
5db0769
Remove redundant return statement from health.go
Jun 28, 2020
7b2f783
Merge branch 'master' into hkatz/dashboard-and-summary-overhaul
Jul 20, 2020
cb0bb73
Update cmd/web.go
Jul 21, 2020
eae73cd
Rename web back to -> dashboard
Jul 22, 2020
5fa8536
Merge branch 'master' into hkatz/dashboard-and-summary-overhaul
Aug 5, 2020
4fa7837
Merge pull request #9 from FairwindsOps/master
hjkatz Aug 10, 2020
9bb57aa
Merge pull request #10 from hjkatz/hkatz/dashboard-and-summary-overhaul
hjkatz Aug 10, 2020
ae15913
Batch Summary Deployments list into a single API call ; Improves spee…
Nov 27, 2019
2360971
Remove superfluous Error prefix in error message
hjkatz Aug 10, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 64 additions & 15 deletions pkg/summary/summary.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ package summary
import (
"strings"

appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/labels"
Expand Down Expand Up @@ -64,6 +65,9 @@ type Summarizer struct {

// cached list of vpas
vpas []v1beta2.VerticalPodAutoscaler

// cached map of deploy/vpa name -> deployment
deploymentForVPANamed map[string]*appsv1.Deployment
}

// NewSummarizer returns a Summarizer for all goldilocks managed VPAs in all Namespaces
Expand Down Expand Up @@ -93,14 +97,15 @@ func (s Summarizer) GetSummary() (Summary, error) {
summary := Summary{
Namespaces: map[string]namespaceSummary{},
}
// cached vpas
if s.vpas == nil {
err := s.UpdateVPAs()
// cached vpas and deployments
if s.vpas == nil || s.deploymentForVPANamed == nil {
err := s.Update()
if err != nil {
return summary, err
}
}

// nothing to summarize
if len(s.vpas) <= 0 {
return summary, nil
}
Expand All @@ -127,10 +132,10 @@ func (s Summarizer) GetSummary() (Summary, error) {
Containers: map[string]containerSummary{},
}

// find all deployments in the namespace for this VPA
deployment, err := s.kubeClient.Client.AppsV1().Deployments(namespace).Get(dSummary.DeploymentName, metav1.GetOptions{})
if err != nil {
klog.Errorf("Error retrieving deployment from API: %v", err)
deployment, ok := s.deploymentForVPANamed[vpa.Name]
if !ok {
klog.Errorf("no matching Deployment found for VPA/%s", vpa.Name)
continue
}

if vpa.Status.Recommendation == nil {
Expand Down Expand Up @@ -183,16 +188,31 @@ func (s Summarizer) GetSummary() (Summary, error) {
return summary, nil
}

// UpdateVPAs updates the list of VPAs that the summarizer uses
func (s *Summarizer) UpdateVPAs() error {
// Update the set of VPAs and Deployments that the Summarizer uses for creating a summary
func (s *Summarizer) Update() error {
err := s.updateVPAs()
if err != nil {
klog.Error(err.Error())
return err
}

err = s.updateDeployments()
if err != nil {
klog.Error(err.Error())
return err
}

return nil
}

func (s *Summarizer) updateVPAs() error {
nsLog := s.namespace
if s.namespace == namespaceAllNamespaces {
nsLog = "all namespaces"
}
klog.V(3).Infof("Looking for VPAs in %s with labels: %v", nsLog, s.vpaLabels)
vpas, err := s.listVPAs()
vpas, err := s.listVPAs(getVPAListOptionsForLabels(s.vpaLabels))
if err != nil {
klog.Error(err.Error())
return err
}
klog.V(10).Infof("Found vpas: %v", vpas)
Expand All @@ -201,10 +221,8 @@ func (s *Summarizer) UpdateVPAs() error {
return nil
}

// Run creates a summary of the vpa info for all namespaces.
func (s Summarizer) listVPAs() ([]v1beta2.VerticalPodAutoscaler, error) {
vpaListOptions := getVPAListOptionsForLabels(s.vpaLabels)
vpas, err := s.vpaClient.Client.AutoscalingV1beta2().VerticalPodAutoscalers(s.namespace).List(vpaListOptions)
func (s Summarizer) listVPAs(listOptions metav1.ListOptions) ([]v1beta2.VerticalPodAutoscaler, error) {
vpas, err := s.vpaClient.Client.AutoscalingV1beta2().VerticalPodAutoscalers(s.namespace).List(listOptions)
if err != nil {
return nil, err
}
Expand All @@ -217,3 +235,34 @@ func getVPAListOptionsForLabels(vpaLabels map[string]string) metav1.ListOptions
LabelSelector: labels.Set(vpaLabels).String(),
}
}

func (s *Summarizer) updateDeployments() error {
nsLog := s.namespace
if s.namespace == namespaceAllNamespaces {
nsLog = "all namespaces"
}
klog.V(3).Infof("Looking for Deployments in %s", nsLog)
deployments, err := s.listDeployments(metav1.ListOptions{})
if err != nil {
return err
}
klog.V(10).Infof("Found deployments: %v", deployments)

// map the deployment.name -> &deployment for easy vpa lookup (since vpa.Name == deployment.Name for matching vpas/deployments)
s.deploymentForVPANamed = map[string]*appsv1.Deployment{}
for _, d := range deployments {
d := d
s.deploymentForVPANamed[d.Name] = &d
}

return nil
}

func (s Summarizer) listDeployments(listOptions metav1.ListOptions) ([]appsv1.Deployment, error) {
deployments, err := s.kubeClient.Client.AppsV1().Deployments(s.namespace).List(listOptions)
if err != nil {
return nil, err
}

return deployments.Items, nil
}