cmd/kubernetes-static: fixes and improvements (#148)

* cmd/kubernetes-static/readme.md: remove trailing whitespace

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

* cmd/kubernetes-static/readme.md: fix documentation

Right now the binary actually expect working directory to be root of the
repository.

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

* go.mod: bump Go version to 1.16

To indicate that Go 1.16 should be used for building, so we can safely
use go:embed directive for kubernetes-static.

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

* cmd/kubernetes-static/data: fix location of api-server metrics

Integration expects it at "api-server" while it was at "apiserver".

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

* cmd/kubernetes-static/main.go: print newline at the end of execution

So when running shell do not spawn at the end of the output. That
improves readability and usability, while it shouldn't interrupt any
consumers, as usually trailing whitespace is properly handled.

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

* src/metric: automatically improve formatting

Using 'gci' formatter.

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

* cmd/kubernetes-static: use go:embed for serving static data

This way, binary is independent from host file system and it's only
important during build.

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

* cmd/kubernetes-static/main.go: simplify service list initialization

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

* cmd/kubernetes-static/main.go: simplify mock client initialization

In majority of cases there is no need to use 'new' keyword.

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

* cmd/kubernetes-static/main.go: simplify returned endpoint address

There is no need to use 'localhost' explicitly.

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

* cmd/kubernetes-static/main.go: remove unnecessary Sleep

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

* cmd/kubernetes-static/main.go: improve imports naming

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

* cmd/kubernetes-static: use only single source file

So it's more intuitive to use 'go run'.

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

* cmd/kubernetes-static/main.go: add missing Service objects

Part of #149

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

* cmd/kubernetes-static/main.go: fix collecting API server metrics

Without this patch, following error is printed:

"entity name and type are required when defining one"

Part of #149

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

* cmd/kubernetes-static/main.go: use constants for controlplane components

So they are less likely to get out of sync with other parts of the code.

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

* src/definition: improve variable names a bit

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

* src/metric/definition.go: mark some metrics as optional

Those metric may not always be available due to various reasons, marking
them as optional silents them from reporting as errors.

If I recall correctly:
- createdAt, createdKind, createdBy, deploymentName metrics won't be
  available for e.g. static pods.
- reason, message metrics will only be available for failing pods.
- cpuRequestedCores, cpuLimitCores, memoryRequestedBytes,
  memoryLimitBytes will only be calculated for pods with resource
  requests configured.
- pvc* metrics will only be available for volumes backed by actual PVC,
  not for EmptyDir volume etc.

Refs #149

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

* src/definition/fetch.go: improve error messages a bit

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

* cmd/kubernetes-static/main.go: mock newer Kubernetes version

To at least align with version of latest test data we have.

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

* src/definition/populate.go: improve error message formatting

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

* src/ksm/group.go: improve errors a bit

To use standard formatting and some minimal error annotation to make
error tracing easier while debugging.

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

* src/ksm/group.go: small styling improvements

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

* src/kubelet: small styling improvements

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

* src/kubelet/metric/metric.go: simplify loop logic

Loop labels are not really needed, as they always stop the closest loop
anyway.

Check for nil Containers slice is also not needed, since iterating over
nil slice will result in no iterations anyway.

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

* src/prometheus: small styling improvements

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

* src/prometheus: improve error messages

So it's clear which is label name and metric name.

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>
Co-authored-by: Roberto Santalla <roobre@roobre.es>

* src/metric/definition.go: improve error messages in toUtilization()

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

* src/definition: small styling and formatting improvements

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

* src/kubelet/metric: improvements to GetMetricsData method

It will now return a pointer to a summary, which should save some memory
copying and is more standard approach for functions, which may return
error in Go.

Additionally, to make use of using structure pointer,
GroupStatsSummary() method is also adopted to take a pointer of the
summary.

There are also improved error messages in GetMetricsData(), which should
be more helpful while debugging.

Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>

Co-authored-by: Roberto Santalla <roobre@roobre.es>

cmd/kubernetes-static/main.go

@@ -1,8 +1,9 @@
 package main
 
 import (
-	"errors"
+	"embed"
 	"fmt"
+	"io/fs"
 	"net"
 	"net/http"
 	"os"
@@ -14,14 +15,15 @@ import (
 	"github.com/newrelic/infra-integrations-sdk/sdk"
 	"github.com/sirupsen/logrus"
 	v1 "k8s.io/api/core/v1"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/apimachinery/pkg/version"
 
 	"github.com/newrelic/nri-kubernetes/v2/src/apiserver"
 	"github.com/newrelic/nri-kubernetes/v2/src/client"
 	"github.com/newrelic/nri-kubernetes/v2/src/controlplane"
 	"github.com/newrelic/nri-kubernetes/v2/src/ksm"
 	"github.com/newrelic/nri-kubernetes/v2/src/kubelet"
-	metric2 "github.com/newrelic/nri-kubernetes/v2/src/kubelet/metric"
+	kubeletmetric "github.com/newrelic/nri-kubernetes/v2/src/kubelet/metric"
 	"github.com/newrelic/nri-kubernetes/v2/src/metric"
 	"github.com/newrelic/nri-kubernetes/v2/src/scrape"
 )
@@ -37,18 +39,19 @@ type argumentList struct {
 
 var args argumentList
 
-func main() {
+// Embed static metrics into binary.
+//go:embed data
+var content embed.FS
 
+func main() {
 	// Determines which subdirectory of cmd/kubernetes-static/ to use
 	// for serving the static metrics
 	k8sMetricsVersion := os.Getenv("K8S_METRICS_VERSION")
 	if k8sMetricsVersion == "" {
 		k8sMetricsVersion = "1_18"
 	}
-	endpoint := startStaticMetricsServer(k8sMetricsVersion)
 
-	// let the http server start...
-	time.Sleep(time.Millisecond * 100)
+	endpoint := startStaticMetricsServer(content, k8sMetricsVersion)
 
 	integration, err := sdk.NewIntegrationProtocol2(integrationName, integrationVersion, &args)
 	if err != nil {
@@ -70,20 +73,85 @@ func main() {
 
 	// Kubelet
 	kubeletClient := newBasicHTTPClient(endpoint + "/kubelet")
-	podsFetcher := metric2.NewPodsFetcher(logger, kubeletClient, true)
+	podsFetcher := kubeletmetric.NewPodsFetcher(logger, kubeletClient, true)
 	kubeletGrouper := kubelet.NewGrouper(
 		kubeletClient,
 		logger,
 		apiServerClient,
 		"ens5",
 		podsFetcher.FetchFuncWithCache(),
-		metric2.CadvisorFetchFunc(kubeletClient, metric.CadvisorQueries))
+		kubeletmetric.CadvisorFetchFunc(kubeletClient, metric.CadvisorQueries))
 	// KSM
 	ksmClient := newBasicHTTPClient(endpoint + "/ksm")
-	k8sClient := new(client.MockedKubernetes)
+	k8sClient := &client.MockedKubernetes{}
+
 	serviceList := &v1.ServiceList{
 		Items: []v1.Service{
 			{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "kube-state-metrics",
+					Namespace: "kube-system",
+				},
+				Spec: v1.ServiceSpec{
+					Selector: map[string]string{
+						"l1": "v1",
+						"l2": "v2",
+					},
+				},
+			},
+			{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "cockroachdb",
+					Namespace: "default",
+				},
+				Spec: v1.ServiceSpec{
+					Selector: map[string]string{
+						"l1": "v1",
+						"l2": "v2",
+					},
+				},
+			},
+			{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "metrics-server",
+					Namespace: "kube-system",
+				},
+				Spec: v1.ServiceSpec{
+					Selector: map[string]string{
+						"l1": "v1",
+						"l2": "v2",
+					},
+				},
+			},
+			{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "kubernetes",
+					Namespace: "default",
+				},
+				Spec: v1.ServiceSpec{
+					Selector: map[string]string{
+						"l1": "v1",
+						"l2": "v2",
+					},
+				},
+			},
+			{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "kube-dns",
+					Namespace: "kube-system",
+				},
+				Spec: v1.ServiceSpec{
+					Selector: map[string]string{
+						"l1": "v1",
+						"l2": "v2",
+					},
+				},
+			},
+			{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "cockroachdb-public",
+					Namespace: "default",
+				},
 				Spec: v1.ServiceSpec{
 					Selector: map[string]string{
 						"l1": "v1",
@@ -93,8 +161,7 @@ func main() {
 			},
 		},
 	}
-	serviceList.Items[0].Namespace = "kube-system"
-	serviceList.Items[0].Name = "kube-state-metrics"
+
 	k8sClient.On("ListServices").Return(serviceList, nil)
 	ksmGrouper := ksm.NewGrouper(ksmClient, metric.KSMQueries, logger, k8sClient)
 
@@ -106,10 +173,10 @@ func main() {
 	// controlPlaneComponentPods maps component.Name to the pod name
 	// found in the file `cmd/kubernetes-static/data/kubelet/pods`
 	controlPlaneComponentPods := map[controlplane.ComponentName]string{
-		"scheduler":          "kube-scheduler-minikube",
-		"etcd":               "etcd-minikube",
-		"controller-manager": "kube-controller-manager-minikube",
-		"apiserver":          "kube-apiserver-minikube",
+		controlplane.Scheduler:         "kube-scheduler-minikube",
+		controlplane.Etcd:              "etcd-minikube",
+		controlplane.ControllerManager: "kube-controller-manager-minikube",
+		controlplane.APIServer:         "kube-apiserver-minikube",
 	}
 
 	for _, component := range controlplane.BuildComponentList() {
@@ -125,7 +192,7 @@ func main() {
 		)
 	}
 
-	k8sVersion := &version.Info{GitVersion: "v1.15.42"}
+	k8sVersion := &version.Info{GitVersion: "v1.18.19"}
 
 	for _, job := range jobs {
 
@@ -145,31 +212,58 @@ func main() {
 	if err := integration.Publish(); err != nil {
 		logrus.Fatalf("Error while publishing: %v", err)
 	}
+
+	fmt.Println()
 }
 
-func startStaticMetricsServer(k8sMetricsVersion string) string {
+func startStaticMetricsServer(content embed.FS, k8sMetricsVersion string) string {
+	listenAddress := "127.0.0.1:0"
 	// This will allocate a random port
-	listener, err := net.Listen("tcp", "127.0.0.1:0")
+	listener, err := net.Listen("tcp", listenAddress)
 	if err != nil {
-		panic(err)
+		logrus.Fatalf("Error listening on %q: %v", listenAddress, err)
 	}
 
-	endpoint := fmt.Sprintf("http://localhost:%d", listener.Addr().(*net.TCPAddr).Port)
+	endpoint := fmt.Sprintf("http://%s", listener.Addr())
 	fmt.Println("Hosting Mock Metrics data on:", endpoint)
 
 	mux := http.NewServeMux()
 
-	dataDir := fmt.Sprintf("./cmd/kubernetes-static/data/%s", k8sMetricsVersion)
-
-	path, err := filepath.Abs(dataDir)
+	path := filepath.Join("data", k8sMetricsVersion)
+	k8sContent, err := fs.Sub(content, path)
 	if err != nil {
-		log.Fatal(errors.New("cannot start server"))
+		logrus.Fatalf("Error taking a %q subtree of embedded data: %v", path, err)
 	}
 
-	mux.Handle("/", http.FileServer(http.Dir(path)))
+	mux.Handle("/", http.FileServer(http.FS(k8sContent)))
 	go func() {
 		logrus.Fatal(http.Serve(listener, mux))
 	}()
 
 	return endpoint
 }
+
+func newBasicHTTPClient(url string) *basicHTTPClient {
+	return &basicHTTPClient{
+		url: url,
+		httpClient: http.Client{
+			Timeout: time.Minute * 10, // high for debugging purposes
+		},
+	}
+}
+
+type basicHTTPClient struct {
+	url        string
+	httpClient http.Client
+}
+
+func (b basicHTTPClient) Do(method, path string) (*http.Response, error) {
+	endpoint := fmt.Sprintf("%s%s", b.url, path)
+	log.Info("Getting: %s", endpoint)
+
+	return b.httpClient.Get(endpoint)
+}
+
+func (b basicHTTPClient) NodeIP() string {
+	return "localhost"
+}

cmd/kubernetes-static/readme.md

@@ -4,23 +4,14 @@ Kubernetes Static is a project used to run the Kubernetes Integration locally on
 
 ## How it works
 
-The files in the `./data` folder are saved outputs from KSM and various kubelet endpoints, all required to run the full integration. 
-The program wil start a temporary HTTP server and serve these files. The groupers are configured to use these endpoints instead of discovering them.
+The files in the `./data` folder are saved outputs from KSM and various kubelet endpoints, which will be embedded at build-time and then served from a temporary HTTP server.
+The groupers are configured to use these endpoints instead of discovering them.
 
 ## Running kubernetes-static
 
-From within this directory, run the following command in your terminal
+From within root of this repository, run the following command in your terminal
 ```shell script
-go run main.go basic_http_client.go 
+go run cmd/kubernetes-static/main.go
 ```
 
-This is not sending any data to an agent, but outputs the JSON to stdout. 
-
-## Configuring your IDE
-
-It's import that the working directory is set to `cmd/kubernetes-static`, because it expects that the `data` folder is in the process's working directory. 
-Some IDE's/editor use a temporary folder as the working directory. 
-
-Example configuration for GoLand:
-![Goland configuration example](./config_example.png)
-
+This is not sending any data to an agent, but outputs the JSON to stdout.

go.mod

@@ -1,6 +1,6 @@
 module github.com/newrelic/nri-kubernetes/v2
 
-go 1.14
+go 1.16
 
 require (
 	github.com/golang/protobuf v1.5.2

src/definition/fetch.go

@@ -1,7 +1,7 @@
 package definition
 
 import (
-	"errors"
+	"fmt"
 )
 
 // RawValue is just any value from a raw metric.
@@ -32,19 +32,19 @@ type TransformFunc func(FetchedValue) (FetchedValue, error)
 // FromRaw fetches metrics from raw metrics. Is the most simple use case.
 func FromRaw(metricKey string) FetchFunc {
 	return func(groupLabel, entityID string, groups RawGroups) (FetchedValue, error) {
-		g, ok := groups[groupLabel]
+		group, ok := groups[groupLabel]
 		if !ok {
-			return nil, errors.New("group not found")
+			return nil, fmt.Errorf("group %q not found", groupLabel)
 		}
 
-		e, ok := g[entityID]
+		entity, ok := group[entityID]
 		if !ok {
-			return nil, errors.New("entity not found")
+			return nil, fmt.Errorf("entity %q not found", entityID)
 		}
 
-		value, ok := e[metricKey]
+		value, ok := entity[metricKey]
 		if !ok {
-			return nil, errors.New("metric not found")
+			return nil, fmt.Errorf("metric %q not found", metricKey)
 		}
 
 		return value, nil

src/definition/fetch_test.go

@@ -1,9 +1,8 @@
 package definition
 
 import (
-	"testing"
-
 	"strings"
+	"testing"
 
 	"github.com/stretchr/testify/assert"
 )
@@ -44,15 +43,15 @@ func TestFromRawErrorsOnNotFound(t *testing.T) {
 	}
 
 	v, err := FromRaw("metric_name_3")("nonExistingGroup", "entity2", raw)
-	assert.EqualError(t, err, "group not found")
+	assert.EqualError(t, err, "group \"nonExistingGroup\" not found")
 	assert.Nil(t, v)
 
 	v, err = FromRaw("metric_name_3")("group1", "nonExistingEntity", raw)
-	assert.EqualError(t, err, "entity not found")
+	assert.EqualError(t, err, "entity \"nonExistingEntity\" not found")
 	assert.Nil(t, v)
 
 	v, err = FromRaw("non_existing_metric")("group1", "entity2", raw)
-	assert.EqualError(t, err, "metric not found")
+	assert.EqualError(t, err, "metric \"non_existing_metric\" not found")
 	assert.Nil(t, v)
 }
 
@@ -99,14 +98,14 @@ func TestTransformBypassesError(t *testing.T) {
 	}
 
 	v, err := Transform(FromRaw("metric_name_3"), transformFunc)("nonExistingGroup", "entity2", raw)
-	assert.EqualError(t, err, "group not found")
+	assert.EqualError(t, err, "group \"nonExistingGroup\" not found")
 	assert.Nil(t, v)
 
 	v, err = Transform(FromRaw("metric_name_3"), transformFunc)("group1", "nonExistingEntity", raw)
-	assert.EqualError(t, err, "entity not found")
+	assert.EqualError(t, err, "entity \"nonExistingEntity\" not found")
 	assert.Nil(t, v)
 
 	v, err = Transform(FromRaw("non_existing_metric"), transformFunc)("group1", "entity2", raw)
-	assert.EqualError(t, err, "metric not found")
+	assert.EqualError(t, err, "metric \"non_existing_metric\" not found")
 	assert.Nil(t, v)
 }