Updating k8s pod and container templates with no host ips and realistic hostnames #111

gizas · 2023-10-06T10:43:39Z

TSDB nightly tests demand a more realistic set of data generated. As part of this enhancement we update kubernetes pod and container templates with following:

Removing ips and mac addresses to simulate the fix we introduced in Consider not adding host.ip metadata to k8s container metrics by default integrations#6674
Adding realistic host and node names. Our template need a set of 90 nodes where the agents will be located. We correlate the node names with host names
Setting same pod.name and pod.uids values

How to test:

Checkout this PR
Produce pod and container samples:

./elastic-integration-corpus-generator-tool generate-with-template ./assets/templates/kubernetes.pod/gotext.tpl ./assets/templates/kubernetes.pod/fields.yml -c ./assets/templates/kubernetes.pod/configs.yml -y gotext -t 2

./assets/templates/kubernetes.container/gotext.tpl ./assets/templates/kubernetes.container/fields.yml -c ./assets/templates/kubernetes.container/configs.yml -y gotext -t 2

Check that host.ips and host.mac fields are removed
Check the node names created and have the same value as hostnames.

Screenshot

aspacca · 2023-10-06T12:57:26Z

assets/templates/kubernetes.pod/gotext_multiline.tpl

-         "fe80::5d:14ff:fec1:2d3a"
-      ],
-      "name": "{{ $agentName }}",
+      "name": "{{- $nodeid := generate "nodeid" -}}",


beware, if you invoke generate "nodeid" two times in the same template you should end up with two different values

you should: because looking at the screenshot you posted it does not seems so, but this is something to investigate, since it seems to be bug :)

ok once per file. Then I refer to nodeid variable

martijnvg

I think the host name doesn't need to be to same as node name? As long as each node name uses the same host name?

martijnvg · 2023-10-06T14:12:08Z

assets/templates/kubernetes.container/gotext_multiline.tpl

@@ -103,12 +104,12 @@
      },
      "node":{
         "uid": "{{ $uId }}" ,
-         "hostname":"{{ $agentName }}.c.elastic-obs-integrations-dev.internal",
-         "name":"{{ $agentName }}-{{ $rangeofid }}",
+         "hostname":"node-{{ $nodeid }}",


Maybe host-{{ $nodeid }}?

martijnvg · 2023-10-06T14:13:39Z

assets/templates/kubernetes.container/gotext_multiline.tpl

@@ -214,7 +215,7 @@
      "dataset":"kubernetes.container"
   },
   "host":{
-      "hostname":"{{ $agentName }}",
+      "hostname":"node-{{ $nodeid }}",


Maybe host-?

martijnvg · 2023-10-06T14:14:53Z

assets/templates/kubernetes.pod/gotext_multiline.tpl

@@ -28,12 +29,12 @@
   "kubernetes": {
      "node":{
         "uid": "{{ $uId }}" ,
-         "hostname":"{{ $agentName }}.c.elastic-obs-integrations-dev.internal",
-         "name":"{{ $agentName }}-{{ $rangeofid }}",
+         "hostname":"node-{{ $nodeid }}",


Maybe host-?

aspacca · 2023-10-10T07:35:40Z

@gizas could you please move the files of each folder inside the schema-b subfolder? thanks :)

gizas · 2023-10-10T09:01:38Z

@gizas could you please move the files of each folder inside the schema-b subfolder? thanks :)

Done @aspacca

…generator-tool into update_k8s_pod_container_templatesforTSDB

aspacca · 2023-11-14T04:48:01Z

assets/templates/kubernetes.container/schema-b/configs.yml

@@ -45,7 +45,7 @@ fields:
  - name: container.name
    enum: ["web", "default-http-backend", "dnsmasq", "csi-driver", "web", "web", "web", "prometheus", "konnectivity-agent", "sidecar", "kubedns", "metrics-server-nanny", "web", "web", "fluentbit", "autoscaler", "gke-metrics-agent", "elastic-agent", "web", "kube-state-metrics", "metrics-server", "fluentbit", "elastic-agent", "web", "prometheus-to-sd-exporter"]
  - name: timedate
-    # Add more dates below in case you want your data to be spread
+    # Add more dates below in case you want your data to be spread.Those are fixed because this is required in the nightly tests of Rally


can you explain about this?
will it be fixed "forever", or only for the current nightly tests of rally.
what happens if we use timestamp with period: 2880m (two days) like I did here

This one is fixed for the current nightly tests of rally. We have specific visualisations and we want our data to be in specific range in order our queries to be be always specific

would it be possible to change the visualisation range to be -2d/now?

@aspacca
I have made some tests today and from generator tool side we can change the timedate to date type with [period: 48h] and all seem ok. So I dont see an issue with this now

Moreover on the rally side we have hardcoded the values of visualisations here

So we need to find a way to import to calculate the new itervals and end_time based on the track provided.
cc @martijnvg for visibility

…s based on pod rangeofid div 110

ruflin · 2023-11-20T07:42:05Z

I assume eventually we will move these files over to the k8s package?

gizas · 2023-11-20T08:39:10Z

I assume eventually we will move these files over to the k8s package?

This is the purpose of 8443 and 8444

gizas · 2023-12-04T12:21:34Z

@aspacca and @martijnvg I think this can also be merged now as it has been tested in various places and different PRs.
Please any approvals to merge?

martijnvg

👍

…generator-tool into update_k8s_pod_container_templatesforTSDB

Removing ips and mac and adding correct host and node names

9fc1ee6

gizas added the Team:Cloudnative-Monitoring Label for the Cloud Native Monitoring team label Oct 6, 2023

gizas self-assigned this Oct 6, 2023

gizas requested review from aspacca, martijnvg and MichaelKatsoulis October 6, 2023 10:46

aspacca reviewed Oct 6, 2023

View reviewed changes

martijnvg reviewed Oct 6, 2023

View reviewed changes

gizas added 3 commits October 6, 2023 18:03

Adding host names and only one reference for generate 'nodeid'

1d37107

Adding host names and only one reference for generate 'nodeid'

58f6a1a

Removing extra lines

54babf0

New folder schema-b for templates

e7bbb0d

Merge branch 'main' of github.com:elastic/elastic-integration-corpus-…

ce0428c

…generator-tool into update_k8s_pod_container_templatesforTSDB

gizas mentioned this pull request Oct 10, 2023

Updating K8s Rally track with new elastic/rally-tracks#467

Merged

aspacca mentioned this pull request Nov 10, 2023

Rally benchmark kubernetes.pod elastic/integrations#8443

Merged

2 tasks

Setting same pod.name and pod.uids values

46f451f

aspacca reviewed Nov 14, 2023

View reviewed changes

gizas added 2 commits November 14, 2023 11:28

Setting node.uid values same with node.name and also calcualting node…

fe73f69

…s based on pod rangeofid div 110

Setting container id as name-rangeofid

9ef0f5c

gizas mentioned this pull request Nov 14, 2023

Updating k8s track with 8.12 templates and new UIds elastic/rally-tracks#513

Merged

martijnvg approved these changes Dec 4, 2023

View reviewed changes

Merge branch 'main' of github.com:elastic/elastic-integration-corpus-…

d36249f

…generator-tool into update_k8s_pod_container_templatesforTSDB

gizas merged commit 605bde0 into main Dec 4, 2023
2 checks passed

gizas deleted the update_k8s_pod_container_templatesforTSDB branch December 4, 2023 13:27

gizas mentioned this pull request Feb 14, 2024

[O11y][Kubernetes] Rally benchmark kubernetes.state_pod elastic/integrations#9136

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updating k8s pod and container templates with no host ips and realistic hostnames #111

Updating k8s pod and container templates with no host ips and realistic hostnames #111

gizas commented Oct 6, 2023 •

edited

Loading

aspacca Oct 6, 2023

gizas Oct 6, 2023

martijnvg left a comment

martijnvg Oct 6, 2023

gizas Oct 6, 2023

martijnvg Oct 6, 2023

martijnvg Oct 6, 2023

aspacca commented Oct 10, 2023

gizas commented Oct 10, 2023

aspacca Nov 14, 2023

gizas Nov 14, 2023

aspacca Nov 14, 2023

gizas Nov 14, 2023

ruflin commented Nov 20, 2023

gizas commented Nov 20, 2023

gizas commented Dec 4, 2023

martijnvg left a comment

Updating k8s pod and container templates with no host ips and realistic hostnames #111

Updating k8s pod and container templates with no host ips and realistic hostnames #111

Conversation

gizas commented Oct 6, 2023 • edited Loading

How to test:

Screenshot

Choose a reason for hiding this comment

Choose a reason for hiding this comment

martijnvg left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aspacca commented Oct 10, 2023

gizas commented Oct 10, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ruflin commented Nov 20, 2023

gizas commented Nov 20, 2023

gizas commented Dec 4, 2023

martijnvg left a comment

Choose a reason for hiding this comment

gizas commented Oct 6, 2023 •

edited

Loading