Add node status conditions metrics #155
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
invidian
force-pushed
the
invidian/add-node-status-metrics
branch
from
3101680
to
f9db301
Compare
roobre
reviewed
Jul 14, 2021
|
I would also update the JsonSchema files for e2e test to capture this, with the basic kubelet conditions. |
invidian
force-pushed
the
invidian/improvements
branch
2 times, most recently
from
95269ca
to
21d1242
Compare
invidian
force-pushed
the
invidian/add-node-status-metrics
branch
3 times, most recently
from
48f0d5b
to
0d6b3d6
Compare
Done, I think 😄 |
invidian
force-pushed
the
invidian/add-node-status-metrics
branch
2 times, most recently
from
f8c1a3a
to
b73df28
Compare
invidian
force-pushed
the
invidian/improvements
branch
from
21d1242
to
03c208f
Compare
invidian
force-pushed
the
invidian/add-node-status-metrics
branch
from
b73df28
to
f5a5d0a
Compare
invidian
force-pushed
the
invidian/improvements
branch
2 times, most recently
from
124dabe
to
6885e7a
Compare
invidian
force-pushed
the
invidian/add-node-status-metrics
branch
from
344de7d
to
6454a43
Compare
roobre
reviewed
Jul 19, 2021
|
E2E tests seem great to me, thanks! |
Right now, if namespace metrics are processed, we still iterate over pod-specific metrics, which will have "namespace" label. Those metrics will be added to the namespace entity collection and will be overwritten by next incoming metric of the same name. To avoid putting unrelated data and overwritting it, this commit adds conditions, which only puts relevant metrics for each raw group. This means, "node" group won't have Pod-specific metrics and "namespace" group won't have "pod", "daemonset" and others "more-specific" metrics. This saves some computation time and makes it easier to debug if one dumps entire raw groups struct. Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>
invidian
force-pushed
the
invidian/add-node-status-metrics
branch
2 times, most recently
from
ec17e3d
to
f49fbf1
Compare
roobre
approved these changes
Jul 20, 2021
|
Looking great, thanks @invidian! |
invidian
force-pushed
the
invidian/add-node-status-metrics
branch
from
f49fbf1
to
fddced0
Compare
Right now, when processed metric family has more than one metric, it will overwrite the content of the previous metric. This commit changes this behavior and now if there is more then one metric in a single metric family, value of the metric will be converted to collection and appended, making all the metrics available for further processing. Right now, only user of this code is KSM metrics and no metrics are duplicated with recently introduced filtering, however incoming node state conditions metrics (#138) will require multiple metrics coming from GroupMetricsBySpec for processing. Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>
This commit adds a WithLabelValueInMetricNameAndWithLabelValueMappedToMetricValue function, which allows to generate a set of metrics with label value in the metric name. This will be used to generate node status conditions metrics (#138), but may potentially be used also to generate HPA condition metrics. Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>
This will be required to test node status condition metrics. Refs #138 Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>
Right now integration creates multiple MetricSets from each job, which makes it harder in e2e tests to validate those metrics against the schema. This commit improves that by merging multiple MetricSets into one before validating against JSON schema. In SDK version 3+ this will no longer be needed. Refs #141 Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>
For node condition metric, which may have "unknown" value, we agreed to send -1 to the backend, so we do not lose this information. Refs #138 Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>
Closes #138 Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>
Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>
invidian
force-pushed
the
invidian/add-node-status-metrics
branch
from
fddced0
to
24b71d0
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Includes commits from #148. Proper diff to look at is here: invidian/improvements...invidian/add-node-status-metricsDraft as the following things are missing: