Add support for OVS flow operations metrics on node #866

yktsubo · 2020-06-24T15:37:30Z

Add support for OVS flow operations metrics on node

Number of OVS flow operations, partitioned by operations(add, modify and delete)
Number of OVS flow operation errors, partitioned by operations(add, modify and delete)
The latency of OVS flow operations, partitioned by operations(add, modify and delete)

This PR is a part of #713 feature request

Signed-off-by: Yuki Tsuboi ytsuboi@vmware.com

antrea-bot · 2020-06-24T15:37:46Z

Thanks for your PR.
Unit tests and code linters are run automatically every time the PR is updated.
E2e, conformance and network policy tests can only be triggered by a member of the vmware-tanzu organization. Regular contributors to the project should join the org.

The following commands are available:

/test-e2e: to trigger e2e tests.
/skip-e2e: to skip e2e tests.
/test-conformance: to trigger conformance tests.
/skip-conformance: to skip conformance tests.
/test-whole-conformance: to trigger all conformance tests on linux.
/skip-whole-conformance: to skip all conformance tests on linux.
/test-networkpolicy: to trigger networkpolicy tests.
/skip-networkpolicy: to skip networkpolicy tests.
/test-windows-conformance: to trigger windows conformance tests.
/skip-windows-conformance: to skip windows conformance tests.
/test-all: to trigger all tests (except whole conformance).
/skip-all: to skip all tests (except whole conformance).

These commands can only be run by members of the vmware-tanzu organization.

srikartati

Thanks for the change, Yuki.

srikartati · 2020-06-24T17:57:38Z

pkg/agent/metrics/prometheus.go

@@ -67,6 +67,54 @@ var (
 		Help:           "Flow count for each OVS flow table. The TableID is used as a label.",
 		StabilityLevel: metrics.STABLE,
 	}, []string{"table_id"})
+
+	OVSFlowAddErrorCount = metrics.NewCounter(


Do you think we can have one metric OVSFlowOpsErrorCount and have add, modify and delete as labels?
Updating correct ref for labels in summary (golang): https://github.com/kubernetes/component-base/blob/release-1.18/metrics/summary.go#L30 SummaryOpts have constLabels to consume.

Sure, I'll take a look at it.

I did some research and think that it's good to use NewConterVec for ErrorCount and NewSummaryVec for Duration.
But please kindly let me know if SummaryOpts is better than them.
I think it's good because constLabels are static and put to all measured metrics.

Hence I try to define the metrics using the below label.

antrea_agent_ovs_flow_ops_error_count - differentiate operations: operation="add|modify|delete"

antrea_agent_ovs_flow_ops_duration_milliseconds - differentiate operations: operation="add|modify|delete"

antoninbas

Choice of metrics lgtm

antoninbas · 2020-06-29T19:45:48Z

pkg/agent/metrics/prometheus.go

+	if err := legacyregistry.Register(OVSFlowOpsErrorCount); err != nil {
+		klog.Error("Failed to register antrea_agent_ovs_flow_ops_error_count with Prometheus")
+	}
+	OVSFlowOpsErrorCount.WithLabelValues("add")


is this "initialization" needed?

From a program perspective, initialization is not required.
But I thought it's good to initialize them so that prometheus can know which metrics are there and there are no errors until now.
Without initialization, antrea_agent_ovs_flow_ops_error_count won't come out until you hit errors.
Please let me know your thoughts on this.

Sounds good to me, thanks for the explanation. I recommend adding a comment in the code with this explanation to avoid confusion in the future.

Sure, thank you for your suggestion.

antoninbas · 2020-06-29T19:45:56Z

pkg/agent/metrics/prometheus.go

+	if err := legacyregistry.Register(OVSFlowOpsDuration); err != nil {
+		klog.Error("Failed to register antrea_agent_ovs_flow_ops_duration_milliseconds with Prometheus")
+	}
+	OVSFlowOpsDuration.WithLabelValues("add")


same question as above?

Commented on above. I'll change this in the same way.

pkg/agent/metrics/prometheus.go

antoninbas · 2020-06-29T19:50:49Z

pkg/agent/metrics/prometheus.go

+	OVSFlowOpsDuration = metrics.NewSummaryVec(
+		&metrics.SummaryOpts{
+			Name:           "antrea_agent_ovs_flow_ops_duration_milliseconds",
+			Help:           "The duration of OVS flow operation",


Suggested change

Help: "The duration of OVS flow operation",

Help: "The latency of OVS flow operations.",

Thank you for your comment, I'll fix it.

srikartati

Vector metrics sound good.

srikartati · 2020-06-29T21:10:35Z

pkg/agent/metrics/prometheus.go

+		[]string{"operation"},
+	)
+
+	OVSFlowOpsDuration = metrics.NewSummaryVec(


Not your change. There are other summary/summary vector metrics in Antrea. As per Kubernetes metric overhaul, it is recommended to use histograms instead of summaries. And the summary metrics are tagged for deprecation.
Main advantages with histogram are aggregation and inexpensive. Any comments @ksamoray ?

@antoninbas @tnqn any thoughts on above?

I am not an expert, but we have been using the STABLE stability level for all these metrics. According to that contract, the type of the metric will not be modified. So while I am fine with using histograms instead of summaries for new metrics, do we actually want to update existing metrics to use histograms? Or should we follow the guidelines and deprecate the old metrics while introducing new ones with the histograms type?

As for new metrics in this PR, I'll follow the recommendation to use histograms.

So while I am fine with using histograms instead of summaries for new metrics, do we actually want to update existing metrics to use histograms? Or should we follow the guidelines and deprecate the old metrics while introducing new ones with the histograms type?

Thanks for the response. Yes, making the current metric deprecated and add the new metrics with histogram type is suggested. After a couple of releases, removing deprecated metrics is suggested as a guideline. I am wondering this makes sense if there are consumers (w/dashboards) or third party software dependent on these metrics. If not, could we just update them with histogram type?

Given that we probably have few users (or none) relying on these metrics. I don't think I would be opposed to just switching the metric type, providing we sent the appropriate notices on Slack and the mailing list and waited a couple of days for feedback. In the future, it may be a good idea to not tag these metrics as STABLE right away...

Not defining them as STABLE sounds good. We should define new metrics as ALPHA and move them to STABLE after a release or two; or when confident that these were being used effectively.
Let me open an issue and post this in slack and mailing list.

Agreed to go with Alpha in a few first releases.
I'll make a change to this PR.
@srikartati @antoninbas But do you think we also need to make existing metrics Alpha?

I think it is better to make them alpha and turn them into STABLE after the next release.

Thank you for your comments.

srikartati · 2020-06-29T21:12:24Z

pkg/agent/metrics/prometheus.go

+		klog.Error("Failed to register antrea_agent_ovs_flow_ops_error_count with Prometheus")
+	}
+	OVSFlowOpsErrorCount.WithLabelValues("add")
+	OVSFlowOpsErrorCount.WithLabelValues("modify")


Same question as Antonin. Are these needed?

Please have a look at the above comment. I'd like to hear your opinion.

Hi Yuki, Will more descriptive 'help' sufficient mentioning label values add, delete and modify?

Thank you for your comment. I'll make help more descriptive and also add comments about why we have initialization for add, delete and modify.

- Number of OVS flow operations, partitioned by operations(add, modify and delete) - Number of OVS flow operation errors, partitioned by operations(add, modify and delete) - Latency of OVS flow operations, partitioned by operations(add, modify and delete) Signed-off-by: Yuki Tsuboi <ytsuboi@vmware.com>

- Number of OVS flow operations, partitioned by operations(add, modify and delete) - Number of OVS flow operation errors, partitioned by operations(add, modify and delete) - The latency of OVS flow operations, partitioned by operations(add, modify and delete) Signed-off-by: Yuki Tsuboi <ytsuboi@vmware.com>

srikartati

LGTM.
I took a crack at adding tests for some Prometheus metrics using existing integration tests.
#916
Do you think these metrics can be tested in similar fashion?

yktsubo · 2020-07-07T14:45:22Z

Hi @srikartati
Thank you for your comment. I think we can test these metrics, as well.
About latency, we cannot expect exact value but we can check it's not zero.
Depending on your PR merge, I can add my test on top of it.

srikartati · 2020-07-07T17:39:13Z

Hi @srikartati
Thank you for your comment. I think we can test these metrics, as well.
About latency, we cannot expect exact value but we can check it's not zero.
Depending on your PR merge, I can add my test on top of it.

Sure, different PR for testing sounds good.

srikartati · 2020-07-07T17:39:26Z

/test-all

yktsubo · 2020-07-09T14:00:23Z

Sure, @srikartati I'll work on a test case in a different PR.

srikartati · 2020-07-13T21:19:30Z

/test-networkpolicy

srikartati · 2020-07-13T21:24:35Z

/test-conformance

srikartati · 2020-07-14T15:56:12Z

/test-all

srikartati

LGTM

srikartati · 2020-07-14T18:17:38Z

/test-e2e

antoninbas

one nit, otherwise LGTM

antoninbas · 2020-07-14T21:08:21Z

pkg/agent/metrics/prometheus.go

+	OVSFlowOpsCount = metrics.NewCounterVec(
+		&metrics.CounterOpts{
+			Name:           "antrea_agent_ovs_flow_ops_count",
+			Help:           "Number of OVS flow operations, partitioned by operations(add, modify and delete).",


s/partitioned by operations(add, modify and delete)/partitioned by operation type (add, modify and delete)

same for the other places below

Thank you for your comment. Fixed.

- Number of OVS flow operations, partitioned by operations(add, modify and delete) - Number of OVS flow operation errors, partitioned by operations(add, modify and delete) - The latency of OVS flow operations, partitioned by operations(add, modify and delete) Signed-off-by: Yuki Tsuboi <ytsuboi@vmware.com>

yktsubo · 2020-07-18T00:40:45Z

The metrics name are different depending on source.

From pods,
antrea_agent_ovs_flow_ops_latency_milliseconds

From prometheus server,
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket
antrea_agent_ovs_flow_ops_latency_milliseconds_count
antrea_agent_ovs_flow_ops_latency_milliseconds_sum

yktsubo · 2020-07-18T00:58:40Z

I'm planning to have separate expected metrics for pods and prometheus server.
Please let me know if it doesn't sound good.

srikartati · 2020-07-18T06:32:13Z

The metrics name are different depending on source.

From pods,
antrea_agent_ovs_flow_ops_latency_milliseconds

From prometheus server,
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket
antrea_agent_ovs_flow_ops_latency_milliseconds_count
antrea_agent_ovs_flow_ops_latency_milliseconds_sum

Thanks for root causing it. It is because of the histogram as you mentioned, Can you elaborate a bit more on why this is happening? Just curious about why the Prometheus server cannot treat it as one metric?

srikartati · 2020-07-20T07:06:53Z

The metrics name are different depending on source.
From pods,
antrea_agent_ovs_flow_ops_latency_milliseconds
From prometheus server,
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket
antrea_agent_ovs_flow_ops_latency_milliseconds_count
antrea_agent_ovs_flow_ops_latency_milliseconds_sum

Thanks for root causing it. It is because of the histogram as you mentioned, Can you elaborate a bit more on why this is happening? Just curious about why the Prometheus server cannot treat it as one metric?

A follow up question: Does Antrea agent metrics handler output for histogram metric is in following format? If so, adding bucket, count, sum metrics make sense.
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{le="0.005",} xx
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{le="0.01",} xx
....
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{le="+Inf",} xx
antrea_agent_ovs_flow_ops_latency_milliseconds_count xx
antrea_agent_ovs_flow_ops_latency_milliseconds_sum xxx

yktsubo · 2020-07-20T07:38:11Z

Hi @srikartati

https://prometheus.io/docs/concepts/metric_types/#histogram
As mentioned in the official page, antrea exposes metrics like below.
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{le="0.005",} xx
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{le="0.01",} xx
....
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{le="+Inf",} xx
antrea_agent_ovs_flow_ops_latency_milliseconds_count xx
antrea_agent_ovs_flow_ops_latency_milliseconds_sum xxx

Here is an example from an agent pod

$ curl -k -H "Authorization: Bearer $token" https://10.16.181.21:10350/metrics 2> /dev/null | grep antrea_agent_ovs_flow_ops_latency_milliseconds
# HELP antrea_agent_ovs_flow_ops_latency_milliseconds [ALPHA] The latency of OVS flow operations, partitioned by operation type (add, modify and delete).
# TYPE antrea_agent_ovs_flow_ops_latency_milliseconds histogram
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="add",le="0.005"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="add",le="0.01"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="add",le="0.025"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="add",le="0.05"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="add",le="0.1"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="add",le="0.25"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="add",le="0.5"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="add",le="1"} 4
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="add",le="2.5"} 9
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="add",le="5"} 10
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="add",le="10"} 10
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="add",le="+Inf"} 10
antrea_agent_ovs_flow_ops_latency_milliseconds_sum{operation="add"} 17
antrea_agent_ovs_flow_ops_latency_milliseconds_count{operation="add"} 10
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="delete",le="0.005"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="delete",le="0.01"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="delete",le="0.025"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="delete",le="0.05"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="delete",le="0.1"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="delete",le="0.25"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="delete",le="0.5"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="delete",le="1"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="delete",le="2.5"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="delete",le="5"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="delete",le="10"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="delete",le="+Inf"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_sum{operation="delete"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_count{operation="delete"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="modify",le="0.005"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="modify",le="0.01"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="modify",le="0.025"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="modify",le="0.05"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="modify",le="0.1"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="modify",le="0.25"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="modify",le="0.5"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="modify",le="1"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="modify",le="2.5"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="modify",le="5"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="modify",le="10"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_bucket{operation="modify",le="+Inf"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_sum{operation="modify"} 0
antrea_agent_ovs_flow_ops_latency_milliseconds_count{operation="modify"} 0

So the behavior of exposing metrics is fine.
The issue is in our test code side. (Sorry I should have provided more clear information)

Let me take antrea_agent_ovs_flow_ops_latency_milliseconds as an example.
In testPrometheusMetricsOnPods in e2e, we use expfmt library to parse metrics from pod
https://github.com/vmware-tanzu/antrea/blob/a858c9ad25c215418003e4cb92a68ec62c7fddbf/test/e2e/prometheus_test.go#L162

This TextToMetricFamillies returns the basename of metrics.
Hence, parsed metrics will get antrea_agent_ovs_flow_ops_latency_milliseconds.

In testMetricsFromPrometheusServer in e2e, we get metrics from prometheus and parse it using json library.
Therefore, it will get <basename>_bucket, <basename>_sum and <basename>_count.

That is why either of the tests failed depending on expected metrics.

I think if we can use basename as expected metrics in testMetricsFromPrometheusServer too, it will be better.
I'm looking at documents to get basename from prometheus API. If it's not possible, I'll see if it's ok we can simply cut back _bucket from metrics.

yktsubo · 2020-07-20T14:04:37Z

I think expfmt gets basename by cutting out _bucket

https://github.com/prometheus/common/blob/546f1fd8d7df61d94633b254641f9f8f48248ada/expfmt/text_parse.go#L665

So I think we can cut out _bucket from metrics gotten from prometheus.
@srikartati let me know if it doesn't sound good.

srikartati · 2020-07-20T16:03:22Z

I think expfmt gets basename by cutting out _bucket

https://github.com/prometheus/common/blob/546f1fd8d7df61d94633b254641f9f8f48248ada/expfmt/text_parse.go#L665

So I think we can cut out _bucket from metrics gotten from Prometheus.
@srikartati let me know if it doesn't sound good.

It sounds good. Keeping the behavior consistent for Antrea components and Prometheus server would be good in test code.

- Number of OVS flow operations, partitioned by operations(add, modify and delete) - Number of OVS flow operation errors, partitioned by operations(add, modify and delete) - The latency of OVS flow operations, partitioned by operations(add, modify and delete) - Use prometheus v2.19.2 image to use API of querying target metadata in the e2e test Signed-off-by: Yuki Tsuboi <ytsuboi@vmware.com>

yktsubo · 2020-07-20T16:37:29Z

On second thought, if we can use the prometheus v2.19.2, we can get basename without cutting out the metrics name.
Since some of our metrics are using _count, I think using querying target metadata API to list up scraped metrics.
https://prometheus.io/docs/prometheus/latest/querying/api/#querying-target-metadata

@antoninbas @ksamoray Do we have any specific reason to use prometheus v2.2.1?
If not, I'd like to upgrade it to v2.19.2 to use a new API in e2e test.

srikartati · 2020-07-21T15:51:22Z

On second thought, if we can use the prometheus v2.19.2, we can get basename without cutting out the metrics name.
Since some of our metrics are using _count, I think using querying target metadata API to list up scraped metrics.
https://prometheus.io/docs/prometheus/latest/querying/api/#querying-target-metadata

@antoninbas @ksamoray Do we have any specific reason to use prometheus v2.2.1?
If not, I'd like to upgrade it to v2.19.2 to use a new API in e2e test.

Is this version Prometheus server version or Prometheus library version. Maybe it's better to do the version change in separate PR? Just cut out the metric names for now?

yktsubo · 2020-07-22T00:18:49Z

Thank you for your response.
I'll create a separate PR for Prometheus version update.

- Number of OVS flow operations, partitioned by operations(add, modify and delete) - Number of OVS flow operation errors, partitioned by operations(add, modify and delete) - The latency of OVS flow operations, partitioned by operations(add, modify and delete) Signed-off-by: Yuki Tsuboi <ytsuboi@vmware.com>

srikartati

Just wondering if you run the Prometheus test on vagrant setup using the following commands?

To load Antrea into the cluster with Prometheus enabled, use: ./infra/vagrant/push_antrea.sh --prometheus
To run the Prometheus tests within the e2e suite, use: go test -v github.com/vmware-tanzu/antrea/test/e2e --prometheus

srikartati · 2020-07-22T17:51:18Z

test/e2e/prometheus_test.go

@@ -276,7 +279,12 @@ func testMetricsFromPrometheusServer(t *testing.T, data *TestData, prometheusJob
 	// Create a map of all the metrics which were found on the server
 	testMap := make(map[string]bool)
 	for _, metric := range output.Data {
-		testMap[metric["__name__"]] = true
+		name := metric["__name__"]
+		switch {


Isn't "if" sufficient here?

sorry, yes 'if' is enough. For the first time, I thought we have to consider summary as well. but it's not required. I'll make a change as you suggested.

srikartati · 2020-07-22T17:52:13Z

test/e2e/prometheus_test.go

-		testMap[metric["__name__"]] = true
+		name := metric["__name__"]
+		switch {
+		case isBucket(name):


strings.Contains and strings.TrimSuffix could be used here to make this more readable.

That makes sense. I'll make a change as you suggested.

yktsubo · 2020-07-23T00:08:55Z

Just wondering if you run the Prometheus test on vagrant setup using the following commands?

To load Antrea into the cluster with Prometheus enabled, use: ./infra/vagrant/push_antrea.sh --prometheus
To run the Prometheus tests within the e2e suite, use: go test -v github.com/vmware-tanzu/antrea/test/e2e --prometheus

Sorry, I wasn't aware that we can run e2e test on my testbed.
I'll make sure that e2e is verified before pushing my code.

- Number of OVS flow operations, partitioned by operations(add, modify and delete) - Number of OVS flow operation errors, partitioned by operations(add, modify and delete) - The latency of OVS flow operations, partitioned by operations(add, modify and delete) Signed-off-by: Yuki Tsuboi <ytsuboi@vmware.com>

srikartati

Small nit otherwise LGTM.

test/e2e/prometheus_test.go

- Number of OVS flow operations, partitioned by operations(add, modify and delete) - Number of OVS flow operation errors, partitioned by operations(add, modify and delete) - The latency of OVS flow operations, partitioned by operations(add, modify and delete) Signed-off-by: Yuki Tsuboi <ytsuboi@vmware.com>

srikartati · 2020-07-24T05:49:49Z

/test-all

srikartati · 2020-07-24T17:40:07Z

/test-windows-conformance

srikartati · 2020-07-24T18:43:40Z

/test-windows-conformance

yktsubo · 2020-07-24T23:02:18Z

@srikartati Thank you so much for your help

srikartati · 2020-07-24T23:04:24Z

Thanks for working on the PR. Merging this.

- Number of OVS flow operations, partitioned by operations(add, modify and delete) - Number of OVS flow operation errors, partitioned by operations(add, modify and delete) - The latency of OVS flow operations, partitioned by operations(add, modify and delete) Signed-off-by: Yuki Tsuboi <ytsuboi@vmware.com>

vmwclabot added the cla-not-required label Jun 24, 2020

srikartati reviewed Jun 24, 2020

View reviewed changes

yktsubo force-pushed the add_ovs_flow_operation_metrics_on_agent branch from 5ef4a78 to 14d4802 Compare June 28, 2020 16:32

antoninbas reviewed Jun 29, 2020

View reviewed changes

antoninbas requested a review from srikartati June 29, 2020 19:51

srikartati reviewed Jun 29, 2020

View reviewed changes

yktsubo force-pushed the add_ovs_flow_operation_metrics_on_agent branch from 14d4802 to 14f47f6 Compare July 3, 2020 15:44

yktsubo changed the title ~~Add metrics of error count and duration of OVS flow operation on node~~ Add support for OVS flow operations metrics on node Jul 3, 2020

yktsubo force-pushed the add_ovs_flow_operation_metrics_on_agent branch from 14f47f6 to 63c68f7 Compare July 3, 2020 15:49

yktsubo force-pushed the add_ovs_flow_operation_metrics_on_agent branch from 63c68f7 to df5e6fd Compare July 3, 2020 15:53

yktsubo force-pushed the add_ovs_flow_operation_metrics_on_agent branch from df5e6fd to 0efd77b Compare July 3, 2020 15:55

srikartati reviewed Jul 7, 2020

View reviewed changes

srikartati previously approved these changes Jul 14, 2020

View reviewed changes

srikartati requested a review from antoninbas July 14, 2020 15:57

antoninbas reviewed Jul 14, 2020

View reviewed changes

yktsubo dismissed srikartati’s stale review via 250d29e July 15, 2020 00:06

yktsubo force-pushed the add_ovs_flow_operation_metrics_on_agent branch from 0efd77b to 250d29e Compare July 15, 2020 00:06

yktsubo force-pushed the add_ovs_flow_operation_metrics_on_agent branch from 6903963 to 4fbe993 Compare July 20, 2020 16:34

yktsubo force-pushed the add_ovs_flow_operation_metrics_on_agent branch from 4fbe993 to 4708fed Compare July 22, 2020 00:32

srikartati reviewed Jul 22, 2020

View reviewed changes

yktsubo force-pushed the add_ovs_flow_operation_metrics_on_agent branch from 4708fed to a0d6530 Compare July 23, 2020 05:59

srikartati previously approved these changes Jul 23, 2020

View reviewed changes

test/e2e/prometheus_test.go Outdated Show resolved Hide resolved

yktsubo dismissed srikartati’s stale review via 5dc364d July 23, 2020 22:57

yktsubo force-pushed the add_ovs_flow_operation_metrics_on_agent branch from a0d6530 to 5dc364d Compare July 23, 2020 22:57

srikartati approved these changes Jul 24, 2020

View reviewed changes

srikartati merged commit 33eee8c into antrea-io:master Jul 24, 2020

	Help: "The duration of OVS flow operation",
	Help: "The latency of OVS flow operations.",

Add support for OVS flow operations metrics on node #866

Add support for OVS flow operations metrics on node #866

Conversation

yktsubo commented Jun 24, 2020 • edited Loading

antrea-bot commented Jun 24, 2020

srikartati left a comment

Choose a reason for hiding this comment

srikartati Jun 24, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

antoninbas left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

srikartati left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yktsubo Jul 1, 2020 • edited Loading

Choose a reason for hiding this comment

srikartati Jul 2, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

srikartati left a comment

Choose a reason for hiding this comment

yktsubo commented Jul 7, 2020

srikartati commented Jul 7, 2020

srikartati commented Jul 7, 2020

yktsubo commented Jul 9, 2020

srikartati commented Jul 13, 2020

srikartati commented Jul 13, 2020

srikartati commented Jul 14, 2020

srikartati left a comment

Choose a reason for hiding this comment

srikartati commented Jul 14, 2020

antoninbas left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yktsubo commented Jul 18, 2020

yktsubo commented Jul 18, 2020

srikartati commented Jul 18, 2020

srikartati commented Jul 20, 2020

yktsubo commented Jul 20, 2020 • edited Loading

yktsubo commented Jul 20, 2020

srikartati commented Jul 20, 2020

yktsubo commented Jul 20, 2020 • edited Loading

srikartati commented Jul 21, 2020

yktsubo commented Jul 22, 2020 • edited Loading

srikartati left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yktsubo commented Jul 23, 2020

srikartati left a comment

Choose a reason for hiding this comment

srikartati commented Jul 24, 2020

srikartati commented Jul 24, 2020

srikartati commented Jul 24, 2020

yktsubo commented Jul 24, 2020

srikartati commented Jul 24, 2020

yktsubo commented Jun 24, 2020 •

edited

Loading

srikartati Jun 24, 2020 •

edited

Loading

yktsubo Jul 1, 2020 •

edited

Loading

srikartati Jul 2, 2020 •

edited

Loading

yktsubo commented Jul 20, 2020 •

edited

Loading

yktsubo commented Jul 20, 2020 •

edited

Loading

yktsubo commented Jul 22, 2020 •

edited

Loading