Group same timestamp metrics values in app_insights metricset #20403

narph · 2020-08-03T16:08:26Z

What does this PR do?

groups same timestamp metric values
removes any nil values

Why is it important?

groups same timestamp metric values
removes any nil values
existing test should cover changes

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

elasticmachine · 2020-08-03T16:11:36Z

Pinging @elastic/integrations (Team:Integrations)

elasticmachine · 2020-08-03T16:11:36Z

Pinging @elastic/integrations-platforms (Team:Platforms)

elasticmachine · 2020-08-03T16:13:51Z

💚 Build Succeeded

Expand to view the summary

Build stats

Build Cause: [Pull request #20403 updated]
Start Time: 2020-08-03T17:15:26.059+0000
Duration: 39 min 50 sec

Test stats 🧪

Test	Results
Failed	0
Passed	824
Skipped	84
Total	908

jsoriano

LGTM, added a couple of small suggestions.

jsoriano · 2020-08-03T17:20:49Z

x-pack/metricbeat/module/azure/app_insights/data.go

+ groupedAddProp[fmt.Sprintf("%sTO%s", item.Body.Value.Start, item.Body.Value.End)] =
+ append(groupedAddProp[fmt.Sprintf("%sTO%s", item.Body.Value.Start, item.Body.Value.End)], *item.Body.Value)


Consider calculating the key in a single place to avoid mistakes.

Suggested change

groupedAddProp[fmt.Sprintf("%sTO%s", item.Body.Value.Start, item.Body.Value.End)] =

append(groupedAddProp[fmt.Sprintf("%sTO%s", item.Body.Value.Start, item.Body.Value.End)], *item.Body.Value)

key := fmt.Sprintf("%sTO%s", item.Body.Value.Start, item.Body.Value.End)

groupedAddProp[key] = append(groupedAddProp[key], *item.Body.Value)

jsoriano · 2020-08-03T17:37:16Z

x-pack/metricbeat/module/azure/app_insights/data.go

@@ -66,7 +103,9 @@ func getMetric(addProp map[string]interface{}) map[string]interface{} {
 switch val.(type) {
 case map[string]interface{}:
 for subKey, subVal := range val.(map[string]interface{}) {
- metricNames[cleanMetricNames(fmt.Sprintf("%s.%s", key, subKey))] = subVal
+ if subVal != nil {


Would the removal of nil values need another changelog entry?

I do not think so, this is normal behavior, the app insights have no data for these metrics so no data will be sent.

…c#20403) * mofidy doc * work on data * changelog * mage fmt update (cherry picked from commit b356494)

#20418) * mofidy doc * work on data * changelog * mage fmt update (cherry picked from commit b356494)

#20419) * mofidy doc * work on data * changelog * mage fmt update (cherry picked from commit b356494)

…ne-2.0 * upstream/master: [docs] Promote ingest management to beta (elastic#20295) Upgrade elasticsearch client library used in tests (elastic#20405) Disable logging when pulling on python integration tests (elastic#20397) Remove pillow from testing requirements.txt (elastic#20407) [Filebeat][ATP Module]Setting user agent field required by the API (elastic#20440) [Ingest Manager] Send datastreams fields (elastic#20402) Add event.ingested to all Filebeat modules (elastic#20386) [Elastic Agent] Fix agent control socket path to always be less than 107 characters (elastic#20426) Improve cgroup_regex docs with examples (elastic#20425) Makes `metrics` config option required in app_insights (elastic#20406) Ensure install scripts only install if needed (elastic#20349) Update container name for the azure filesets (elastic#19899) Group same timestamp metrics values in app_insights metricset (elastic#20403) add_process_metadata processor adds container id even if process metadata not accessible (elastic#19767) Support "cluster" scope in Metricbeat elasticsearch module (elastic#18547) [Filebeat][SophosXG Module] Renaming module and fileset (elastic#20396) Update Suricata dashboards (elastic#20394) [Elastic Agent] Improve version, restart, enroll CLI commands (elastic#20359) Prepare home directories for docker images in a different stage (elastic#20356)

…allation * upstream/master: (23 commits) [docs] Promote ingest management to beta (elastic#20295) Upgrade elasticsearch client library used in tests (elastic#20405) Disable logging when pulling on python integration tests (elastic#20397) Remove pillow from testing requirements.txt (elastic#20407) [Filebeat][ATP Module]Setting user agent field required by the API (elastic#20440) [Ingest Manager] Send datastreams fields (elastic#20402) Add event.ingested to all Filebeat modules (elastic#20386) [Elastic Agent] Fix agent control socket path to always be less than 107 characters (elastic#20426) Improve cgroup_regex docs with examples (elastic#20425) Makes `metrics` config option required in app_insights (elastic#20406) Ensure install scripts only install if needed (elastic#20349) Update container name for the azure filesets (elastic#19899) Group same timestamp metrics values in app_insights metricset (elastic#20403) add_process_metadata processor adds container id even if process metadata not accessible (elastic#19767) Support "cluster" scope in Metricbeat elasticsearch module (elastic#18547) [Filebeat][SophosXG Module] Renaming module and fileset (elastic#20396) Update Suricata dashboards (elastic#20394) [Elastic Agent] Improve version, restart, enroll CLI commands (elastic#20359) Prepare home directories for docker images in a different stage (elastic#20356) New multiline mode in Filebeat: while_pattern (elastic#19662) ...

…c#20403) * mofidy doc * work on data * changelog * mage fmt update

…c#20403) (elastic#20419) * mofidy doc * work on data * changelog * mage fmt update (cherry picked from commit fe78a57)

narph added 3 commits July 23, 2020 11:01

mofidy doc

938e66c

Merge branch 'master' of github.com:elastic/beats

4daef08

work on data

f7a57de

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Aug 3, 2020

changelog

5897911

narph self-assigned this Aug 3, 2020

narph added [zube]: In Review Team:Integrations Label for the Integrations team Team:Platforms Label for the Integrations - Platforms team labels Aug 3, 2020

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Aug 3, 2020

narph added the needs_backport PR is waiting to be backported to other branches. label Aug 3, 2020

mage fmt update

fcd3637

jsoriano approved these changes Aug 3, 2020

View reviewed changes

kaiyan-sheng approved these changes Aug 3, 2020

View reviewed changes

narph merged commit b356494 into elastic:master Aug 4, 2020

narph deleted the insights-return branch August 4, 2020 08:04

narph added a commit to narph/beats that referenced this pull request Aug 4, 2020

Group same timestamp metrics values in app_insights metricset (elasti…

3d50613

…c#20403) * mofidy doc * work on data * changelog * mage fmt update (cherry picked from commit b356494)

narph mentioned this pull request Aug 4, 2020

Cherry-pick #20403 to 7.x: Group same timestamp metrics values in app_insights metricset #20418

Merged

6 tasks

narph added a commit to narph/beats that referenced this pull request Aug 4, 2020

Group same timestamp metrics values in app_insights metricset (elasti…

4b0b6b2

…c#20403) * mofidy doc * work on data * changelog * mage fmt update (cherry picked from commit b356494)

narph mentioned this pull request Aug 4, 2020

Cherry-pick #20403 to 7.9: Group same timestamp metrics values in app_insights metricset #20419

Merged

6 tasks

narph added a commit that referenced this pull request Aug 4, 2020

Group same timestamp metrics values in app_insights metricset (#20403) (

1a75279

#20418) * mofidy doc * work on data * changelog * mage fmt update (cherry picked from commit b356494)

narph added a commit that referenced this pull request Aug 4, 2020

Group same timestamp metrics values in app_insights metricset (#20403) (

cabf3ee

#20419) * mofidy doc * work on data * changelog * mage fmt update (cherry picked from commit b356494)

melchiormoulin pushed a commit to melchiormoulin/beats that referenced this pull request Oct 14, 2020

Group same timestamp metrics values in app_insights metricset (elasti…

ed27b20

…c#20403) * mofidy doc * work on data * changelog * mage fmt update

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Group same timestamp metrics values in app_insights metricset #20403

Group same timestamp metrics values in app_insights metricset #20403

narph commented Aug 3, 2020 •

edited

Loading

elasticmachine commented Aug 3, 2020

elasticmachine commented Aug 3, 2020

elasticmachine commented Aug 3, 2020 •

edited by jenkins-beats-ci bot

Loading

Build stats

Test stats 🧪

jsoriano left a comment

jsoriano Aug 3, 2020

jsoriano Aug 3, 2020

narph Aug 4, 2020

		groupedAddProp[fmt.Sprintf("%sTO%s", item.Body.Value.Start, item.Body.Value.End)] =
		append(groupedAddProp[fmt.Sprintf("%sTO%s", item.Body.Value.Start, item.Body.Value.End)], *item.Body.Value)

Group same timestamp metrics values in app_insights metricset #20403

Group same timestamp metrics values in app_insights metricset #20403

Conversation

narph commented Aug 3, 2020 • edited Loading

What does this PR do?

Why is it important?

Checklist

elasticmachine commented Aug 3, 2020

elasticmachine commented Aug 3, 2020

elasticmachine commented Aug 3, 2020 • edited by jenkins-beats-ci bot Loading

💚 Build Succeeded

Build stats

Test stats 🧪

jsoriano left a comment

Choose a reason for hiding this comment

jsoriano Aug 3, 2020

Choose a reason for hiding this comment

jsoriano Aug 3, 2020

Choose a reason for hiding this comment

narph Aug 4, 2020

Choose a reason for hiding this comment

narph commented Aug 3, 2020 •

edited

Loading

elasticmachine commented Aug 3, 2020 •

edited by jenkins-beats-ci bot

Loading