Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change to use one period between start and end time #27327

Merged
merged 4 commits into from
Aug 16, 2021
Merged

Change to use one period between start and end time #27327

merged 4 commits into from
Aug 16, 2021

Conversation

kaiyan-sheng
Copy link
Contributor

@kaiyan-sheng kaiyan-sheng commented Aug 11, 2021

What does this PR do?

This PR is to change startTime and endTime of GetMetricData API in cloudwatch metricbeat to be only one collection period apart. Bug in previous code is:

  • First Metricbeat collection cycle: startTime = 06:50:35 endTime = 07:00:35 -> cloudwatch metric timestamp = 06:55:00

  • Second Metricbeat collection cycle: startTime = 06:55:35 endTime = 07:05:35 -> cloudwatch metric timestamp = 06:55:00

The problem is caused by the startTime and endTime parameters in AWS CloudWatch GetMetricData API only look at the hour and minute (e.g: 06:50 and 07:00) in a timestamp. And also the gap between startTime and endTime is double the period, which causes more than one data point gets returned.

This PR is to first round the timestamps to ignore seconds. Then change startTime to be one collection period ahead of endTime. If no data is collected, which means services have a delay to send metrics into AWS CloudWatch. In this case, latency config parameter should be used.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Aug 11, 2021
@kaiyan-sheng kaiyan-sheng self-assigned this Aug 11, 2021
@kaiyan-sheng kaiyan-sheng added backport-v7.14.0 Automated backport with mergify backport-v7.15.0 Automated backport with mergify Team:Integrations Label for the Integrations team labels Aug 11, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations (Team:Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Aug 11, 2021
@mergify
Copy link
Contributor

mergify bot commented Aug 11, 2021

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b start_end_time upstream/start_end_time
git merge upstream/master
git push upstream start_end_time

@kaiyan-sheng kaiyan-sheng requested a review from aspacca August 11, 2021 22:14
@elasticmachine
Copy link
Collaborator

elasticmachine commented Aug 11, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2021-08-16T15:37:19.814+0000

  • Duration: 106 min 44 sec

  • Commit: 0b3074b

Test stats 🧪

Test Results
Failed 0
Passed 9435
Skipped 2517
Total 11952

Trends 🧪

Image of Build Times

Image of Tests

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 9435
Skipped 2517
Total 11952

@kaiyan-sheng kaiyan-sheng merged commit 35454a7 into elastic:master Aug 16, 2021
@kaiyan-sheng kaiyan-sheng deleted the start_end_time branch August 16, 2021 18:09
mergify bot pushed a commit that referenced this pull request Aug 16, 2021
mergify bot pushed a commit that referenced this pull request Aug 16, 2021
kaiyan-sheng added a commit that referenced this pull request Aug 16, 2021
(cherry picked from commit 35454a7)

Co-authored-by: kaiyan-sheng <kaiyan.sheng@elastic.co>
kaiyan-sheng added a commit that referenced this pull request Aug 16, 2021
(cherry picked from commit 35454a7)

Co-authored-by: kaiyan-sheng <kaiyan.sheng@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-v7.14.0 Automated backport with mergify backport-v7.15.0 Automated backport with mergify Team:Integrations Label for the Integrations team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants