-
Notifications
You must be signed in to change notification settings - Fork 204
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Development #5
Merged
Merged
Development #5
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In some edge cases, the chunked response might exceed the max response size that we can handle. We use limited reader to read up to the max size.
For "container in container" case, cadvisor will emit 2 stats for the same container which will cause assert failure in extractor.Merge(). The fix has two parts, either one of them can solve the problem: * Ignore the stats of cgroup path for "container in container" * Tolerate two same metrics with different timestamps in Merge(). When a conflict is detected, only keep the metric with earlier timestamp
Background: CWAgent is deployed as Daemonset into the cluster to retrieve per Node metrics. But for the control plane KPI Server metrics, only one copy of metrics is needed, otherwise we will get duplicate CP metrics. K8s leaderelection is used so only the leading CWAgent will retrieve the kpiserver metrics: https://github.com/kubernetes/client-go/blob/master/tools/leaderelection/leaderelection.go#L213 However the code above only allows one CWAgent acts as leader once during its lifetime. During the EKS Cluster upgrade, Amazon EKS launching new API server nodes with the updated Kubernetes version to replace the existing ones. When this takes more than 1 minutes, the leading CWAgent will release the leader role and other CWAgent will take the leader role. But in case the cluster fleet is small and the kpi server reboots multiple times (e.g. upgrade twice without upgrade node group), then all CWAgents have acted as leader once and no CWAgent will take the leader role anymore. Fix: Do multiple leader elections in a loop so that a pod can be elected as leader multiple times (A pod can be elected at most once per leader election process. With multiple leader elections, a pod can becomes leader multiple times)
gmealy1
added a commit
to gmealy1/amazon-cloudwatch-agent
that referenced
this pull request
Jul 7, 2022
# This is the 1st commit message: # This is a combination of 6 commits. # This is the 1st commit message: Initial commit for GetMetricData query utilities added configuration file for CWA test inital commit for data collection/GetMetricData portion of test initial commit for main test driver that starts agent and then calls data collection changed test error to fatal if error is returned by data collection added comments removed unnecessary code that prints output copy CWA config file to it's destination changed log print to error return in utilities. few small formatting changes implemented feedback from PR. Biggest change is dynamically grabbing metric names from config file instead of them being declared in the code fixed creation of metric names for query and added comments fixed config spacing fixed agent runtime addressed more pull request comments. Moved config parsing to a new function Performance Tracking Integration Test and Data Collection (aws#497) * Initial commit for GetMetricData query utilities added configuration file for CWA test inital commit for data collection/GetMetricData portion of test initial commit for main test driver that starts agent and then calls data collection changed test error to fatal if error is returned by data collection added comments removed unnecessary code that prints output * copy CWA config file to it's destination * changed log print to error return in utilities. few small formatting changes * implemented feedback from PR. Biggest change is dynamically grabbing metric names from config file instead of them being declared in the code * fixed creation of metric names for query and added comments * fixed config spacing * fixed agent runtime * addressed more pull request comments. Moved config parsing to a new function added statistics calculation and log monitoring to CWA config. test now writes to log files while CWA monitors Initial commit for GetMetricData query utilities added configuration file for CWA test inital commit for data collection/GetMetricData portion of test initial commit for main test driver that starts agent and then calls data collection changed test error to fatal if error is returned by data collection added comments removed unnecessary code that prints output copy CWA config file to it's destination changed log print to error return in utilities. few small formatting changes implemented feedback from PR. Biggest change is dynamically grabbing metric names from config file instead of them being declared in the code fixed creation of metric names for query and added comments fixed config spacing fixed agent runtime addressed more pull request comments. Moved config parsing to a new function Performance Tracking Integration Test and Data Collection (aws#497) * Initial commit for GetMetricData query utilities added configuration file for CWA test inital commit for data collection/GetMetricData portion of test initial commit for main test driver that starts agent and then calls data collection changed test error to fatal if error is returned by data collection added comments removed unnecessary code that prints output * copy CWA config file to it's destination * changed log print to error return in utilities. few small formatting changes * implemented feedback from PR. Biggest change is dynamically grabbing metric names from config file instead of them being declared in the code * fixed creation of metric names for query and added comments * fixed config spacing * fixed agent runtime * addressed more pull request comments. Moved config parsing to a new function Performance Tracking Integration Test and Data Collection (aws#497) * Initial commit for GetMetricData query utilities added configuration file for CWA test inital commit for data collection/GetMetricData portion of test initial commit for main test driver that starts agent and then calls data collection changed test error to fatal if error is returned by data collection added comments removed unnecessary code that prints output * copy CWA config file to it's destination * changed log print to error return in utilities. few small formatting changes * implemented feedback from PR. Biggest change is dynamically grabbing metric names from config file instead of them being declared in the code * fixed creation of metric names for query and added comments * fixed config spacing * fixed agent runtime * addressed more pull request comments. Moved config parsing to a new function fixed merge conflicts # The commit message aws#2 will be skipped: # Reduce timeout for scrapping IMDS and give instruction when fail to scrape IMDS inside container (aws#480) # # * Fix Aggregrator Shut Down Behavior # # * Always setting hops to 2 if CloudWatchAgent is deployed as container # # * Reduce timeout for scrapping IMDS and give instrucstion when timeout--ammend # # * Reduce timeout for scrapping IMDS and give instrucstion when timeout--ammend # # * Reduce timeout for scrapping IMDS and give instrucstion when timeout--ammend # # * Reduce timeout for scrapping IMDS and give instrucstion when timeout--ammend # # * Reduce timeout for scrapping IMDS and give instrucstion when timeout--ammend # # * Reduce timeout for scrapping IMDS and give instrucstion when timeout--ammend # # * Reduce timeout for scrapping IMDS and give instruction when fail to scrape IMDS inside container # The commit message aws#3 will be skipped: # Export Go Bin Path (aws#498) # # The commit message aws#4 will be skipped: # Performance Tracking Integration Test and Data Collection (aws#497) # # * Initial commit for GetMetricData query utilities # # added configuration file for CWA test # # inital commit for data collection/GetMetricData portion of test # # initial commit for main test driver that starts agent and then calls data collection # # changed test error to fatal if error is returned by data collection # # added comments # # removed unnecessary code that prints output # # * copy CWA config file to it's destination # # * changed log print to error return in utilities. few small formatting changes # # * implemented feedback from PR. Biggest change is dynamically grabbing metric names from config file instead of them being declared in the code # # * fixed creation of metric names for query and added comments # # * fixed config spacing # # * fixed agent runtime # # * addressed more pull request comments. Moved config parsing to a new function # The commit message aws#5 will be skipped: # Update release note V352 (aws#502) # # The commit message aws#6 will be skipped: # Support escape path for MacOs, Linux (aws#499) # # * Support scrapt paths for Linux,MacOs # # * Support escape path for MacOs, Linux # The commit message aws#2 will be skipped: # Initial commit for GetMetricData query utilities # # added configuration file for CWA test # # inital commit for data collection/GetMetricData portion of test # # initial commit for main test driver that starts agent and then calls data collection # # changed test error to fatal if error is returned by data collection # # added comments # # removed unnecessary code that prints output # # copy CWA config file to it's destination # # changed log print to error return in utilities. few small formatting changes # # implemented feedback from PR. Biggest change is dynamically grabbing metric names from config file instead of them being declared in the code # # fixed creation of metric names for query and added comments # # fixed config spacing # # fixed agent runtime # # addressed more pull request comments. Moved config parsing to a new function # # Performance Tracking Integration Test and Data Collection (aws#497) # # * Initial commit for GetMetricData query utilities # # added configuration file for CWA test # # inital commit for data collection/GetMetricData portion of test # # initial commit for main test driver that starts agent and then calls data collection # # changed test error to fatal if error is returned by data collection # # added comments # # removed unnecessary code that prints output # # * copy CWA config file to it's destination # # * changed log print to error return in utilities. few small formatting changes # # * implemented feedback from PR. Biggest change is dynamically grabbing metric names from config file instead of them being declared in the code # # * fixed creation of metric names for query and added comments # # * fixed config spacing # # * fixed agent runtime # # * addressed more pull request comments. Moved config parsing to a new function # # added statistics calculation and log monitoring to CWA config. test now writes to log files while CWA monitors # # Initial commit for GetMetricData query utilities # # added configuration file for CWA test # # inital commit for data collection/GetMetricData portion of test # # initial commit for main test driver that starts agent and then calls data collection # # changed test error to fatal if error is returned by data collection # # added comments # # removed unnecessary code that prints output # # copy CWA config file to it's destination # # changed log print to error return in utilities. few small formatting changes # # implemented feedback from PR. Biggest change is dynamically grabbing metric names from config file instead of them being declared in the code # # fixed creation of metric names for query and added comments # # fixed config spacing # # fixed agent runtime # # addressed more pull request comments. Moved config parsing to a new function # # Performance Tracking Integration Test and Data Collection (aws#497) # # * Initial commit for GetMetricData query utilities # # added configuration file for CWA test # # inital commit for data collection/GetMetricData portion of test # # initial commit for main test driver that starts agent and then calls data collection # # changed test error to fatal if error is returned by data collection # # added comments # # removed unnecessary code that prints output # # * copy CWA config file to it's destination # # * changed log print to error return in utilities. few small formatting changes # # * implemented feedback from PR. Biggest change is dynamically grabbing metric names from config file instead of them being declared in the code # # * fixed creation of metric names for query and added comments # # * fixed config spacing # # * fixed agent runtime # # * addressed more pull request comments. Moved config parsing to a new function # # Performance Tracking Integration Test and Data Collection (aws#497) # # * Initial commit for GetMetricData query utilities # # added configuration file for CWA test # # inital commit for data collection/GetMetricData portion of test # # initial commit for main test driver that starts agent and then calls data collection # # changed test error to fatal if error is returned by data collection # # added comments # # removed unnecessary code that prints output # # * copy CWA config file to it's destination # # * changed log print to error return in utilities. few small formatting changes # # * implemented feedback from PR. Biggest change is dynamically grabbing metric names from config file instead of them being declared in the code # # * fixed creation of metric names for query and added comments # # * fixed config spacing # # * fixed agent runtime # # * addressed more pull request comments. Moved config parsing to a new function # # fixed merge conflicts # # changed period in GetMetrics call 30->10 # The commit message aws#3 will be skipped: # Reduce timeout for scrapping IMDS and give instruction when fail to scrape IMDS inside container (aws#480) # # * Fix Aggregrator Shut Down Behavior # # * Always setting hops to 2 if CloudWatchAgent is deployed as container # # * Reduce timeout for scrapping IMDS and give instrucstion when timeout--ammend # # * Reduce timeout for scrapping IMDS and give instrucstion when timeout--ammend # # * Reduce timeout for scrapping IMDS and give instrucstion when timeout--ammend # # * Reduce timeout for scrapping IMDS and give instrucstion when timeout--ammend # # * Reduce timeout for scrapping IMDS and give instrucstion when timeout--ammend # # * Reduce timeout for scrapping IMDS and give instrucstion when timeout--ammend # # * Reduce timeout for scrapping IMDS and give instruction when fail to scrape IMDS inside container # The commit message aws#4 will be skipped: # Export Go Bin Path (aws#498) # # The commit message aws#5 will be skipped: # Performance Tracking Integration Test and Data Collection (aws#497) # # * Initial commit for GetMetricData query utilities # # added configuration file for CWA test # # inital commit for data collection/GetMetricData portion of test # # initial commit for main test driver that starts agent and then calls data collection # # changed test error to fatal if error is returned by data collection # # added comments # # removed unnecessary code that prints output # # * copy CWA config file to it's destination # # * changed log print to error return in utilities. few small formatting changes # # * implemented feedback from PR. Biggest change is dynamically grabbing metric names from config file instead of them being declared in the code # # * fixed creation of metric names for query and added comments # # * fixed config spacing # # * fixed agent runtime # # * addressed more pull request comments. Moved config parsing to a new function # The commit message aws#6 will be skipped: # Update release note V352 (aws#502) # # The commit message aws#7 will be skipped: # Support escape path for MacOs, Linux (aws#499) # # * Support scrapt paths for Linux,MacOs # # * Support escape path for MacOs, Linux
sethAmazon
pushed a commit
that referenced
this pull request
Aug 11, 2022
* Basic functionality for backend transmitter removed unit tests playing around with github changed package name seperated the test case reformatted removed seperate test changed package new IntegrationTest Setup fixed integrationTest.yml applied pr feedback: -added new stored data called "CommitDate" -reorganized code -updated code to go v2 -changed metrics struct cleaned workflow init auto run typo auto test cleanworkflow init added performance test to integrationTest changed epoch time to int -after that became an issue in frontend Delete fastPerformanceTest.yml Added PR requested changes and comments backend Fix * go mod tidy update * Workflow (#5) * concurrent workflow added *update item added *release detection added * edited test * added pt to master * remove extra vars * go mod tidy * fixed imports * import bug * fixed value bug * updated iam roles * New workflow (#8) * fixed sendItem nil map error * added concurreny retrying * fixed no value bug * fixed value bug * demo-4 * demo 1 * Fix debug logging and revert Windows ctl script PS1 changes (#516) * change to use fmt * Revert "Using cmd/c to avoid PowerShell ISE does not accept stderr (#473)" This reverts commit 89d1912. * remove log entirely * update release notes * Performance Benchmarking Log monitor and TPS tests (#513) * added concurrent log monitoring/writing to test and tests several combinations of # of logs monitored and different tps * fixed call to StartLogWrite * restructured json packet to send to database. CWA configs are now generated at runtime. log number is given as an environment variable so tests can run concurrently, so commented code is included for that change * rewrote how databse entry packets are crafted/create, defer log stream/group deletions, addressed other small pr comments * removed extra metrics from CWA config * separated delete log group and streams so they can be called separately, but kept the original function definition as to not break other code. removed bad calls to DeleteLogGroupAndStreams * testing with _GAVIN'S_ code * debug print check * changed parser * testing with concurrency version * changeg setenv to string * added comments Co-authored-by: Andrew Huynh <17056456+SaxyPandaBear@users.noreply.github.com> Co-authored-by: gmealy1 <107051245+gmealy1@users.noreply.github.com> * demo-0 * test for pr * testing running performance test on prev. releases * test 1 * test - 2 pRPT * genMatrix test * test * test * t * t * t * te * test * test * test * test * test * test * test * t * t * testing hard coded * full test w 3 old releases * test * testing with default * redid terraform for previousData.yml * debugging * test binary making * trying with new terraform * try * testing new terraform with git stash * forgot comma * one more comma :/ * sha to old_sha * testing getting releases on workflow * same * test * t * no formatting * git show --tags * trying with sep shell script * trying .sh directly * log cat * :/ * git * git show --tags * trying query tag * git fetch * git fetch w show * simplify * git tests * query * git describe tags * :( * not shallow * query t * testing * t * 3 test * testing update config * added concurrent update release tag * try2 * Testing releases from 1627262978 to 1658798978 * working release * Testing releases from 1627262978 to 1653934651 * Testing releases from 1622398651 to 1653934651 * Testing releases from to * Testing releases from Sun May 30 18:17:31 UTC 2021 to Mon May 30 18:17:31 UTC 2022 * Testing releases from 20210530 to 20220530 * updated shell * Testing releases from 2021/05/30 to 2022/05/30 * updated previousData.yml * fixed stoplocal dep * generator duplicate fix * fixed the assume role in env * Testing releases from 2021/05/30 to 2022/05/30 * testing Co-authored-by: Andrew Huynh <17056456+SaxyPandaBear@users.noreply.github.com> Co-authored-by: gmealy1 <107051245+gmealy1@users.noreply.github.com>
sethAmazon
added a commit
that referenced
this pull request
Aug 11, 2022
* Basic functionality for backend transmitter removed unit tests playing around with github changed package name seperated the test case reformatted removed seperate test changed package new IntegrationTest Setup fixed integrationTest.yml applied pr feedback: -added new stored data called "CommitDate" -reorganized code -updated code to go v2 -changed metrics struct cleaned workflow init auto run typo auto test cleanworkflow init added performance test to integrationTest changed epoch time to int -after that became an issue in frontend Delete fastPerformanceTest.yml Added PR requested changes and comments backend Fix * go mod tidy update * Workflow (#5) * concurrent workflow added *update item added *release detection added * edited test * added pt to master * remove extra vars * go mod tidy * fixed imports * import bug * fixed value bug * updated iam roles * New workflow (#8) * fixed sendItem nil map error * added concurreny retrying * fixed no value bug * fixed value bug * demo-4 * demo 1 * Fix debug logging and revert Windows ctl script PS1 changes (#516) * change to use fmt * Revert "Using cmd/c to avoid PowerShell ISE does not accept stderr (#473)" This reverts commit 89d1912. * remove log entirely * update release notes * Performance Benchmarking Log monitor and TPS tests (#513) * added concurrent log monitoring/writing to test and tests several combinations of # of logs monitored and different tps * fixed call to StartLogWrite * restructured json packet to send to database. CWA configs are now generated at runtime. log number is given as an environment variable so tests can run concurrently, so commented code is included for that change * rewrote how databse entry packets are crafted/create, defer log stream/group deletions, addressed other small pr comments * removed extra metrics from CWA config * separated delete log group and streams so they can be called separately, but kept the original function definition as to not break other code. removed bad calls to DeleteLogGroupAndStreams * testing with _GAVIN'S_ code * debug print check * changed parser * testing with concurrency version * changeg setenv to string * added comments Co-authored-by: Andrew Huynh <17056456+SaxyPandaBear@users.noreply.github.com> Co-authored-by: gmealy1 <107051245+gmealy1@users.noreply.github.com> * demo-0 * test for pr * testing running performance test on prev. releases * test 1 * test - 2 pRPT * genMatrix test * test * test * t * t * t * te * test * test * test * test * test * test * test * t * t * testing hard coded * full test w 3 old releases * test * testing with default * redid terraform for previousData.yml * debugging * test binary making * trying with new terraform * try * testing new terraform with git stash * forgot comma * one more comma :/ * sha to old_sha * testing getting releases on workflow * same * test * t * no formatting * git show --tags * trying with sep shell script * trying .sh directly * log cat * :/ * git * git show --tags * trying query tag * git fetch * git fetch w show * simplify * git tests * query * git describe tags * :( * not shallow * query t * testing * t * 3 test * testing update config * added concurrent update release tag * try2 * Testing releases from 1627262978 to 1658798978 * working release * Testing releases from 1627262978 to 1653934651 * Testing releases from 1622398651 to 1653934651 * Testing releases from to * Testing releases from Sun May 30 18:17:31 UTC 2021 to Mon May 30 18:17:31 UTC 2022 * Testing releases from 20210530 to 20220530 * updated shell * Testing releases from 2021/05/30 to 2022/05/30 * updated previousData.yml * fixed stoplocal dep * generator duplicate fix * fixed the assume role in env * Testing releases from 2021/05/30 to 2022/05/30 * testing Co-authored-by: Andrew Huynh <17056456+SaxyPandaBear@users.noreply.github.com> Co-authored-by: gmealy1 <107051245+gmealy1@users.noreply.github.com> Co-authored-by: okankoAMZ <107267850+okankoAMZ@users.noreply.github.com> Co-authored-by: Andrew Huynh <17056456+SaxyPandaBear@users.noreply.github.com> Co-authored-by: gmealy1 <107051245+gmealy1@users.noreply.github.com>
sethAmazon
pushed a commit
that referenced
this pull request
Aug 15, 2022
* Basic functionality for backend transmitter removed unit tests playing around with github changed package name seperated the test case reformatted removed seperate test changed package new IntegrationTest Setup fixed integrationTest.yml applied pr feedback: -added new stored data called "CommitDate" -reorganized code -updated code to go v2 -changed metrics struct cleaned workflow init auto run typo auto test cleanworkflow init added performance test to integrationTest changed epoch time to int -after that became an issue in frontend Delete fastPerformanceTest.yml Added PR requested changes and comments backend Fix * go mod tidy update * Workflow (#5) * concurrent workflow added *update item added *release detection added * edited test * added pt to master * remove extra vars * go mod tidy * fixed imports * import bug * fixed value bug * updated iam roles * New workflow (#8) * fixed sendItem nil map error * added concurreny retrying * fixed no value bug * fixed value bug * demo-4 * demo 1 * Fix debug logging and revert Windows ctl script PS1 changes (#516) * change to use fmt * Revert "Using cmd/c to avoid PowerShell ISE does not accept stderr (#473)" This reverts commit 89d1912. * remove log entirely * update release notes * Performance Benchmarking Log monitor and TPS tests (#513) * added concurrent log monitoring/writing to test and tests several combinations of # of logs monitored and different tps * fixed call to StartLogWrite * restructured json packet to send to database. CWA configs are now generated at runtime. log number is given as an environment variable so tests can run concurrently, so commented code is included for that change * rewrote how databse entry packets are crafted/create, defer log stream/group deletions, addressed other small pr comments * removed extra metrics from CWA config * separated delete log group and streams so they can be called separately, but kept the original function definition as to not break other code. removed bad calls to DeleteLogGroupAndStreams * testing with _GAVIN'S_ code * debug print check * changed parser * testing with concurrency version * changeg setenv to string * added comments Co-authored-by: Andrew Huynh <17056456+SaxyPandaBear@users.noreply.github.com> Co-authored-by: gmealy1 <107051245+gmealy1@users.noreply.github.com> * demo-0 * test for pr * testing running performance test on prev. releases * test 1 * test - 2 pRPT * genMatrix test * test * test * t * t * t * te * test * test * test * test * test * test * test * t * t * testing hard coded * full test w 3 old releases * test * testing with default * redid terraform for previousData.yml * debugging * test binary making * trying with new terraform * try * testing new terraform with git stash * forgot comma * one more comma :/ * sha to old_sha * testing getting releases on workflow * same * test * t * no formatting * git show --tags * trying with sep shell script * trying .sh directly * log cat * :/ * git * git show --tags * trying query tag * git fetch * git fetch w show * simplify * git tests * query * git describe tags * :( * not shallow * query t * testing * t * 3 test * testing update config * added concurrent update release tag * try2 * Testing releases from 1627262978 to 1658798978 * working release * Testing releases from 1627262978 to 1653934651 * Testing releases from 1622398651 to 1653934651 * Testing releases from to * Testing releases from Sun May 30 18:17:31 UTC 2021 to Mon May 30 18:17:31 UTC 2022 * Testing releases from 20210530 to 20220530 * updated shell * Testing releases from 2021/05/30 to 2022/05/30 * updated previousData.yml * fixed stoplocal dep * generator duplicate fix * fixed the assume role in env * Testing releases from 2021/05/30 to 2022/05/30 * testing * updated the performance test to most recent * only use s3 not make build * Testing releases from 2022/06/01 to 2022/08/01 * added debug param * Testing releases from 2022/07/06 to 2022/08/01 * Testing releases from 2021/05/30 to 2022/05/30 * Testing releases from 2022/06/01 to 2022/08/01 * Testing releases from 2022/06/01 to 2022/08/01 * export fix in terraform * terraform typo * formatting changes * add print statement * export added to tpdate release * typo fix * Testing releases from 2021/05/30 to 2022/05/30 Co-authored-by: Andrew Huynh <17056456+SaxyPandaBear@users.noreply.github.com> Co-authored-by: gmealy1 <107051245+gmealy1@users.noreply.github.com>
sethAmazon
pushed a commit
that referenced
this pull request
Aug 16, 2022
* Basic functionality for backend transmitter removed unit tests playing around with github changed package name seperated the test case reformatted removed seperate test changed package new IntegrationTest Setup fixed integrationTest.yml applied pr feedback: -added new stored data called "CommitDate" -reorganized code -updated code to go v2 -changed metrics struct cleaned workflow init auto run typo auto test cleanworkflow init added performance test to integrationTest changed epoch time to int -after that became an issue in frontend Delete fastPerformanceTest.yml Added PR requested changes and comments backend Fix * go mod tidy update * Workflow (#5) * concurrent workflow added *update item added *release detection added * edited test * added pt to master * remove extra vars * go mod tidy * fixed imports * import bug * fixed value bug * updated iam roles * New workflow (#8) * fixed sendItem nil map error * added concurreny retrying * fixed no value bug * fixed value bug * demo-4 * demo 1 * Fix debug logging and revert Windows ctl script PS1 changes (#516) * change to use fmt * Revert "Using cmd/c to avoid PowerShell ISE does not accept stderr (#473)" This reverts commit 89d1912. * remove log entirely * update release notes * Performance Benchmarking Log monitor and TPS tests (#513) * added concurrent log monitoring/writing to test and tests several combinations of # of logs monitored and different tps * fixed call to StartLogWrite * restructured json packet to send to database. CWA configs are now generated at runtime. log number is given as an environment variable so tests can run concurrently, so commented code is included for that change * rewrote how databse entry packets are crafted/create, defer log stream/group deletions, addressed other small pr comments * removed extra metrics from CWA config * separated delete log group and streams so they can be called separately, but kept the original function definition as to not break other code. removed bad calls to DeleteLogGroupAndStreams * testing with _GAVIN'S_ code * debug print check * changed parser * testing with concurrency version * changeg setenv to string * added comments Co-authored-by: Andrew Huynh <17056456+SaxyPandaBear@users.noreply.github.com> Co-authored-by: gmealy1 <107051245+gmealy1@users.noreply.github.com> * demo-0 * test for pr * testing running performance test on prev. releases * test 1 * test - 2 pRPT * genMatrix test * test * test * t * t * t * te * test * test * test * test * test * test * test * t * t * testing hard coded * full test w 3 old releases * test * testing with default * redid terraform for previousData.yml * debugging * test binary making * trying with new terraform * try * testing new terraform with git stash * forgot comma * one more comma :/ * sha to old_sha * testing getting releases on workflow * same * test * t * no formatting * git show --tags * trying with sep shell script * trying .sh directly * log cat * :/ * git * git show --tags * trying query tag * git fetch * git fetch w show * simplify * git tests * query * git describe tags * :( * not shallow * query t * testing * t * 3 test * testing update config * added concurrent update release tag * try2 * Testing releases from 1627262978 to 1658798978 * working release * Testing releases from 1627262978 to 1653934651 * Testing releases from 1622398651 to 1653934651 * Testing releases from to * Testing releases from Sun May 30 18:17:31 UTC 2021 to Mon May 30 18:17:31 UTC 2022 * Testing releases from 20210530 to 20220530 * updated shell * Testing releases from 2021/05/30 to 2022/05/30 * updated previousData.yml * fixed stoplocal dep * generator duplicate fix * fixed the assume role in env * Testing releases from 2021/05/30 to 2022/05/30 * testing * updated the performance test to most recent * only use s3 not make build * Testing releases from 2022/06/01 to 2022/08/01 * added debug param * Testing releases from 2022/07/06 to 2022/08/01 * Testing releases from 2021/05/30 to 2022/05/30 * Testing releases from 2022/06/01 to 2022/08/01 * Testing releases from 2022/06/01 to 2022/08/01 * export fix in terraform * terraform typo * formatting changes * add print statement * export added to tpdate release * typo fix * Testing releases from 2021/05/30 to 2022/05/30 * Testing releases from 2022/07/01 to 2022/08/01 * Testing releases from 2022/07/01 to 2022/08/01 * Testing releases from 2022/07/01 to 2022/08/01 * Testing releases from 2022/07/01 to 2022/08/01 * Testing releases from 2022/07/24 to 2022/08/01 * Testing releases from 2022/07/24 to 2022/08/01 * Testing releases from 2022/07/27 to 2022/08/01 * Testing releases from 2022/07/27 to 2022/08/01 * trying hard code version * dep fix * Testing releases from 2021/05/30 to 2022/05/30 Co-authored-by: Andrew Huynh <17056456+SaxyPandaBear@users.noreply.github.com> Co-authored-by: gmealy1 <107051245+gmealy1@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.