-
Notifications
You must be signed in to change notification settings - Fork 29
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
First ECS metric sanity test - ECS EC2 Launch Type Daemon Deployment - Container Insights #26
Merged
Merged
Changes from all commits
Commits
Show all changes
109 commits
Select commit
Hold shift + click to select a range
673a1be
make an always failing test that gets called for ecs in benchmark test
ChenaLee 4283eab
Terraform required files for daemon run
ChenaLee 4482e8a
Put new parameter for test
ChenaLee 4f7c280
Fix volumes to volume ecs task definition
ChenaLee 222e454
fix volume - terraform doesn't support what json supports
ChenaLee 3d41278
Remove comma so terraform apply doesn't error out
ChenaLee ad21cd0
Fix ssm param name for cwagent config
ChenaLee 4d4025a
Remove sourcePath as fargate doesn't support it
ChenaLee a55321f
correct test directory path
ChenaLee bae3901
Fix genereator test dir name
ChenaLee 572281b
Add dependency for ssm
ChenaLee b87ba85
add session as dependency
ChenaLee b305092
Return error only for ssm param put
ChenaLee 09459f7
remove unused declaration in put param
ChenaLee 57e4cf5
fix list syntax
ChenaLee a47d029
Add flag as dependency for build
ChenaLee f41e028
fix build issues
ChenaLee 2901e76
fix build error
ChenaLee bfd3905
Build fix by changing to non test file
ChenaLee c3f96bb
Fix build failures
ChenaLee 6c6d646
fix build failure
ChenaLee 0028108
Remove import cycle
ChenaLee f36f9fb
fix redeclaration problem
ChenaLee 2a3c004
Fix pointer issue
ChenaLee ddce041
fix cpu config path
ChenaLee 473d2cc
print error for put param
ChenaLee e21c45a
fix import
ChenaLee 861acd2
allow ssm param overwrite
ChenaLee c8f0923
make boolean pointer
ChenaLee 8efa964
ListTasks in order to stop that task
ChenaLee d73195c
fix build issue
ChenaLee 1497046
Fix servicename flag
ChenaLee 68ddee4
Stop cwagent task in order to restart to get the new config
ChenaLee 251e5f1
force update service to restart service
ChenaLee aed95f8
container insights test
ChenaLee 12dc3ba
add volumes
ChenaLee 72caf40
Add ec2 creation for ec2 launch type ecs
ChenaLee c03791c
fix volume definitions for task def for ec2
ChenaLee 68b9563
Change from fargate to ec2 launch type
ChenaLee bc368c8
fix volume definition syntax
ChenaLee fd94756
Fix volume definition syntax 2
ChenaLee 01761bb
Fix iam role for instance profile
ChenaLee a132857
fix iam duplicated name
ChenaLee 0c1f2ce
Change task definition to ec2 launch type
ChenaLee e4d982b
Change network mode to bridge to avoid private subnet problems
ChenaLee 738355d
change launch config user data to add container instance propagate tags
ChenaLee ce13255
change user data to instance tags
ChenaLee b2482ce
Change to al2 ami hoping that is ecs optimized
ChenaLee 87c58bd
Add ecs agent install script to each launched ec2 instance
ChenaLee 74f236d
dedup name for iam policy
ChenaLee 85a907e
use ecs optimized ami
ChenaLee 834a366
fix policy arn for ec2 register to ecs
ChenaLee 4a7ddcc
change user data
ChenaLee 2d4318d
Remove empty volumesFrom
ChenaLee 52db802
add mountpoints to default taskdef
ChenaLee 8fae912
Fix json format
ChenaLee 508896e
change to daemon deployment
ChenaLee 7d0b4b4
organize terraform and ecs templates for ecs ec2 launch daemon
ChenaLee 1336bf8
Fix fargate test generator
ChenaLee 39b6e94
Remove desired count from daemon deployment
ChenaLee 64b2345
Fix daemon restart with no desired count
ChenaLee 0d16995
fix syntax
ChenaLee 02a9d55
Remove test timeout
ChenaLee c461144
Cleanup environment var flag getting logic
ChenaLee 46e63b4
Fix how env variables get passed
ChenaLee e677594
Add agent run strategy poc
ChenaLee b7dbdcc
Fix printf arg
ChenaLee c693c16
Fix flag parsing logic
ChenaLee 8378afb
Move on to see what validation is done
ChenaLee d6b5afe
add query for test validation container insights
ChenaLee d288412
remove mountpoints from fargate task def
ChenaLee 5c4aeaf
Make ECS flags not required for non ecs tests
ChenaLee 9008d7e
add ec2 as compute type for ec2 test
ChenaLee 573438b
add logs for container insights query
ChenaLee 645b510
add more logs around erros
ChenaLee e29e997
add more logs and fix build
ChenaLee 2393776
Try to add Env to each query fetcher struct
ChenaLee 9399fc4
Fix import
ChenaLee 062d916
Fix base member variable usage
ChenaLee e14cd42
Fix getEnv method
ChenaLee 1300420
Fix template file path for existing fargate test
ChenaLee 2720bb1
fix dimensions
ChenaLee 68be845
Fix agent run time and dimensions
ChenaLee 27da732
Fix resource relative path
ChenaLee e3dd20a
Remove debug logs and add test result print
ChenaLee e60fc2e
add log
ChenaLee c1fecdc
Remove always fail
ChenaLee daba08e
Readme edit
ChenaLee 5dee2fa
Fix Fetch implementation
ChenaLee 7794e41
fix build failure
ChenaLee f6449e8
Add instance id dimension for diskio and procstat
ChenaLee da188c5
Print test result in test run instead of teardown because teardown do…
ChenaLee faacd84
delete unused file for now
ChenaLee c181ae5
make ecs test fail to see if logs get printed
ChenaLee 11947ed
make test successful and see if logs get printed still
ChenaLee 8722278
Fix no logs if test succeed problem for ecs test
ChenaLee aa26acb
Edit readme
ChenaLee 0299c7a
Resolve PR comments for /environment
ChenaLee 25be84b
Remove aws-sdk-go v1 dependency
ChenaLee dd94fb6
Remove unnecessary terraform comment
ChenaLee 7255367
Remove unnecessary pointer arguments
ChenaLee d788326
Some minor style changes
ChenaLee e440dce
Remove covered test list since out of scope
ChenaLee 28f8d98
Style changes
ChenaLee 03014e6
fix build
ChenaLee dfbaf94
Merge conflict resolution
ChenaLee 442bdf6
go mod tidy
ChenaLee 1af9eee
Minor style changes
ChenaLee 7e7ac96
go mod tidy
ChenaLee File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,96 @@ | ||
| // Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. | ||
| // SPDX-License-Identifier: MIT | ||
|
|
||
| //go:build integration | ||
SaxyPandaBear marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| // +build integration | ||
|
|
||
| package environment | ||
|
|
||
| import ( | ||
| "flag" | ||
| "github.com/aws/amazon-cloudwatch-agent-test/environment/computetype" | ||
| "github.com/aws/amazon-cloudwatch-agent-test/environment/ecsdeploymenttype" | ||
| "github.com/aws/amazon-cloudwatch-agent-test/environment/ecslaunchtype" | ||
| "github.com/aws/amazon-cloudwatch-agent-test/test" | ||
| "log" | ||
| ) | ||
|
|
||
| type MetaData struct { | ||
| ComputeType computetype.ComputeType | ||
| EcsLaunchType ecslaunchtype.ECSLaunchType | ||
| EcsDeploymentStrategy ecsdeploymenttype.ECSDeploymentType | ||
| EcsClusterArn string | ||
| EcsClusterName string | ||
| CwagentConfigSsmParamName string | ||
| EcsServiceName string | ||
| } | ||
|
|
||
| type MetaDataStrings struct { | ||
| ComputeType string | ||
| EcsLaunchType string | ||
| EcsDeploymentStrategy string | ||
| EcsClusterArn string | ||
| CwagentConfigSsmParamName string | ||
| EcsServiceName string | ||
| } | ||
|
|
||
| func registerComputeType(dataString *MetaDataStrings) { | ||
| flag.StringVar(&(dataString.ComputeType), "computeType", "", "EC2/ECS/EKS") | ||
| } | ||
| func registerECSData(dataString *MetaDataStrings) { | ||
| flag.StringVar(&(dataString.EcsLaunchType), "ecsLaunchType", "", "EC2 or Fargate") | ||
SaxyPandaBear marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| flag.StringVar(&(dataString.EcsDeploymentStrategy), "ecsDeploymentStrategy", "", "Daemon/Replica/Sidecar") | ||
| flag.StringVar(&(dataString.EcsClusterArn), "clusterArn", "", "Used to restart ecs task to apply new agent config") | ||
| flag.StringVar(&(dataString.CwagentConfigSsmParamName), "cwagentConfigSsmParamName", "", "Used to set new cwa config") | ||
| flag.StringVar(&(dataString.EcsServiceName), "cwagentECSServiceName", "", "Used to restart ecs task to apply new agent config") | ||
| } | ||
|
|
||
| func fillComputeType(e *MetaData, data *MetaDataStrings) *MetaData { | ||
| computeType, ok := computetype.FromString(data.ComputeType) | ||
| if !ok { | ||
| log.Panic("Invalid compute type. Needs to be EC2/ECS/EKS. Compute Type is a required flag. :" + data.ComputeType) | ||
| } | ||
| e.ComputeType = computeType | ||
| return e | ||
| } | ||
|
|
||
| func fillECSData(e *MetaData, data *MetaDataStrings) *MetaData { | ||
SaxyPandaBear marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| if e.ComputeType != computetype.ECS { | ||
| return e | ||
| } | ||
|
|
||
| ecsLaunchType, ok := ecslaunchtype.FromString(data.EcsLaunchType) | ||
| if !ok { | ||
| log.Printf("Invalid launch type %s. This might be because it wasn't provided for non-ECS tests", data.ComputeType) | ||
| } else { | ||
| e.EcsLaunchType = ecsLaunchType | ||
| } | ||
|
|
||
| ecsDeploymentStrategy, ok := ecsdeploymenttype.FromString(data.EcsDeploymentStrategy) | ||
| if !ok { | ||
| log.Printf("Invalid deployment strategy %s. This might be because it wasn't provided for non-ECS tests", data.ComputeType) | ||
| } else { | ||
| e.EcsDeploymentStrategy = ecsDeploymentStrategy | ||
| } | ||
|
|
||
| e.EcsClusterArn = data.EcsClusterArn | ||
SaxyPandaBear marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| e.CwagentConfigSsmParamName = data.CwagentConfigSsmParamName | ||
| e.EcsServiceName = data.EcsServiceName | ||
| e.EcsClusterName = test.GetClusterName(data.EcsClusterArn) | ||
|
|
||
| return e | ||
| } | ||
|
|
||
| func RegisterEnvironmentMetaDataFlags(metaDataStrings *MetaDataStrings) *MetaDataStrings { | ||
| registerComputeType(metaDataStrings) | ||
| registerECSData(metaDataStrings) | ||
| return metaDataStrings | ||
| } | ||
|
|
||
| func GetEnvironmentMetaData(data *MetaDataStrings) *MetaData { | ||
| metaData := &(MetaData{}) | ||
| metaData = fillComputeType(metaData, data) | ||
| metaData = fillECSData(metaData, data) | ||
|
|
||
| return metaData | ||
| } | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| // Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. | ||
| // SPDX-License-Identifier: MIT | ||
|
|
||
| //go:build integration | ||
| // +build integration | ||
|
|
||
| package computetype | ||
|
|
||
| import "strings" | ||
|
|
||
| type ComputeType string | ||
|
|
||
| const ( | ||
| EC2 ComputeType = "EC2" | ||
| ECS ComputeType = "ECS" | ||
| EKS ComputeType = "EKS" | ||
| ) | ||
|
|
||
| var ( | ||
| computeTypes = map[string]ComputeType{ | ||
| "EC2": EC2, | ||
| "ECS": ECS, | ||
| "EKS": EKS, | ||
| } | ||
| ) | ||
|
|
||
| func FromString(str string) (ComputeType, bool) { | ||
| c, ok := computeTypes[strings.ToUpper(str)] | ||
| return c, ok | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| // Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. | ||
| // SPDX-License-Identifier: MIT | ||
|
|
||
| //go:build integration | ||
| // +build integration | ||
|
|
||
| package ecsdeploymenttype | ||
|
|
||
| import "strings" | ||
|
|
||
| type ECSDeploymentType string | ||
|
|
||
| const ( | ||
| DAEMON ECSDeploymentType = "DAEMON" | ||
| REPLICA ECSDeploymentType = "REPLICA" | ||
| SIDECAR ECSDeploymentType = "SIDECAR" | ||
| ) | ||
|
|
||
| var ( | ||
| ecsDeploymentTypes = map[string]ECSDeploymentType{ | ||
| "DAEMON": DAEMON, | ||
| "REPLICA": REPLICA, | ||
| "SIDECAR": SIDECAR, | ||
| } | ||
| ) | ||
|
|
||
| func FromString(str string) (ECSDeploymentType, bool) { | ||
| c, ok := ecsDeploymentTypes[strings.ToUpper(str)] | ||
| return c, ok | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| // Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. | ||
| // SPDX-License-Identifier: MIT | ||
|
|
||
| //go:build integration | ||
| // +build integration | ||
|
|
||
| package ecslaunchtype | ||
|
|
||
| import "strings" | ||
|
|
||
| type ECSLaunchType string | ||
|
|
||
| const ( | ||
| EC2 ECSLaunchType = "EC2" | ||
| FARGATE ECSLaunchType = "FARGATE" | ||
| ) | ||
|
|
||
| var ( | ||
| ecsLaunchTypes = map[string]ECSLaunchType{ | ||
| "EC2": EC2, | ||
| "FARGATE": FARGATE, | ||
| } | ||
| ) | ||
|
|
||
| func FromString(str string) (ECSLaunchType, bool) { | ||
| c, ok := ecsLaunchTypes[strings.ToUpper(str)] | ||
| return c, ok | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| [ | ||
| { | ||
| "instanceType":"t3a.xlarge", | ||
| "ami": "cloudwatch-agent-integration-test-al2*" | ||
| } | ||
| ] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,5 @@ | ||
| [ | ||
| { | ||
|
|
||
| } | ||
| ] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -23,7 +23,6 @@ | |
| }, | ||
| "cpu": 1, | ||
| "mountPoints": [], | ||
| "memory": 2048, | ||
| "volumesFrom": [] | ||
| "memory": 2048 | ||
| } | ||
| ] | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| [] |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that this comment got missed somewhere. The file name shouldn't be capitalized.