Skip to content

Commit

Permalink
Start v5.0.4 - enable new 3-zone ZoneAwareness for Elasticsearch (#57)
Browse files Browse the repository at this point in the history
Signed-off-by: Irving Popovetsky <irving@chef.io>
  • Loading branch information
Irving Popovetsky authored Sep 9, 2019
1 parent a0c503a commit 2f11759
Show file tree
Hide file tree
Showing 9 changed files with 114 additions and 71 deletions.
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,8 @@ You can launch this stack with the push of a button:

| Edition | Version | Est Monthly cost | Launch button |
| ------- | ------- | ---------------- | ------------- |
| High Performance | 5.0.3 | [$1500*](https://calculator.s3.amazonaws.com/index.html#r=IAD&s=EC2&key=calc-E9251374-D80C-45B7-BED7-C9C9778B1D6C) | <p><a href="https://console.aws.amazon.com/cloudformation/home#/stacks/new?templateURL=https:%2F%2Fs3.amazonaws.com%2Faws-native-chef-server%2F5.0.3%2Fmain.yaml&amp;stackName=my-chef-stack" target="_blank"><img src="https://s3.amazonaws.com/cloudformation-examples/cloudformation-launch-stack.png" alt="Launch Stack" /></a></p> |
| Lowest Cost | 5.0.3 | [$320*](https://calculator.s3.amazonaws.com/index.html#r=IAD&s=EC2&key=files/calc-165638db8e331664846c0d6654d743377bd3eac6&v=ver20190604sQ) | <p><a href="https://console.aws.amazon.com/cloudformation/home#/stacks/new?templateURL=https:%2F%2Fs3.amazonaws.com%2Faws-native-chef-server%2F5.0.3%2Fmarketplace.yaml&amp;stackName=my-chef-stack" target="_blank"><img src="https://s3.amazonaws.com/cloudformation-examples/cloudformation-launch-stack.png" alt="Launch Stack" /></a></p> |
| High Performance | 5.0.4 | [$1500*](https://calculator.s3.amazonaws.com/index.html#r=IAD&s=EC2&key=calc-E9251374-D80C-45B7-BED7-C9C9778B1D6C) | <p><a href="https://console.aws.amazon.com/cloudformation/home#/stacks/new?templateURL=https:%2F%2Fs3.amazonaws.com%2Faws-native-chef-server%2F5.0.4%2Fmain.yaml&amp;stackName=my-chef-stack" target="_blank"><img src="https://s3.amazonaws.com/cloudformation-examples/cloudformation-launch-stack.png" alt="Launch Stack" /></a></p> |
| Lowest Cost | 5.0.4 | [$320*](https://calculator.s3.amazonaws.com/index.html#r=IAD&s=EC2&key=files/calc-165638db8e331664846c0d6654d743377bd3eac6&v=ver20190604sQ) | <p><a href="https://console.aws.amazon.com/cloudformation/home#/stacks/new?templateURL=https:%2F%2Fs3.amazonaws.com%2Faws-native-chef-server%2F5.0.4%2Fmarketplace.yaml&amp;stackName=my-chef-stack" target="_blank"><img src="https://s3.amazonaws.com/cloudformation-examples/cloudformation-launch-stack.png" alt="Launch Stack" /></a></p> |

_*NOTE: Estimated costs are just that, your mileage will vary. Use reserved instances to greatly reduce costs for long-running clusters._

Expand All @@ -48,7 +48,7 @@ However, the most repeatable and least error-prone way to launch this stack is t
```bash
MYBUCKET=aws-native-chef-server
MYID=mycompany
VERSION=5.0.3
VERSION=5.0.4
EDITION=main # main or marketplace
# Configure the automate_stack_parameters.json and then launch the cloudformation stack:
# If you're using your own bucket, uncomment the next line:
Expand All @@ -69,7 +69,7 @@ If you've made changes to the template content or parameters and you wish to upd
```bash
MYBUCKET=aws-native-chef-server
MYID=mycompany
VERSION=5.0.3
VERSION=5.0.4
EDITION=main # main or marketplace
# If you're using your own bucket, uncomment the next line:
# aws s3 sync . s3://$MYBUCKET/$VERSION/ --exclude "*" --include "*.yaml" --include "files/*" && \
Expand Down
2 changes: 1 addition & 1 deletion automate.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
AWSTemplateFormatVersion: '2010-09-09'
Description: AWS Native Automate 2 (auto-recovering standalone) v5.0.3
Description: AWS Native Automate 2 (auto-recovering standalone) v5.0.4

Parameters:
# Required Parameters
Expand Down
94 changes: 58 additions & 36 deletions chef_elasticsearch.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
AWSTemplateFormatVersion: '2010-09-09'
Description: Chef Server Elasticsearch 5.0.3
Description: Chef Server Elasticsearch 5.0.4

Parameters:
# Required Parameters
Expand All @@ -21,20 +21,35 @@ Parameters:
Description: The Instance type to use for ElasticSearch instances (Note, must have ephemeral storage, the instance type affects the total amount of elasticsearch storage. i3 strongly recommended)
Type: String
Default: 'i3.large.elasticsearch'
AllowedValues: [
'i3.large.elasticsearch', 'i3.xlarge.elasticsearch', 'i3.2xlarge.elasticsearch', 'i3.4xlarge.elasticsearch', 'i3.8xlarge.elasticsearch', 'i3.16xlarge.elasticsearch',
'i2.xlarge.elasticsearch', 'i2.2xlarge.elasticsearch',
'm3.medium.elasticsearch', 'm3.large.elasticsearch', 'm3.xlarge.elasticsearch', 'm3.medium.elasticsearch',
'r3.large.elasticsearch', 'r3.xlarge.elasticsearch', 'r3.2xlarge.elasticsearch', 'r3.4xlarge.elasticsearch', 'r3.8xlarge.elasticsearch' ]
AllowedValues:
[
'i3.large.elasticsearch',
'i3.xlarge.elasticsearch',
'i3.2xlarge.elasticsearch',
'i3.4xlarge.elasticsearch',
'i3.8xlarge.elasticsearch',
'i3.16xlarge.elasticsearch',
'i2.xlarge.elasticsearch',
'i2.2xlarge.elasticsearch',
'm3.medium.elasticsearch',
'm3.large.elasticsearch',
'm3.xlarge.elasticsearch',
'm3.medium.elasticsearch',
'r3.large.elasticsearch',
'r3.xlarge.elasticsearch',
'r3.2xlarge.elasticsearch',
'r3.4xlarge.elasticsearch',
'r3.8xlarge.elasticsearch',
]
ElasticSearchVersion:
Description: Version of ElasticSearch to use
Type: String
Default: '5.6'
AllowedValues:
- '2.3'
- '5.3'
- '5.5'
- '5.6'
- '2.3'
- '5.3'
- '5.5'
- '5.6'
ElasticSearchShardCount:
Description: Number of ElasticSearch hosts to provision at launch (3 recommended, 2 provides HA)
Default: 3
Expand All @@ -54,16 +69,15 @@ Parameters:
Type: String

Conditions:
2ZoneES:
!Equals [!Ref ElasticSearchShardCount, 2]
2ZoneES: !Equals [!Ref ElasticSearchShardCount, 2]

Resources:
# ElasticSearch
#########################################################################################
# ElasticSearch
#########################################################################################
ESSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: "Elasticsearch Frontend Access"
GroupDescription: 'Elasticsearch Frontend Access'
VpcId: !Ref VPC
SecurityGroupIngress:
- IpProtocol: tcp
Expand All @@ -77,40 +91,49 @@ Resources:
ElasticsearchVersion: !Ref ElasticSearchVersion
ElasticsearchClusterConfig:
InstanceCount: !Sub ${ElasticSearchShardCount}
ZoneAwarenessEnabled: !If [2ZoneES, true, false]
ZoneAwarenessEnabled: true
ZoneAwarenessConfig:
AvailabilityZoneCount: !If [2ZoneES, 2, 3]
InstanceType: !Ref ElasticSearchInstanceType
DedicatedMasterEnabled: false
SnapshotOptions:
AutomatedSnapshotStartHour: 0
AccessPolicies:
Version: "2012-10-17"
Version: '2012-10-17'
Statement:
- Effect: "Allow"
- Effect: 'Allow'
Principal:
AWS: !Ref ChefRole
Action: "es:*"
Resource: !Sub "arn:aws:es:${AWS::Region}:${AWS::AccountId}:*"
Action: 'es:*'
Resource: !Sub 'arn:aws:es:${AWS::Region}:${AWS::AccountId}:*'
VPCOptions:
SubnetIds: !If [
2ZoneES,
[ !Select [ 0, !Ref ChefServerSubnets ], !Select [ 1, !Ref ChefServerSubnets ]],
[ !Select [ 0, !Ref ChefServerSubnets ] ]
]
SubnetIds:
!If [
2ZoneES,
[
!Select [0, !Ref ChefServerSubnets],
!Select [1, !Ref ChefServerSubnets],
],
[
!Select [0, !Ref ChefServerSubnets],
!Select [1, !Ref ChefServerSubnets],
!Select [2, !Ref ChefServerSubnets],
],
]
SecurityGroupIds:
- !Ref ESSecurityGroup
AdvancedOptions:
rest.action.multi.allow_explicit_index: "true"
rest.action.multi.allow_explicit_index: 'true'
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-ES
- Key: X-Dept
Value: !Ref ContactDept
- Key: X-Contact
Value: !Ref ContactEmail

- Key: Name
Value: !Sub ${AWS::StackName}-ES
- Key: X-Dept
Value: !Ref ContactDept
- Key: X-Contact
Value: !Ref ContactEmail

# Monitoring
#########################################################################################
# Monitoring
#########################################################################################
ESClusterRed:
Type: AWS::CloudWatch::Alarm
Properties:
Expand All @@ -130,7 +153,6 @@ Resources:
- Name: ClientId
Value: !Ref AWS::AccountId


ESClusterYellow:
Type: AWS::CloudWatch::Alarm
Properties:
Expand Down
65 changes: 43 additions & 22 deletions chef_rds.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
AWSTemplateFormatVersion: '2010-09-09'
Description: Chef Server PostgreSQL v5.0.3
Description: Chef Server PostgreSQL v5.0.4

Parameters:
# Required Parameters
Expand All @@ -25,10 +25,32 @@ Parameters:
Description: EC2 Instance type for RDS DBs (M5 class recommended for performance)
Default: 'db.m5.large'
Type: String
AllowedValues: [db.t3.small, db.t3.medium, db.t3.large, db.t3.xlarge, db.t3.2xlarge,
db.m4.large, db.m4.xlarge, db.m4.2xlarge, db.m4.4xlarge, db.m4.10xlarge, db.m4.16xlarge,
db.r4.large, db.r4.xlarge, db.r4.2xlarge, db.r4.4xlarge, db.r4.8xlarge, db.r4.16xlarge,
db.m5.large, db.m5.xlarge, db.m5.2xlarge, db.m5.4xlarge, db.m5.12xlarge, db.m5.24xlarge]
AllowedValues:
[
db.t3.small,
db.t3.medium,
db.t3.large,
db.t3.xlarge,
db.t3.2xlarge,
db.m4.large,
db.m4.xlarge,
db.m4.2xlarge,
db.m4.4xlarge,
db.m4.10xlarge,
db.m4.16xlarge,
db.r4.large,
db.r4.xlarge,
db.r4.2xlarge,
db.r4.4xlarge,
db.r4.8xlarge,
db.r4.16xlarge,
db.m5.large,
db.m5.xlarge,
db.m5.2xlarge,
db.m5.4xlarge,
db.m5.12xlarge,
db.m5.24xlarge,
]
DBStorageType:
Description: Select gp2 or io1 DB storage. io1 is much higher performing, must select minimum 100GB storage and an IOPS value
Default: 'gp2'
Expand Down Expand Up @@ -62,14 +84,13 @@ Parameters:
Type: String

Conditions:
UsePiopsStorage:
!Equals [ !Ref DBStorageType, 'io1' ]
UsePiopsStorage: !Equals [!Ref DBStorageType, 'io1']

Resources:
DBSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: "RDS Frontend Access"
GroupDescription: 'RDS Frontend Access'
VpcId: !Ref VPC
SecurityGroupIngress:
- IpProtocol: tcp
Expand All @@ -85,7 +106,7 @@ Resources:

DBPostgres:
Type: AWS::RDS::DBInstance
DeletionPolicy: "Snapshot"
DeletionPolicy: 'Snapshot'
Properties:
DBName: chef
AllocatedStorage: !Ref DBAllocatedStorage
Expand Down Expand Up @@ -116,26 +137,26 @@ Resources:
Type: AWS::RDS::EventSubscription
Properties:
EventCategories:
- "availability"
- 'availability'
# - "backup"
- "configuration change"
- "creation"
- "deletion"
- "failover"
- "failure"
- "low storage"
- "maintenance"
- "notification"
- 'configuration change'
- 'creation'
- 'deletion'
- 'failover'
- 'failure'
- 'low storage'
- 'maintenance'
- 'notification'
# - "read replica"
- "recovery"
- "restoration"
- 'recovery'
- 'restoration'
SnsTopicArn: !Ref AlertNotificationTopic
SourceIds:
- !Ref DBPostgres
SourceType: db-instance

# Monitoring
#########################################################################################
# Monitoring
#########################################################################################
RDSWriteLatency:
Type: AWS::CloudWatch::Alarm
Properties:
Expand Down
4 changes: 2 additions & 2 deletions chef_server_ha.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
AWSTemplateFormatVersion: '2010-09-09'
Description: AWS Native Chef Server (auto-scaling HA) v5.0.3
Description: AWS Native Chef Server (auto-scaling HA) v5.0.4

Parameters:
# Required Parameters
Expand Down Expand Up @@ -185,7 +185,7 @@ Parameters:
Type: String
TemplateVersion:
Type: String
Default: "5.0.3"
Default: "5.0.4"

Metadata:
cfn-lint:
Expand Down
2 changes: 1 addition & 1 deletion chef_server_ops_dashboard.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
AWSTemplateFormatVersion: '2010-09-09'
Description: AWS Native OpsDashboard for Chef Server v5.0.3
Description: AWS Native OpsDashboard for Chef Server v5.0.4

Parameters:
# Required Parameters
Expand Down
4 changes: 2 additions & 2 deletions main.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
AWSTemplateFormatVersion: '2010-09-09'
Description: AWS Native Chef Stack v5.0.3
Description: AWS Native Chef Stack v5.0.4

Parameters:
# Required Parameters
Expand Down Expand Up @@ -194,7 +194,7 @@ Parameters:
TemplateVersion:
Description: The version of this template to use (do not change this unless directed by support)
Type: String
Default: "5.0.3"
Default: "5.0.4"
Route53HostedZone:
Type: String
Default: ''
Expand Down
4 changes: 2 additions & 2 deletions marketplace.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
AWSTemplateFormatVersion: '2010-09-09'
Description: AWS Native Chef Stack for Marketplace v5.0.3
Description: AWS Native Chef Stack for Marketplace v5.0.4

Parameters:
# Required Parameters
Expand Down Expand Up @@ -186,7 +186,7 @@ Parameters:
TemplateVersion:
Description: The version of this template to use (do not change this unless directed by support)
Type: String
Default: "5.0.3"
Default: "5.0.4"
LogsRetentionInDays:
Description: Specifies the number of days you want to retain cloudwatch log events.
Type: Number
Expand Down
2 changes: 1 addition & 1 deletion supermarket.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
AWSTemplateFormatVersion: '2010-09-09'
Description: AWS Native Supermarket (auto-recovering standalone) v5.0.3
Description: AWS Native Supermarket (auto-recovering standalone) v5.0.4

Parameters:
# Required Parameters
Expand Down

0 comments on commit 2f11759

Please sign in to comment.