Table of Contents generated with DocToc
- Three Tier Infrastructure
- Architecture
- VPC and Network setup
- Application Server Setup
- IAM Role - Identity and Access Management
- EC2 - Application Server 1 Setup
- ssh testing
- initial server setup / patch
- verify http apache server
- amend landing page - optional
- S3 configuration
- IAM Role adjustment
- Adjust ec2 app server via ssh
- update user data script
- Lunch Dev-1 EC2 Application Server 1 from scratch
- Check Services and daemons
- EC2 - Application Server 2 Setup
- Load Balancer Configuration
- Auto Scaling Groups Introduction
- Create and Configure MySql DB instance
- DNS Management
- Terraform
Piotr Szczepanski
References:
Sai Kiran Rathan
https://www.udemy.com/setup-aws-infrastructure-for-production-learn-terraform/learn/v4/overview
- Region - geographic
- VPC - private IP space
- two availability zones - resilience
- 6 subnets - three per each availability zone
- Private subnet
- Public Subnet
- DMZ subnet
- 2 Elastic load balancers - DMZ Subnet
- 2 Application servers - public subnet
- 2 database servers - private subnets
Client can only access ELB DMZ subnet.
APP servers communicate only with ELB. Database servers communicate with App servers only
Security groups act as a firewall - restricting access.
VLSM / CIDR Subnet calculator
- www.vlsm-calc.net
- Vpc->create Vpc [specify name and CIDR range with subnet mask, no dedicated Tenancy] 192.160.0.0/19
- Set Up 6 Subnets to override the 3 default subnets - size 256 hosts each
Basic route table has been configured automatically once the initial VPC setup has been done.
- Configure new Gateway and attach to VPC
- Set Up Main Route Table - RT: Go to routes/ routes tab/edit/ and add another route: 0.0.0.0/0 - all web; select target IGW: Dev-1-IGW; save
- Set up new RT - Subnet specific - DMZ: Add internet address and target Dev-1-IGW. save and switch to subnet associations tab and add two DMZ subnets
- Set up new RT Subnet specific - Public: Add relevant subnets
- Set up new RT Subnet specific - Private: Add relevant subnets
The purpose of NAT - all private instances/ subnets cannot be accessed via Internet BUT can access internet without Public IP - NAT. 9. NAT Gateways > Create NAT GW - two (HIGH availability HA) One NAT GW per availability zone select public subnet 1 and allocate new EIP - elastic IP - NAT GW 1 select public subnet 2 and allocate new EIP - elastic IP - NAT GW 2 10. Set up two new NAT private route tables (HA)
- 0.0.0.0/0 - all web; select target NAT GW 1; save
- switch to subnet associations tab and Private subnet 1
- 0.0.0.0/0 - all web; select target NAT GW 2; save
- switch to subnet associations tab and Private subnet 2
Public IPs are required for application servers to be available over the internet.
- VPC/Subnets section
- select Public Subnet 1 > modify auto assign IP setting (subnet actions) => tick enable
- select Public Subnet 2 > modify auto assign IP setting (subnet actions) => tick enable
Allows entities to call AWS services on one's behalf. Purpose: to allow the instance to perform specific actions and stronger security as AWS will handle permissions behind the scenes.
IAM => Roles - pick service that will be using the role: EC2 Always go for the least privileged access method.
- Create a role with no permissions as these will be established and added later. Name it Dev-1EC2-Role
- Launch new instance - EC2-Dev-1
- Amazon Linux
- t2 micro
- change network to VPC-Dev-1
- select Public Subnet 1 (app server available in public domain)
- ensure auto assign IP setting is enabled
- Select Role Dev-1-EC2-Role
- ensure shutdown behaviour is set to Stop
- Tenancy - shared
- keep network interface settings default
- go to next config steps - storage
- select default 8G general purpose storage
- ensure Delete on termination is ticked (to avoid extra costs when not using it)
- go to add tags: (tags are needed also in terms of accountancy/billing - to recognize what instance added to costs)
- Name: DEV-1 App Server
- Environment: Development
- configure new security group Dev-1 Public SG Leave SSH port open to single - own IP - for testing
- Add Rule - HTTP 0.0.0.0/0
- create and save new Key Pair - Dev-1-KeyPair.pem when testing ssh - ensure .pem file permissions are set to min 400
- ssh -i /KeyPair/file/location.pem ec2-user@x.x.x.x
sudo yum update -y [option -y => promptless] sudo yum install -y httpd php sudo service httpd start
browser or curl to server public IP
cd /var/www/html
echo "Piotr Testing Page" > index.html- configure standard S3 bucket, add new folder => builds and upload sample page/application
- Create policy IAM>Roles>Dev-1-EC2-role> Add inline policy>JSON Tab For assistance: https://docs.aws.amazon.com/AmazonS3/latest/dev/using-with-s3-actions.html
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "dev1s3accesspolicy",
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::piotr.szczepanski/Dev-1/*",
"arn:aws:s3:::piotr.szczepanski/Dev-1"
]
}
]
}- Attach above policy to Dev-1-EC2-role
aws s3 cp help
aws s3 cp s3://piotr.szczepanski/Dev-1/builds/Dev-1-landingPage.zip Dev-1-landingPage.zip
rm index.html
unzip Dev-1-landingPage.zip
rm -fR Dev-1-landingPage.zip-y => promptless command flag
#!/bin/bash
sudo su
yum update -y
yum install -y httpd php
cd /var/www/html
aws s3 cp s3://piotr.szczepanski/Dev-1/builds/Dev-1-landingPage.zip Dev-1-landingPage.zip
unzip Dev-1-landingPage.zip
mv Dev-1-landingPage/* .
rm -rf Dev-1-landingPage.zip Dev-1-landingPage
sudo service httpd start- Add steps as before plus add a script in advanced setting - user data
service --status-all
- To create App Server 2 use right-click > launch more like this Adjust name and subnet => change to 2nd availability
https://docs.aws.amazon.com/elasticloadbalancing/latest/application/introduction.html
https://docs.aws.amazon.com/elasticloadbalancing/latest/application/introduction.html
- Load balancer type => select Application LB Network LP option = > select only for high-performance networks, static IPs etc.
Set name, internet facing, protocol http,select VPC,select avail. zones and related subnets: eu-west-2a DMZ Subnet 1 eu-west-2b DMZ Subnet 2
Add tags: name, and environment 24. Configure Security Groups => name and create new security group => leave default TCP 80 ; to be fine-tuned later 25 Configure Routing and Health checks- name and create new target group leave
- Register Targets Add two app servers: Dev-1 EC2 Application Server 1 Dev-1 EC2 Application Server 2 Review and create
All stats (latency, requests, health and more) can be obtained Load Balancer or target groups. To access app servers via the App Load balancer, copy-paste into browser DNS A record (Load Balancer/ description tab)
Ensure app servers can access internet - unidirectional way - outbound only - needed when updating/ patching systems - access to external repositories, packages etc.
Ensure client interaction is restricted only to Load balancers level - DMZ zones in following way:
-
ensure only load balancers are facing incoming web traffic
-
ensure public subnet(app servers) can receive web traffic (port 80) from App Load balancers ONLY.
- Amend Dev-1 Public SG (App servers SG) so it gets traffic from Dev-1-External-Application-Load-Balancer-SG by pasting ALB SG ID into source field.
-
- ensure traffic from load balances - outbound - goes to app servers ONLY (not further to DB) - by pasting Dev-1 Public SG (Apps Servers SG) ID into App Load Balancers SG - outbound - destination field
- ensure there is no wide open ssh access to app servers
https://docs.aws.amazon.com/autoscaling/ec2/userguide/what-is-amazon-ec2-auto-scaling.html
- EC2>Launch Auto Scaling
Add default storage, tick delete on termination Select existing public security group for the EC2 app servers - Dev-1 Public SG Review, select the .pem file and proceed to
Save and update as follows:
![https://github.com/szczepanski/cloud-aws/blob/master/dev-1-architecture/update%20asg.png]](https://github.com/szczepanski/cloud-aws/blob/master/dev-1-architecture/update%20asg.png%5D){kind=link}
-
Go to SNS and create 4 topics: Dev-1 Scale-up alarm Dev-1 Scale down alarm Dev-1-Service-Anomaly Dev-1-AutoScalingActivityAlarm
-
EC2>Auto Scaling Target Groups Scaling Policies> Add policy> create simple Scale policy
Create Scale UP policy
and create new alarm - High CPU -to trigger/add one new instance)
-
Create Scale down policy and alarm - low CPU - for CPU utilization <=20% - to remove one oldest instance, save
-
switch to notifications tab and create a new notification
create new notification linked to Dev-1-AutoScalingActivityAlarm This will alarm whenever there is new launch, termination, fail to launch or terminate.
- Create New Cloudwatch Alarms
Load Balancing/Target Groups/ Monitoring Tab / Create New Alarm (Cloudwatch)
Create: Dev-1-Application-High-Average-Latency-Alarm
Create: Dev-1-Application-High-Average-Latency-Alarm-Recovery-Notice Trigger it when latency is = < 3s
- Add auto scaling actions to previously created Alarms.
Click on the specific alarm and open directly in CloudWatch/modify it:
- Add Auto Scaling Action - Scale Up -(add one instance) - while the alarm is on - state: Alarm
- Add Auto Scaling Action - Scale Down -(remove one instance) - while alarm recovery notice is on - State: Alarm
- Configuring SNS Topic Subscriptions SNS/Topics/Subscribe to topic
- Set Protocol to email
- set endpoint to mail address
Open the subscribed email and confirm the subscription.
To test go to Auto Scaling/Autoscaling Groups/Details Tab/ Edit
- temporarily change desired from 2 to 3; save
- Create a new security group
Ensure when opening ports - here 3306 (MySQL) that any traffic via this port is coming from (source) public subnet - in this architecture - app servers subnet (not from DMZ - internet facing zone).
Whenever additional ports are needed - set the source to public (app servers / non-internet facing) security group.
- Create new subnet group:
- go to RDS/Subnet groups
Add two subnets -for 2 availability zones:
go to RDS/Instances/ Launch pick DB type - here Amazon Aurora (MySql) and configure all required settings.
SSL certificate needs to be assigned to load balancer by creating HTTPS listeners that forward the traffic to specific target groups.
Certificate Manager > request a certificate
- add domain name
- select validation method => email (AWS validation email message to be sent to the domain owner registered address).
Once approved by the owner certificate will show as issued.
Load Balancing/ Load Balancers/ Listeners/ add https port 443
- Go to Route 53/Create Hosted Zone
provide domain name and type: hosted zone and create.
In order to have Route 53 managing the DNS, original (GoDaddy, 1&1, etc) nameservers entries need to be changed to the nameservers entries provided by AWS.
Once this is done, entire DNS can be then managed from within Route 53.
- Create A record within Route 53 pointing to the Dev-1 Application Load Balancer (Simple route policy).
- Wait some time to test/resolve new A record (pointing to app LB)
Open source by hashicorp
Use cases: - infrastructure version control and back up if prod config breaks; for multiple environments - minimization of config drift - consistency.
3 Basic components:
- config file.tf - written in hashicorp configuration language - HCL
- cli => terraform plan
- cli => terraform apply
Easy variable declaration Good documentation. Referencing files => such as user data.
Create a new user in IAM with programmatic access: dev-1-terraform-user
add it to the group: dev-1-admin-programmatic-access
provider "aws" { region = "us-west-2" shared_credentials_file = "/Users/tf_user/.aws/creds" profile = "customprofile" }