Skip to content

sverze/aws-terraform-emr-network

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

AWS / Terraform EMR Network with VPC Service Endpoint

VPC 1 contains a bastion host that can be used for testing access to the associated VPC endpoint interface. This interface is connected to an ENI that in turn routes to the service endpoint in VPC 2

Bastion VPC

Set-Up

You will need the following tools and accounts to make it happen

AWS Account

You will need an AWS account, if you don't have one then just Sign-Up You will need a development user with admin access and security access keys in order to use the AWS CLI

Install the AWS CLI and set-up a profile using your access keys

SSH Key

You will need to set-up your desired AWS region with a secure key pair. This project is using London (eu-west-2) as the region I suggest you keep this the same as there are AMI's referenced that are region sensitive.

EC2 Key Pairs has instructions on how to set-up your key pair.

Once you have set-up key pair you will have access to the PEM file that needs to be stored safely. Terraform assumes that the PEM is available in your local key chain, you can add it it to your key-chain by running the following command

ssh-add -K your-key.pem

Terraform

Install terraform command line tool. Depends on your OS, Install Terraform has some descent instructions.

If you are using OSX I suggest you use Homebrew to install the terraform package.

Building / Running

Plan

I suggest that you run a plan to determine that access to your AWS account is as expected. Run the following command in the root of the terraform source src/main/terraform Note that it is recommended that you override the default values for aws_profile and the bastion_network_cidr

terraform plan -var "aws_key_name=your-key-name" -var "aws_region=us-east-1"

The output of this command will detail all the resources that will be created once applied. Note that it is recommended that you override the default values for aws_region (eu-west-1), aws_profile (default) and the bastion_network_cidr (0.0.0.0/0) e.g.

terraform plan -var "aws_key_name=you-key" -var "aws_profile=bastion-lab" -var "bastion_network_cidr=199.250.200.0/24"

Apply

Once you are happy with the plan apply and decided on what variables to override you can apply your plan e.g.

terraform apply -var "aws_key_name=you-key" -var "aws_profile=bastion-lab" -var "bastion_network_cidr=199.250.200.0/24"

Testing

The best way to test the application is to SSH on to the test host and attempt accessing the internet.

  1. Copy your EC2 key to the bastion host - scp -i <YOUR_KEY>.pem <YOUR_KEY>.pem ec2-user@<BASTION_HOST_IP>:.ssh/
  2. SSH to the bastion host - ssh ~/.ssh/<YOUR_KEY>.pem ec2-user@<BASTION_HOST_IP>
  3. Connect to the VPC endpoint interface. The DNS entry for the endpoint is an output value from the bastion module, otherwise you can locate the value by searching for the only interface endpoint associated with VPC 1, the following is an example - curl http://vpce-02911c401b55c789b-abum0ynl.vpce-svc-0838beea528393c7f.ap-southeast-2.vpce.amazonaws.com

Destroy

Clean up your environment by detroying, this will remove all traces:

terraform destroy -var 'aws_key_name=your-key.pem'

Troubleshooting

TODO - add FAQ for potential issues

About

Isolated EMR network with working example

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published