AWS Resources

Collection of Amazon Web Services Cloud resources and Utilities

Utilities I could use

• script that ensured an S3 bucket is created with the given name and prefix - to use in SAM based scripts that require a S3 bucket

TODO

• add SSL, hostname, domainname
• pass in S3 and stack name config better

Feb 27, 2019; repurpose this repository; it will contain resources that can launch various AWS stacks and implementations; it will be 100% self-contained with the caveat that Docker images have been built and deployed to Docker Hub (using the Docker resources in this git) to push the other implementations. The Dockerfile is a helper to build the stack as long as there is an AWS credentials available to the container.

Contents

• Build Image
• Cloud Formation
• VPC
• References

Build Image

This image was designed to be able to bootstrap this GIT into an AWS runtime

Usage

Assume this is built as 'mcliff/aws'

docker run --rm mcliff/aws:1.0.0

To run locally without anything docker build -t local . docker run local

Steps to Build

This was built on an EC2 instance set up that has docker installed.

• Build the image (and tag) docker build -t mcliff1/aws .
• docker build -t mcliff/aws .

These steps created a latest tag; repeated with :0.0.1 appended to the end for a v0.0.1 tag as well, that is what other build scripts should reference

Cloud Formation

These templates are based natively in Cloud Formation, each on should have a clearly defined Parameters section and Outputs section.

• ec2-slsworkstation.json - creates EC2 REACT/SLS Dev workstation in the selected VPC and Subnet
• vpc.json - creates a VPC in the region imported into with public and private subnets in 3 availability zones
• bastion.json - adds a bastion host (uses vpc.json)
• rds-snapshot.json - restores a RDS database from snapshot (uses vpc.json)
• rds-postgres.json - creates a new RDS PostgreSQL database (uses vpc.json)

SLS Workstation

ec2-slsworkstation.json
The ec2-slsworkstation.json Cloud Formation template will build out a Serverless workstation with NodeJS, NPM, SLS, and REACT frameworks install. Create Details Parameters myKeyPair myDomain (optional) Outputs Public S3 Url https://s3.amazonaws.com/mcliff/cliffconsulting.com/

Kaggle Workstation

ec2-kaggle.json
The ec2-kaggle.json Cloud Formation template will build out a workstation with Docker and the latest Kaggle image pulled. Create Details Parameters myKeyPair myDomain (optional) Outputs Public S3 Url https://s3.amazonaws.com/mcliff/cliffconsulting.com/

VPC

vpc.json

The *vpc.json* Cloud Formation template will build out a VPC with private and public subnets. This provides outputs for subnet and security information that other scripts may leverage. This provides the basis for other templates. Create Details Parameters VPC Stack Name - referenced by other stacks Project - used to Tag associated resources VPC Subnet IPv4 CIDR range Public Subnet 1 IPv4 CIDR range Public Subnet 2 IPv4 CIDR range Public Subnet 3 IPv4 CIDR range Private Subnet 1 IPv4 CIDR range Private Subnet 2 IPv4 CIDR range Private Subnet 3 IPv4 CIDR range Outputs {StackName}-VpcId {StackName}-PublicSubnet1Id {StackName}-PublicSubnet2Id {StackName}-PublicSubnet3Id {StackName}-PrivateSubnet1Id {StackName}-PrivateSubnet2Id {StackName}-PrivateSubnet3Id {StackName}-TopicArn SNS topic that can be used for events in this VPC TODO set up [VPC end points](https://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-endpoints.html); also what kind of termination protection/monitoring framework

Bastion

bastion.json

The bastion.json Cloud Formation template will build create a bastion host on a VPC from the template. Create Details Creates EC2 instance and security group, the new security group allows connectivity from outside the VPC, the instance also gets the security group that allows internal access to all resources (private and public) in the specified VPC. Requires [VPC.json](#vpc) Parameters Stack Name - referenced by other stacks Project - used to Tag associated resources VPC StackName - from VPC stack Network to allow SSH access from Key Pair to use for SSH access Update Route53 -TODO not implemented Instance Type Outputs TODO setup EIP (it's free as long as VM is running) rename export name of internal access security group add the DNS update

RDS Snapshot

rds-snapshot.json

The *rds-snapshot.json* Cloud Formation template will build create a RDS instance in the VPC, from a snapshot. Create Details Can restore any RDS DB instance Parameters Stack Name - referenced by other stacks Project - used to Tag associated resources VPC StackName - from VPC stack ARN of the snapshot Update Route53 -TODO not implemented Instance Type AutoMinorVersionUpgrade MultiAvailabilityZone TcpPort Outputs {StackName}-DbId {StackName}-DbHost {StackName}-DbPort

RDS Postgres

rds-postgres.json

The *rds-postgres.json* Cloud Formation template will build create a PostgreSQL instance in the VPC. Create Details Builds a new RDS PostgreSQL database in the Private Subnets of the VPC. Parameters Stack Name - referenced by other stacks Project - used to Tag associated resources VPC StackName - from VPC stack Update Route53 -TODO not implemented Instance Type AllocatedStorage AutoMinorVersionUpgrade MultiAvailabilityZone DbName DbUser DbPassword MultiAvailabilityZone DbVersion TcpPort Outputs

Serverless

back to top These use the Serverless framework, which is designed to provide abstraction to the underlying Cloud provider (AWS in our case).

Frameworks with API Gateway/Lambda

• simpleservice - builds a generic Lambda function and API gateway for POST to /; logs and echos both the event and context objects
• urlshortener - service to take a link, create a short-link and store in DynamoDB
• linklist - provides a framework for serverless list of links with comments uses Cognito, CloudFront, DynamoDB
• staticweb - provides framework to host S3 content behind a provided domainname
• slsweb - provides framework for serverless web (with Route 53, CloudFront, ...)
• restitemapi - a REST API for a generic item

contains some serverless code

• ref: building a sls dev environment - setting up credential

Simple Service
Creates a Lambda function that is generic. Create Details Parameters Outputs Public S3 Url https://s3.amazonaws.com/mcliff/cliffconsulting.com/

Item REST API

Creates RESTful endpoint for **item** object, which is a generic *JSON* object. The key is **ItemKey**, this will automatically be generated if not present. ref API *GET /item[/{ItemKey}]* - retrieve *POST /item* - create *PUT /item/{ItemKey}* - update *DELETE /item/{ItemKey}* - delete Create Details Creates REST API, appropriate lambda function, and DynamoDB backend. Parameters Domain Name (optional) Outputs REST End-Point DynamoDB Table Public S3 Url https://s3.amazonaws.com/mcliff/cliffconsulting.com/ TODO Add identity stuff (URL signing) Make the term *item* generic

SLSWEB

This project implements a Stack that will deploy a fully functional serverless website. Builds evertyhing in _us-east-1_ region (CloudFront ACM certs must be there). Create Details create a serverless workstation (if you don't have one) From AWS CLI aws cloudformation create-stack --stack-name <stackName> --template-body file://cfn-ec2workstation.json --parameters ParameterKey=myKeyPair 1 2 From AWS Console - use cfn-ec2workstation.json requires a KeyPair, optionally can add a domainname to set a convenince URL sls-{region}.{domainname} Steps to build 1 Check out this Git Repository and change into the slsweb directory 1 TODO enter required configuration otherwise you will get errors about SSLCert being unknown. domainname TODO have the git repository https://github.com/mcliff1/acg-react hard coded in the CodeBuild configuration Run sls deploy --domainname <domain> and it will deploy entire stack Run Code Build (from CLI?) aws codebuild .... Work Process, once we make an update to the Git repository, then run the CodeBuild process. Alternatives Use CodePipeline to watch the github source add plug-in to github with AWS credentials to trigger CodeBuild on push Parameters Outputs Public S3 Url https://s3.amazonaws.com/mcliff/cliffconsulting.com/ TODO Current State (as of 4/30); added fix to set content-type on s3 upload (this was causeing index.html to be downloaded); needed to manually add the S3 trigger to the Lambda function (some access issues still); need to manually run the CodeBuild after deploy is completed. To fully deploy this (we assume we have some GIT Repo or code base to deploy as web content) Building this I set my content in https://github.com/mcliff1/acg-react Next Steps - have SNS Topic message get sent after the build is complete Start flushing out changes to the REACT side of the app

References

back to top

• AWS Bastion Host

docker entrypoint vs cmd