In a typical high-availability AWS application footprint there are several resources required: Elastic Load Balancers, EC2 instances, EBS volumes, SimpleDB domains and an RDS instance. They also setup Auto Scaling, EC2 and RDS Security Groups, configure CloudWatch monitoring and alarms, and use SNS for notifications.
There is a good getting started page here: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-using-cli.html
Make sure the CLI is working. You should be able to run aws cloudformation list-stacks and see an empty structure if you have no formations yet.
Now let's start using the cloudformation api through the cli tools
As an example, let's suppose we need an in-memory key-value cache. We decide between memcached and redis which AWS supports, to use Redis, and we find a template for it on the above page and download it.
It took me about an hour to get the syntax for the CLI working, especially since some options are not documented in the tool and I had to search for 45 minutes to figure out there is one parameter that takes a hardcoded value, namely
--capabilities=CAPABILITY_IAMAnyway, here was my final command
aws cloudformation create-stack --stack-name=Claytons-redis-stack \ --template-body=file:///Users/mitchhan/aws-cli/tests/ElastiCache_Redis.template \ --parameters ParameterKey=KeyName,ParameterValue=aws_rsa --capabilities=CAPABILITY_IAM
Once created, list it with
aws cloudformation list-stacks
Now let's create Tomcat and MySql components, or beanstalk and RDS components in AWS terminology. I find the ElasticBeanstalk_Simple.template from the above page, save it locally in case I want to modify anything and execute
aws cloudformation create-stack --stack-name=Claytons-beanstalk-stack \ --template-body=file:///Users/mitchhan/aws-cli/tests/ElasticBeanstalk_Simple.template \ --capabilities=CAPABILITY_IAM
I wanted to add my existing keypair into the template, and found that this was not as simple as I had thought. For one, the beanstalk resource sets this property differntly than the EC2 resource, even though in the end it's the EC2 which gets the property, cloudformation needs it passed in the beanstalk in a different way than a direct EC2 resource in cloudformation. So, making EC2's in cloudformation would have the key line look like the example below.
"Resources": {
"instance1": {
"Type": "AWS::EC2::Instance",
"Properties": {
"AvailabilityZone": "us-east-1d",
"DisableApiTermination": "FALSE",
"ImageId": "ami-12345678",
"InstanceType": "m1.small",
"KernelId": "aki-87654321",
"KeyName": "MyKeyPair",
"Monitoring": "false",
"SecurityGroups": [{"Ref": "sgfoo"}]
}
},
Notice it uses an OptionsSettings object with EC2KeyName, under the path of
Resources->Properties->ConfigurationTemplates->OptionsSettings
And where the name was KeyName in EC2, it now is EC2KeyName in the Beanstalk.
"Resources" : {
"sampleApplication" : {
"Type" : "AWS::ElasticBeanstalk::Application",
"Properties" : {
"Description" : "AWS Elastic Beanstalk Sample Application",
"ApplicationVersions" : [{
"VersionLabel" : "Initial Version",
"Description" : "Version 1.0",
"SourceBundle" : {
"S3Bucket" : { "Fn::Join" : ["-", ["elasticbeanstalk-samples", { "Ref" : "AWS::Region" }]]},
"S3Key" : "elasticbeanstalk-sampleapp.war"
}
}],
"ConfigurationTemplates" : [{
"TemplateName" : "DefaultConfiguration",
"Description" : "Default Configuration Version 1.0 - with SSH access",
"SolutionStackName" : "64bit Amazon Linux running Tomcat 7",
"OptionSettings" : [{
"Namespace" : "aws:autoscaling:launchconfiguration",
"OptionName" : "EC2KeyName",
"Value" : "aws_rsa"
}]
}]
}
},
For the complete working template:
I bring some further customization requirements to play for the sake of understanding what is possible, easy or hard, or impossible in cloudformation. Generally I have some reason I want to modify the template, and in this case, I already have a security group with ingress rules I like to re-use, so why not re-use that in my template instead of create a new security group which would do the same thing but have a different name? Sounds reasonable I think.
There is a post on just that, https://forums.aws.amazon.com/thread.jspa?threadID=73197 What that would look like for an EC2 resources is below. I will need to modify that for a beanstalk resource and see if I have learned anything from my key pair custom work above that will help here.
"Resources" : {
:
"Ec2Instance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"SecurityGroups" : [ "MyExistingSecurityGroup" ],
"ImageId" : "ami-12345678"
}
},
That works for an EC2 Instance type object, but not in the Beanstalk Application object. I found this developer guide on beanstalk and VPCs which has some hints at the bottom of the page, http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/AWSHowTo-vpc-basic.html
According to that guide, it appears one can use the below list of options
option_settings:
- namespace: aws:autoscaling:launchconfiguration
option_name: EC2KeyName
value: ec2keypair
- namespace: aws:ec2:vpc
option_name: VPCId
value: vpc-170647c
- namespace: aws:ec2:vpc
option_name: Subnets
value: subnet-4f195024
- namespace: aws:ec2:vpc
option_name: ELBSubnets
value: subnet-fe064f95
- namespace: aws:autoscaling:launchconfiguration
option_name: InstanceType
value: m1.small
- namespace: aws:autoscaling:launchconfiguration
option_name: SecurityGroups
value: sg-7f1ef110
You may also find this resource useful to lookup options settings for beanstalks: "ResourceStatusReason": "Configuration validation exception: The security group 'sg-c7a86aa2' does not exist"OK, at least the config is passing, and I have something else wrong now. Although I am sure that is my security group ID in one of my EC2 instances. Maybe I can't use that for some reason. Googling a bit I found the EC2 API Error Code doc http://docs.aws.amazon.com/AWSEC2/latest/APIReference/api-error-codes.html
Adding in the VPCId of my security group BEFORE the security group option solved the security group error, but requires that i do more to specify elements of the VPC ( that I already have ). Big sigh.
"ResourceStatusReason": "Configuration validation exception: Specify the subnets for the VPC.",
I'm not sure why cloudformation with the beanstalk application I had, when I didn't mention a VPC, created the networking elements just fine on its own, but now that I mention a specific VPC, I need to specify more details. This is leading into a deeper exploration of the AWS VPC which is covered on the next page.