Building aws ec2 with puppet
Puppet Enterprise has a useful system called Cloud provisioner which will manage the creation, deletion and puppet classification of virtual machines or “instances”. This cloud provisioner system supports three well known virtualization / cloud systems; Amazon Web Services, Google Compute Engine and VMware. This document is going to cover the building aws ec2 with puppet on Amazon Web Services. I will cover the commands Puppet Enterprise provides and give an example script for you to use on your own systems.
How does it work?
Amazon Web services or AWS (I will refer to it as AWS from now on) comes with an API (Application Programming Interface” which allows you to create EC2 instances “Programmatically”. Puppet Enterprise utilizes this API and allows us to build instances, optionally classify them as a puppet group and very usefully install the puppet agent. This means potentially you can have your instances being built and being managed by puppet in one step.
Setting up the prerequisites
Get Puppet Enterprise
You will need a puppet master server running Enterprise on once of your EC2 instances. An m3.medium is powerful enough for testing and probably for some production setups For this example I am using a Red Hat Enterprise 6.5 as my operating system – but you can do this very easily with Centos or Ubuntu et al.
Create a .Fog File
I am not sure what fog means and its an odd name for a file. Anyway suffice to say you need a .fog file to allow puppet to authenticate your connection to cloud systems in this case aws. It simply contains your API credentials.
So within your users home directory create a .fog file. In my case the user I am using is the AWS supplied ec2-user, but you can use whichever account you want.
Add the following entries to the empty .fog file remembering to replace it with your own AWS access codes
Note I just made up the above key so I hope that it doesn’t actually refer to someones real account 🙂
Create a Puppet Friendly AWS EC2 Security Group
Create a security group, mine is called PuppetHosts and has traffic / firewall rules as shown below. The rules shown will ensure that your newly create instance can talk to the puppet master and you can SSH. Naturally the rules can be tightened by specifing actual IP addresses rather than using the broad “Anywhere” as I have.
Optionally Add a Puppet Group
Building an AWS Instance
This is very easy. The only tricky part is typing all of the command options on one line at the command prompt. You can run it from a shell script to make this less cumbersome as I have done in the following example.
so create a file in your home directory called aws_build_instance.sh
edit it and enter the following text.
NOTE: You will need to make some customizations to your environment and these are all found in the section between #Parameters for building a host and #End Parameters
#Parameters for building a host
#This is the SSH key you want to inject into your instance to allow you to login.
#Set the security group you wish this instance to belong to. This group should allow for puppet traffic 8140 and 61613
#Set the ami image you want e.g Redhat Ubuntu
#Set the type / Size of Instance
#Set the Region where this instance will reside
#Set the availability Zone
#Build The Options String to Create the Node
puppet node_aws create $options
save the file and run it
if all goes to plan you should see output like so:
Notice: Creating new instance ...
Notice: Creating new instance ... Done
Notice: Creating tags for instance ...
Notice: Creating tags for instance ... Done
Notice: Launching server i-02efd430 ...
Notice: Server i-02efd430 is now launched
Notice: Server i-02efd430 public dns name: ec2-23-125-44-97.eu-west-1.compute.amazonaws.com
Optionally Puppet Classify your Instance
If you are going to manage your instance with Puppet its a good idea to add it to a group so the customisation and classes can be added. This is pretty easy.
So create a file called
edit it and enter the following text ( You will need to alter it to match your own environment. In particular the AWSINSTANCE parameter as this names the server you wish to classify)
#Parameters for Classifying a host
#Enter the dns/host name of the instance you just built
PUPPETMASTER_CLOGIN=qets.puppetAdmin@quindell.com #Puppet Console U/Name
PUPPETMASTER_CPASSWD=Sunsh1ne #Puppet Console P/Word
#Enter the NODE group of the server you are
puppet node classify $options
Now run it – if all is good you will get a simple response like so
Notice: Contacting https://localhost:443/ to classify ec2-23-125-44-97.eu-west-1.compute.amazonaws.com
That is that your instance is now classified. When you connect it to puppet all you software will be delivered and installed.
Install the Puppet Agent
We can script and push the puppet install. And this section will cover this. We need two files one to run the installer and a second
Create a file called aws_install_pupp.sh
Now edit the file
#The puppet agent cert name
#The installer answer file which will be uploaded
#The login of the install (root for some like Centos or ec2-user if it is an official Amazon AMI)
#The path to your ssh key that was injected in the instance (for authentication)
#Puppet Installation Path (where the install of puppet lives )
/usr/local/bin/puppet node install --debug $options
Now create a second file which will be the answer file which will automate the agent install. Edit the file and add the following answers. This will ensure the installer runs silently on the box and doesn’t fail.
Save the file and then run the installer we created earlier
This will kick off the puppet installer agent.