How to Easily Setup and Manage Live Infrastructure with Chef's Own, Nathen Harvey

Interviewed by Christophe Limpalair on 08/31/2015

Learn how to configure and test your infrastructure locally, and then deploy it live for the world to enjoy. Once it's live, Nathen shows us how to manage and update nodes from our local command line.

Next, you'll want to deploy code. Don't let errors slow you down or scare you from it. Track every error as it happens with Rollbar, just like I do. Try it free.


Similar Interviews

Docker, The Art of Monitoring, and Kickstarter Engineering with James Turnbull

How Netflix handles 36% of the U.S. bandwidth, and how reddit handles 160M users

Links and Resources


Interview Snippets

This episode is a sequel where we take the webserver we set up in the first episode and deploy it to a live server so anyone can access our index.html page. Then, Nathen takes it a step further by showing us how to manage live nodes from our local command line.

Here is the repo with the exact code we used in the episode. I recommend you download it and follow along that way, then create your own and apply what you've learned.

I also recommend you watch the first episode if you're not at all familiar with Chef, but I wouldn't say it is absolutely necessary because Nathen gives a quick refresher at the beginning of the episode.

Chef workflow review

Before we get started, keep this workflow in mind:
1) Change your code locally (on your workstation)
2) Test it
3) Push your changes to the Chef Server
4) Deploy to your nodes

First Step

To begin this episode, Nathen modified his
file to forward port 80 to 8080 in Vagrant:
- ["forwarded_port", {guest: 80, host: 8080}]

Vagrant Driver settings to open port 80 in Chef

This will make it so we can have access to our Kitchen environment from our browser. Nothing new if you've worked with Vagrant before.

It works the same if you use Docker or if you want to use DigitalOcean or AWS. Just change the name of the driver. Remember that this is for testing, so keeping it local should work fine for most people.

Launch your testing VM

After making your change in
go to your command line and change your directory to cookbooks/webserver/ then run
kitchen list

kitchen list
lists the different machines we have configured.

Kitchen list command to list machines in Chef

To boot it up, type
kitchen converge

After you run the converge command, a number of things will happen:

  1. Spins up a virtual machine

  2. Installs Chef on the machine

  3. Runs through your recipes:

    1. Update APT caches

    2. Install Apache

    3. Write HTML file

Once it's done running and performing all these operations, you can open your browser and navigate to localhost:8080

Kitchen Tests

At this point, you can run your Kitchen tests to make sure everything that you expected is working, but that's not necessary here because we see it is working on our browser.

kitchen test

By the way, our tests are in:

Let's go live with EC2!

Here's an overview of what we want to accomplish in the next few sections:
1) We want to take our cookbook and publish it to the Chef server.
2) After that, we want to launch an EC2 node on AWS. (Chef works with any service, not just AWS)
3) That node will check with the Chef server and pull down its policy (cookbook).

Chef Server

Bigger organizations will set up a Chef server of their own. This is time consuming and perhaps not necessary if you just have a few nodes you want to manage. Thankfully Chef offers a hosted Chef Server. This is free up to 5 nodes.

Let's set up the free one.

Go to the main Chef website and click on Get Chef.

Sign up for Hosted Chef.

Create your organization, then Download Starter Kit.


Move that chef-repo/.chef you just unzipped to your main chef folder.

That directory contains PEM files which authenticate you. There is also a knife.rb file which contains information like your

Knife.rb file in Chef

We now have our local workstation configured to communicate with our Chef Server

Upload your Cookbook

Go in your
folder and run
berks install

Berks install command in Chef

This will resolve any dependencies we have, and download dependency cookbooks locally.

Now we need to put them on our Chef Server:

berks upload

Image of Berks upload in Chef

If you run
knife cookbook list
, you will see which cookbooks are available on our Chef Server. You can also do this from your Chef Server dashboard of course. (Under Policy)

Launch a node on AWS

In order to do this, we need to modify our knife.rb. file we just downloaded.

vim .chef/knife.rb

We need to add our AWS credentials.

Do this by creating a new user in your AWS management console under Services > Administration & Security. More on setting up users in AWS.

Place this new user's AWS credentials like the access key ID and secret access key in your knife.rb file as shown in the picture:

Knife AWS credentials in Chef

Now just to make sure you won't run into issues, make sure you have this gem installed:

chef gem list knife-ec2

Chef Gem List command

That gem comes with the Chef Development Kit so you should already have it if you followed the last episode. If you don't have it, go ahead and download that here.

Time to create our instance!

knife ec2 server create --image ami-00615068 
--ssh-user ubuntu
--flavor t2.micro
--security-group-ids sg-16899172
--run-list 'recipe[webserver]'
--node-name production-web

Before you run this command, make sure you customize it for your needs. For example, the security group ID will most likely be different. If you're not sure what a security group ID is, refer to this post.

Security groups are super important, because depending on the group, your port 80 could be blocked and you won't be able to visit your site.

Knife EC2 create command in Chef

What happens after you run the command:

  1. Waits for machine to boot up

  2. Waits for SSH to be available

  3. Installs Chef on that node

  4. Configures Chef on that node

  5. Runs Chef

This may take a few minutes.

Thanks to Rollbar for making my life easier

I take sponsorship at ScaleYourCode very seriously. If it's not a product or a company that I think can really benefit the audience, then I don't think they're a good fit for the show.

That's why Rollbar is such a great fit. If you're a developer who touches any code, whether it's in production or development, and you're not using error tracking, go to and stop wasting hours of debugging.

Get better visibility with your apps, see things breaking when your users run into errors and get notifications so you can have someone fix them before more users run into problems.

Or even in the debugging phase of development. You're working on a feature and you are running into errors...Rollbar can help you fix them faster by showing you exactly what went wrong.

I really think you're going to like it.

EC2 continued

Just FYI- Nathen ran into an error with the knife EC2 command, because the first time he hadn't uploaded his webserver cookbook. The second time he ran the command, there was already a client by that name in the Chef Server.

Since Chef already had a key for that client, and he was trying to access it with a different client, it caused a failure.

He fixed it by renaming it to production-web1.

I added this to warn you about it.

Anyway, once the command successfully completes, it will return the instance's information:

EC2 Node info after running Knife create in Chef

At this point you should be able to take the Public DNS Name and open it in your browser.

(Obviously your DNS address will be different)

At this point, Nathen walks us through our Chef Dashboard where we can see a lot of information about our node(s). The amount of total RAM, used RAM and free RAM, for example, and much more.

All this information is displayed using a tool called ohai. Ohai is a system profiler that you can execute on nodes so that your recipes can make decisions about information from your systems.

For example, if you know that your system has 8GB of memory, you could assign a portion of that memory to a particular process -- like 20%. Then it doesn't really matter if your system has 8GB or 4GB, that service will still use 20%.

Knife commands

Here are some useful Knife commands:

knife node show [node name]

This will show you the node IP, run list, recipes, platforms, and more.

knife node show [node name] -l

This shows a lot more information, like the uptime, machine type, memory, cpu, and much more. If that's too much data, you can filter:

knife node show [node name] -a ec2

This returns only information from EC2.

This is great because you could write scripts that utilize this data to build reports, or something like that.

knife search node "name:*web*"

This will search for all nodes on your Chef Server, where their name is like "web".

knife search node "name:*web*" -a platform

This will return the available platforms from your infrastructure where the node name is like "web".

knife search node "name:*web*" -a platform_version

Same as the one above except it just returns the OS version number.

You can find more knife commands here.

What can we use this type of searching for?

One of the reasons why this searching is so important is because you can execute remote commands on your entire infrastructure.

Let's say for example that we want to know every Ubuntu node's uptime.

knife ssh "platform:ubuntu" "uptime"

This is going to query the Chef Server from our local workstation and ask it to return a list of all nodes that match the Ubuntu platform. Once we get the list, we are going to make a parallel ssh connection to all the matching nodes from our local workstation.

Knife SSH in ubuntu machines only to check uptime, with Chef

As you can see, this gives us full control over our infrastructure.

Change your configuration and push changes live

Instead of just having a webpage that says
Hello, Chris!
, lets display some of the instance's settings from our ohai data we talked about earlier.

We can use a command we discussed earlier to find EC2 parameters we could use:

knife node show production-web1 -a ec2

Knife node show filter for EC2 only in Chef

Nathen picked ami_id and added:
	<%= node['ec2']['ami_id'] %>
<%= node['ec2']['local_hostname'] %>

In cookbooks/webserver/templates/default/index.html.erb

Node EC2 attributes in index.html

Those attributes are made available in the node object.

However, these are only available on ec2, so we need to define defaults for our local workstation or our kitchen test will fail.

Go to your cookbooks/webserver/kitchen.yml and add:
ami_id: foo
local_hostname: bar

After making those changes, you'll want to update the metadata.rb


That way you can make sure your infrastructure at the previous version will not change.

Manually going in the file and changing the version can get tedious. So instead, you can run this command:
knife spork bump webserver

After the changes, you run
kitchen test

This will destroy your old VMs and launch new ones with the new changes.

In the meantime, run
berks install
which tells Berks that you have a new version of the webserver cookbook.

And then,
berks upload
to push this new version to Chef Server.

Now we can do
knife cookbook list
to see each cookbook's version.

But if we do a
knife cookbook show webserver
, we will see all the different versions stored on our Chef Server.

This is really useful if you're testing new configurations on a few nodes but you want your old and stable infrastructure to remain the same, for example.

Apply changes to your node(s)

We need to re-run the Chef client. When you do that, your nodes will check in with our Chef Server and receive its new policy.

Lets update our nodes:

knife ssh "platform:ubuntu" "sudo chef-client"

Like we discussed earlier, this is going to find all our nodes with Ubuntu installed, SSH into that node, and run the command
sudo chef-client

Refresh the page, and you should see Hell, Chris! followed by the

Using Attributes

So we are pulling values from our EC2 nodes, but what if we wanted to set those values ourselves, no matter what service we decided to use?

You accomplish this by using attributes.

First, generate an attributes file:

chef generate attribute . default

In your

You will find a default.rb file in cookbooks/webserver/attributes/default.rb

Add your attributes:

default['webserver']['greeting'] = "Scale Your Code"

Then use it in one of your files, like index.html in cookbooks/webserver/templates/default/index.html.erb for example.

Using attributes in Chef

Don't forget to change your version in the metadata.rb with
knife spork bump webserver
, then
Berks install && Berks update
, and finally
Berks upload

Note that this should be run in the chef-repo directory (outside of cookbooks/).

Run the
knife ssh "platform:ubuntu" "sudo chef-client"
again to push our changes to all our nodes that have Ubuntu installed.

How do you build out to more than one cookbook? (ie: more services instead of just webserver)

Typically, you would create multiple, separate cookbooks.

When you run the chef-client, it looks at the run list. For example in our cookbooks/webserver/.kitchen.yml we have:
- recipe[webserver::default]

We can add multiple recipes:
- recipe[webserver::default]
- recipe[database::client]

So you can break out different components in different cookbooks. This makes it easy to utilize cookbooks in different locations and on many different, but separate, nodes.

Taking down our node

Once we're done messing around, how do we turn our nodes off? Don't forget to do this if you don't need them anymore, because you could get charged.

Delete Nodes

What now?

This episode walked through how to push our infrastructure live, and how to continually update it and reflect those changes on our live nodes.

All of this was using a webserver -- Apache in this case. But what about installing a database and application? There are great tutorials from Chef that explain how to do this.

There are also more great videos and articles that explain DevOps, security, and what we just went through here.

Nathen says that the best way to go from here is to think about a problem you are trying to solve. Make sure you fully understand that and then with your newfound understanding of Chef, go out there and build something amazing!

(And show us when you do :) )

Connect with Nathen

@NathenHarvey on Twitter.

How did this interview help you?

If you learned anything from this interview, please thank our guest for their time.

Oh, and help your followers learn by clicking the Tweet button below :)