Using Fabric to Create a LAMP server on HP Cloud

May 29 2013

Fabric is a network-aware build tool. Think of it like make (or rake, or pake, etc.) for remote servers. With Fabric, it is just as easy to script a remote environment as it is to script a local one. In fact, with Fabric, you can do both.

Fabric can actually be used to automate installing a server in the cloud. In this article, I show how to use Fabric with the hpcloud UNIX client to create an HP Cloud server instance and then configure it as a LAMP server.

While I show this with HP Cloud, the technique could readily be adapted to other cloud providers like OpenStack or AWS.

Updated: Fixed indenting that was breaking Python. Thanks to aeronotix. <!--break-->

Prerequisites

From the outset, I assume a few things:

  1. You have the hpcloud UNIX command line client installed and configured. (There is no reason why you can't substitute some other cloud CLI and tweak for another platform.)
  2. You have already set up your cloud provider's account and created an SSH key for your cloud account. My key is called hpcloud.
  3. You have installed Fabric.
  4. You have security groups set up for Compute. (A security group is like a firewall for cloud instances.) Mine is called Web.

Later on in this article the fabric script puts a file called index.php onto the server. This is what that file looks like:


    <?php
    print "Hello World";
    ?>

Getting Started with Fabric

Just as make reads a Makefile, fabric reads a fabfile.py. A fabfile doesn't have its own format; it's just a Python script. Here's a basic fabfile to get started with:


from fabric.api import *

@task
def hello():
    puts("Hello")

The first line (from fabric.api import *) imports the fabric system. Most Fabric scripts start with this line.

Next, we define a Fabric task. We use the @task decorator to tell the Fabric system that this is a task that can be run from the command line. Our example hello task prints "Hello" to the console.

At this point, we have a working fabfile. Fabric will tell us all of the commands that it knows about by running fab --list in the directory with the fabfile.py:

    $ fab --list

    Available commands:

    hello

We can execute our command with fab hello:

    $ fab hello
    Hello

    Done.

And there we have it. From here we can do something much more interesting.

A Task for HP Cloud

Fabric can run commands locally on the host operating system. Since we have the hpcloud client installed locally, we can run it programmatically with fabric. For example, let's write a quick command that displays a list of all of the servers in our current availability zone:


@task
def server_details(name = ''):
    """Get the server details."""
    local("hpcloud servers %s" % name)

The task above runs the local command hpcloud servers. If name is specified, it runs hpcloud servers NAME. But if name is empty, it will print a list of all of the servers.

Running fab server_details should print out a list of all of your servers. Running fab server_details:foo will print the details of the server whose name is foo (assuming you have one). We'll use that form of the command a little later.

Really, all Fabric is doing here is running the command hpcloud servers for us.

Create a New Server

Let's build a more complex command that uses local() to create a server:


@task
def create_server(name):
    """Create a new HP Cloud server."""
    image = 81078 # Ubuntu 12.04 in AZ-1
    flavor = 100  # extra-small
    security_group = 'Web'
    key = 'hpcompute'

    # This is the local command we'll run.
    template = "hpcloud servers:add %s %s -i %s -k %s -s %s"
    cmd = template % (name, flavor, image, key, security_group)

    local(cmd)

This command is a little wordier, but all it really does is run hpcloud servers:add with a bunch of pre-configured settings. Notice that since the function create_server() takes a parameter name, and since name does not have a value, it will be a required parameter. To run it, we can do this:

    $ fab create_server:dev
    [localhost] local: hpcloud servers:add dev 100 -i 81078 -k hpcompute -s Web
    Created server 'dev' with id '1512143'.

    Done.

fab create_server gets one parameter (name), which we set to dev. Fabric does use the rather non-conventional method of passing parameters to tasks using the colon syntax above. But as odd as it looks at first, it's easy to use.

Now we can use the server_details task we created earlier to see the server's status:

    $ fab server_details:dev
    [localhost] local: hpcloud servers dev
      +---------+------+--------+-------+----------------+--------------+-----------+-----------------+----------------------+--------+
      | id      | name | flavor | image | public_ip      | private_ip   | keyname   | security_groups | created              | state  |
      +---------+------+--------+-------+----------------+--------------+-----------+-----------------+----------------------+--------+
      | 1512143 | dev  | 100    | 81078 | 15.185.111.149 | 10.5.250.104 | hpcompute | Web             | 2013-05-29T02:52:47Z | ACTIVE |
      +---------+------+--------+-------+----------------+--------------+-----------+-----------------+----------------------+--------+

    Done.

This tells us that our server has been created, and is now ACTIVE and ready for work.

Installing a LAMP Stack

Now that we have a server available, we can create a new task to install the LAMP stack on this server. While the other tasks have been running local commands, this one will use SSH and SCP (in the background) to interact with the server.

Here's our new task:


@task
def install_lamp():
    """Install a LAMP stack on the server."""

    # Install the packages
    sudo("apt-get -q update")
    packages = "apache2 mysql-server libapache2-mod-php5 php5-mysql"
    sudo("apt-get install -q %s" % packages)

    # Push the index.php file to the server and get rid of
    # the old index.html file.
    put("index.php", "/var/www/index.php", use_sudo=True)
    sudo("rm /var/www/index.html")

The install_lamp task does the following:

  • It runs apt-get update to get the latest package repository.
  • It then uses apt-get install to install the packages we need.
  • It copies a file called index.php (in the local directory) to /var/www/index.php
  • And then it removes the old /var/www/index.html file.

There are two important commands used above:

  • sudo: This performs a sudo command on the remote server. If you don't need root permissions, use run instead of sudo.
  • put: Copies a file to the remote server.

These are built into fabric, which also has many other useful functions like this.

To run this task, we do have to provide an additional piece of input: We need to tell Fabric which server to run the command on. Often times, we could store this information inside of the fabfile.py or in a config file, but here I'll pass it in on the command line using the -H flag. Note that it follows SSH, specifying USER@HOST:

    $ fab -H ubuntu@15.185.111.149 install_lamp
    [ubuntu@15.185.111.149] Executing task 'install_lamp'
    [ubuntu@15.185.111.149] sudo: apt-get -q update
    Lots and lots of lines…
    [ubuntu@15.185.111.149] put: index.php -> /var/www/index.php
    [ubuntu@15.185.111.149] sudo: rm /var/www/index.html

    Done.
    Disconnecting from ubuntu@15.185.111.149... done.

When you run this on your own, you will notice something that I didn't capture above: If the remote server prompts for input, Fabric passes that prompt on to you. So you can actually do interactive work as you go (like settings passwords).

At the end of the installation process, we can do a simple browser-based test. By pointing a browser toward the public IP address of the server (http://15.185.111.149) we can see whether our new index.php gets executed correctly.

Conclusion

The fabfile.py script that we covered here is very basic, and requires multiple steps to configure the server. But we could easily build more involved Fabric scripts that, for example, do the entire setup by calling one task. In the past, I've used Fabric to set up block storage and other cloud services, too.

Better still, a single Fabric file can become a tool for not just installing servers, but maintaining them. Fabric can be used to automate application deployment, remote backups, and other standard maintenance tasks. And since they can be used like Makefiles, it's not at all uncommon to see a fabfile.py at the root of a project.

The entire script that I used (including a destroy_server task I did not discuss here) is available in this Gist.