Using Salt Stack and Vagrant for Drupal Development

What if, just like Bill Murray in Groundhog Day, you could wake up to a fresh and identical development environment completely free of yesterday's experiments and mistakes? Vagrant lets you do exactly that.

Or, what if, like Jake Epping in Stephen King's 11/22/63, you could make changes and script the past without fear, play around with some new Drupal modules, and quickly reset everything just by leaving and then walking back down the stairs of the pantry again?

Would you like to automate the creation and installation of a pristine Drupal environment, instead of manually installing Apache, PHP and all of the needed libraries?

Recently, I read a post from Treehouse Agency titled, “End 'Works on My Machine' Surprises with Vagrant” (treehouseagency.com/blog/steven-merrill/2011/11/02/end-works-my-machine-surprises-vagrant), which is about using Vagrant and Puppet for Drupal development. You may have seen it as well on Drupal Planet (planet.drupal.org) and wondered about the benefits described there.

This is a great technique that outlines using various tools to build a consistent development environment quickly that is repeatable and shareable with everyone on your team.

Linux Journal also recently featured an article that introduced Vagrant (www.linuxjournal.com/content/introducing-vagrant). After you're finished here, I urge you to go read that one too, because it offers more general information about Vagrant itself.

Salt Stack (salstack.org) is a tool similar to Puppet, but it's powered by Python, instead of Ruby. If you are interested in standalone Salt installation and use, check out my November 2012 LJ article on that topic (www.linuxjournal.com/content/getting-started-salt-stack-other-configuration-management-system-built-python), where I introduce Salt Stack and show how to install it to control one or thousands of other machines. I won't go in depth into Salt's installation here, because Salty-Vagrant takes care of all of that for you.

I've been using Salt, Vagrant (vagrantup.com) and a Vagrant gem called Salty-Vagrant (https://github.com/saltstack/salty-vagrant) for Drupal development quite a bit lately. I've found that having a standard configuration that mirrors my development, testing, staging and production environments has streamlined my workflow and helped prevent a lot of the unknowns between different stages of the Drupal development life cycle. I've been able to minimize a lot of the errors and headaches that come with integrating multiple software stacks and speed up my workflow too.

I also enjoy being able to try new things quickly and easily without spending hours reconfiguring and re-installing the entire software stack needed for Drupal development.

I then can share these configurations quickly and easily with other developers on my team and know that we are each working with the same versions of software already installed. In this article, I describe how you can do that too.

With Salt Stack, you store your configurations in text files, similar to Puppet. The difference is that the configurations rely on YAML (yaml.org) templates instead of Ruby code. This makes them easy for humans to read, even if they aren't Python coders.

These configurations can be used to launch development machines, as well as testing, staging and production machines—and all of them are identical and easily transportable. I use Git to manage my team's configurations. This brings all of the great aspects of version control to my server configurations too.

With Vagrant, you easily can delete a development environment and then re-create it in about as much time as it takes for the server to boot and install the packages and software you specify—kind of like a quick undo or reset button.

You need to download and install a few things to get all of these pieces to work together. Each component has very good documentation on-line for installation and use, so I won't duplicate the instructions here. Instead, follow the links for each piece of software and check the projects' sites for the most up-to-date information.

This article focuses on using an Ubuntu-based 64-bit VirtualBox, but Salty-Vagrant with Salt Stack supports BSD, Red Hat, OS X and Windows virtual machines as well.

The beauty of using Vagrant with Salty-Vagrant is that the configuration files, also known as SLS files, transfer exactly to production servers, as well as anywhere else you might use them. The benefit of creating one template and then re-using it again and again can help you direct more energy toward Drupal development, instead of troubleshooting server-software installation too.

You need to do this installation and configuration only once. After that, it is as simple as vagrant up!

VirtualBox allows you to run a guest operating system inside your own host operating system. It is similar to dual-booting, except you still can use your main operating system and have access to additional running operating systems as well. For example, you can run an Ubuntu server inside your Windows, Mac or Linux host operating system and still access your e-mail and text editor from your native host operating system.

Follow the instructions at virtualbox.org to install VirtualBox. Versions are available for most major operating systems. A major portion of popular Linux distributions also have packages available through their respective package managers. This is the most effective and efficient way to install the majority of tools I describe here.

Vagrant is a Ruby gem that allows you to create and destroy virtual machines quickly, and then reset them almost instantly.

Gems are like Drupal modules, but for the Ruby programming language. These plugins let you extend Ruby's functionality and add additional capabilities. You should ensure that you have a recent version of Ruby installed. I recommend using your package manager if you don't have it, but a lot of distributions include Ruby already, so you likely won't need to install it. You easily can find out whether you have Ruby installed by typing ruby --version from your terminal.

Salty-Vagrant is another Ruby gem that brings the functionality and automated configuration tools of Salt Stack to Vagrant. Salty-Vagrant acts as a bridge between the Salt Stack configuration management system and Vagrant/VirtualBox. The project's page has installation instructions that are updated on a regular basis. It would be a good idea to refer to these instructions just in case something has changed after this article is published. If you know a little Ruby or shell scripting, feel free to join in this open-source project and help contribute back. You'll find the development community to be friendly and welcoming.

Get the base box from vagrantbox.es. You'll need a box in which to create and run your environment. Boxes are what Vagrant uses for your virtual machines, and they are simply virtual machines that have been configured specifically with Vagrant support. You can create your own from scratch, but it is much quicker and easier just to download a pre-made one that has all of the necessary Vagrant configurations already set up. I used the precise64 box. This is an Ubuntu 12.04 64-bit Linux server with the necessary Vagrant tools and guest additions already in place. You're downloading an entire operating system, so be patient, because it might take a little while. It usually is best to download your box and then save it locally somewhere so you can use it again.

If you're using a 32-bit host operating system, you should instead download the 32-bit version.

Hopefully by this point, you have everything you need. Let's use the command line to interact with Vagrant and import your new box.

To import your box, use:

vagrant box add precise64 /path/to/your/downloaded/box

Relative path names don't work well here, so if you downloaded your box to your home directory, you need to type out the full path—for example, ~/bhosmer/precise64.box should be typed out like this: /home/bhosmer/precise64.box.

The vagrant box add command creates a copy of your box and stores it so that Vagrant can access it. This box is used to create the instantiation of the particular box you happen to be using. The nice part is that you always can press the “reset” button by using vagrant destroy to delete anything you've done and set it back to its original state before you started. I go into that in more detail a little later.

Now, launch your box just to make sure that the VirtualBox guest additions are up to date. If they aren't, Salty-Vagrant may not work properly.

Where you store your Vagrant configurations is entirely up to you. I like to keep mine in my home directory under a vagrant folder, just so I can keep track of the different environments I have.

Create a new directory called salty-vagrant and cd into that directory.

Now use vagrant init to initialize a Vagrantfile there. This is a standard Vagrant configuration file that is well documented. You won't need to change any of the configurations, except for line 10:

config.vm.box = "base"

If you followed the quick-start instructions from www.vagrantup.com, Vagrant specifies your default boxes as base. Change this to the name of the box you imported, precise64:

config.vm.box = "precise64"

Notice that I left the .box extension off.

Now that your Vagrantfile points to the correct box, use vagrant up to start your virtual machine.

If you get a warning that your guest additions are out of date, and it is quite likely that you will if you just installed VirtualBox, proceed to the next step. If you didn't, you can skip to the section “Add a Salty-Vagrant Vagrantfile”.

If you do need to update the guest additions installed on your box, in this section, I explain how to do it on the Ubuntu-based box you just downloaded. This will be one of the few times that your changes will be permanent within a box. And this time, you want them to be. The guest additions allow your host operating system to interact with the guest operating system. This includes sharing files and network resources between them. You store your configuration files on your host operating system, and then they are copied to the guest operating system when you start it up each time, so it is important that the shared-folder functionality works.

After Vagrant finishes booting the guest operating system, you'll need to connect to this box using SSH. Use the Vagrant-supplied command: vagrant ssh.

You then will be logged in to the guest operating system as the “vagrant” user.

Updating the guest additions probably is the trickiest and most-involved task. If you run into any problems, lots of resources are available on-line. Most distributions mention a number of solutions in their respective forums, and the VirtualBox Manual (https://www.virtualbox.org/manual/ch04.html) also has some detailed information. I'm going to show you how to download and update the virtualbox guest additions from virtualbox.org, and update them on the precise64 box that you just downloaded as an example.

First, you need some packages to build and compile the Linux kernel. Use Ubuntu's package manager, apt, to install them:

sudo apt-get install dkms linux-headers-3.2.0-23-generic

If you're using a non-Debian machine, search for the proper names in your particular package manager and use the header version that matches your particular kernel.

With kernel support now in place, browse to download.virtualbox.org/virtualbox and locate the folder that matches the version of VirtualBox that you have installed. Generally, you can find what version you have through the VirtualBox graphical interface or the warning message generated by Vagrant when you booted your virtual machine. Now, locate the VBoxGuestAdditions_x.x.x.iso that also matches the version of Virtualbox that you have installed. Replace the x in the filename with the specific version for your version of VirtualBox.

You can copy the URL to this file and paste it into the terminal window of your guest Ubuntu system that you had previously SSH'd into. Use wget to download the ISO:

wget http://download.virtualbox.org/virtualbox/x.x.x/
↪VBoxGuestAdditions_x.x.x.iso

Again, replace the x in the filename with the version that matches your VirtualBox installation.

You now should have an .iso file in your home directory.

After it's downloaded, mount it so that you can access the files contained in it by first creating a temporary directory in which to mount this .iso. Within your home directory, use this command:

mkdir -p tmp/vbox

Now, mount the .iso into that folder:

sudo mount -o loop VBoxGuestAdditions_x.x.x.iso tmp/vbox/

Within the .iso, you'll find a VBoxLinuxAdditions.run script. Copy this to your home directory:

cp VBoxLinuxAdditions.run ~

Now, make it executable by adding the execute flag:

chmod +x VBoxLinuxAdditions.run

And finally, run the installation script:

sudo ./VBoxLinuxAdditions.run

If you get a warning about failing to install the window system drivers, you safely can ignore it. The box you are installing these guest additions on is a server and doesn't have a window manager installed. You also can use the --nox11 flag like this when you install the updated guest additions:

sudo ./VBoxLinuxAdditions.run --nox11

The last step in this installation is to reboot the machine to ensure that the guest additions are up to date with:

sudo reboot

After the box restarts, log back in with:

vagrant ssh

Now you can unmount the .iso using sudo umount /mnt, if it isn't already, and delete the VBoxLinuxAdditions.run and the .iso file to save space. You also optionally can do some housekeeping by deleting the .bash_history file.

Now that the guest additions are updated, you should repackage a new box so the guest additions match each time you turn on this Vagrant box. This keeps you from needing to update them every time. As long as you don't change the version of VirtualBox that you are using, you won't need to update your guest additions. If you do, you'll need to update them again.

Using the vagrant package command, export your new box. This will shut down your machine and export a package.box file in the current directory.

Rename this to something more descriptive, like myprecise64.box.

Now add this new box to Vagrant:

vagrant box add myprecise64 myprecise64.box

The names you give your boxes are arbitrary and entirely up to you. Feel free to use a description that makes sense to you.

You're done with the original box now, so destroy it with vagrant destroy. You can remove it entirely from Vagrant with vagrant box remove precise64.

Add your new myprecise64 box, or whatever you happened to call it, like you did previously with the base box you downloaded with vagrant box add.

Make sure to change the name in your Vagrantfile to match the new box, and test it with vagrant up.

Congratulations! You now have an updated Ubuntu 12.04 VirtualBox suitable for Salt Stack and Salty-Vagrant automation.

The final piece of software you'll need to install is the Salty-Vagrant gem. It allows the automatic installation of Salt Stack on your guest virtual machines and uses Salt Stack to initiate your configuration and installation of software on them as well.

Back on your host machine (that is, the one that is not your virtual machine), use the Vagrant gem command vagrant gem install vagrant-salt to install the Salty-Vagrant gem.

Now you need to add a slightly customized Vagrantfile to configure some Salt-specific parameters, share your state tree and automatically install Drupal and the Linux Apache MySQL stack after your VirtualBox is started. If you added a Vagrantfile previously, you can delete it now. You're going to add a smaller, more customized one that is available from the Salty-Vagrant project page at https://github.com/saltstack/salty-vagrant. You simply can copy and paste this into a new Vagrantfile.

Locate the line config.vm.box = "precise64" and change "precise64" to the name of your box. If you've been following along, earlier in the tutorial, I exported my box as "myprecise64".

I also like to add guest/host network communications with this line to my Vagrantfile too:

config.vm.network :hostonly, "192.168.33.19"

You can use any available IP address for your internal VirtualBox network. Keep in mind, this is separate from your own network and is created by VirtualBox. This allows you to access your Drupal site from your own operating system's Web browser using the IP address that you specified. You can choose another IP address if you want.

In the same directory as your Vagrantfile, create a salt directory, srv directory and lamp-drupal directory like this:

mkdir -p salt/srv/lamp-drupal

Create a new minion.conf file in your salt directory that contains this line:

file_client: local

This instructs Salt not to search for a master server, but instead to use the local file structure for its configuration information. This is what you'll see referred to in the Salt Stack documentation as a masterless configuration.

Locate the line in your Vagrantfile labeled salt.minion_conf and uncomment it by removing the #. This tells Vagrant where your minion configuration is located. The path is relative to your Vagrantfile.

My Drupal SLS file is available in the salt-states repository (https://github.com/saltstack/salt-states/tree/master/small/lamp-drupal). You can use this to install all of the needed packages in Ubuntu for Drupal development. Download that file now and place it in your /srv/salt/lamp-drupal directory. You'll also need to create a /srv/salt/top.sls file that looks like this:

base:
   '*':
     - lamp-drupal

For more details about the top file, see my previously mentioned article about installing Salt Stack.

Within the salt/srv/lamp-drupal directory, copy the init.sls file from the salt-states repository.

This file instructs Salt that when the machine is booted to install all of the needed packages for running a Drupal Web server. I've even included a Drush installation. If you examine the file, you'll find that it is fairly understandable already. I won't go into much more detail here, but feel free to take a look at the init.sls file to see what actually is being installed.

Now edit your Vagrantfile to reflect the location of this state file by editing the line:

config.vm.share_folder "salt_file_root", "/srv", 
 ↪"/path/to/salt_file_root"

to:

config.vm.share_folder "salt_file_root", "/srv/salt", "salt/srv"

With your minion configuration file in place and your state file ready to go, this last step will start your machine automatically, install Salt and install all of the necessary components needed for a Drupal development server.

Ready? Use vagrant up to start your machine.

You can watch the output and responses as Vagrant, Salty-Vagrant and Salt Stack automatically download all of the software you specified and configure it for you.

After the server is started, simply SSH to your virtual server using vagrant ssh again. You'll find a /var/www folder ready to receive your Drupal installation.

This SLS file is reusable across multiple environments, and I haven't configured it to change the permissions of the /var/www folder. You'll notice this folder is owned by root, just like the Ubuntu package manager sets it.

The Drush directory located within the vagrant user's home directory is also owned by root. You'll get an error with Drush if you don't change the ownership of that folder to the user that is initiating the Drush command.

By combining four very powerful tools: Vagrant, Salty-Vagrant, VirtualBox and Salt, you can speed up your Drupal development considerably. Maintaining small development environments and then scripting their configuration helps create a win for everyone involved by adding consistency across all your developers' platforms. Maintaining small, text-based configuration files can save money on storage costs too. You and your developers also can save time transferring new configurations to remote servers.

You can try out new configurations and install additional libraries if you need or want to, without fear of breaking your local or production systems.

Each of the tools I have outlined are available for no cost and are open-source projects. You'll find the Salt development community to be one of the friendliest and most welcoming communities around the Open Source world.

Each project is updated on a regular basis, and it would be a good idea to review the documentation, as well as changes to the software, if you encounter any problems.