At the Forge

One of the watchwords for modern Web developers is scalability. Whether we're following the latest news about Twitter's servers or writing our own applications, developers always are thinking about whether their system will be scalable.

This issue has been particularly prominent during the spring and summer of 2008, as Ruby on Rails (my preferred platform for Web development) has been criticized for its use of RAM and its relatively slow execution speed. The massive server problems that Twitter experienced during the first half of 2008 were widely described as stemming from Twitter's use of Rails (despite denials from Twitter's technical team) and led to speculation that Rails cannot be used for a scalable application. One of the hosts of the weekly RailsEnvy podcast makes a point of sarcastically saying that “Rails doesn't scale” in each episode, because it was said so frequently.

There's no doubt that Rails is more resource-intensive than many other application development frameworks. This is partly due to the need for improvements in the Ruby language itself—improvements that look like they'll be available within the coming year. And, it also is true that the Rails framework uses more CPU and memory than some of its counterparts, such as Django, because of the nature of the features that it offers.

But, there's a difference, I believe, between calling Rails resource-intensive and calling it inherently unscalable. Scalability has more to do with the architecture and design of an application, allowing it to grow naturally from a single box containing both the Web and database servers to a network of servers. A Web application written in C might execute very quickly and, thus, handle a larger load on a single server, but that doesn't mean the application is inherently more scalable. At a certain point, even an efficient C program will reach its capacity, and if it isn't designed with this in mind, the more efficient application will be the less scalable one.

So, I tend to think about scalability as an architectural problem, one that ignores the specific programming language in which an application is implemented, and which is different from the issue of execution speed and efficiency. You can have highly scalable programs written in an inefficient framework, but it does take a bit more discipline and requires that programmers think carefully about the way they are writing the code. Even if you're starting on a single computer, designing the software in a scalable way allows you to distribute the load (and tasks) across a number of specialized servers.

One of the most important issues having to do with scalability actually has little or nothing to do with the Web application framework on which the program is written. Most modern Web applications use a relational database for persistent data storage, which means that the database server can be a bottleneck. Even if the database server isn't pushing its limits, the fact is that it takes time for a relational database to process a query, retrieve one or more appropriate rows and send them back to the querying application.

If your application is highly dynamic, it might use as many as a dozen SQL calls for each page, which will not only stress your database, but also significantly reduce the speed with which you can service each HTTP request. Longer request times mean that your users will be drumming their fingers longer and that your server will need more processes to handle the same number of requests.

One solution is to use multiple database servers. There are solutions for hooking together multiple servers from an open-source database (for example, PostgreSQL or MySQL), not to mention proprietary (and expensive) solutions for commercial databases, such as Oracle and MS-SQL. But, this is a tricky business, and many of the solutions involve what's known as master-slave replication, in which one database server (the master) is used for data modification, and the other (the slave) can be used for reading and retrieving information. This can help, but it isn't always the kind of solution you need.

But, there is another solution—one that is simple to understand and relatively easy to implement: memcached (pronounced “mem-cash-dee”). Memcached is an open-source, distributed storage system that acts as a hash table across a network. You can store virtually anything you like in memcached, as well as retrieve it quickly and easily. There are client libraries for numerous programming languages, so no matter what framework you enjoy using, there probably is a memcached solution for you.

This month, we take a quick look at memcached. When integrated into a Web application, it should help make that application more scalable—meaning it can handle a large number of users, spread across a large number of servers, without forcing you to rewrite large amounts of code. Version 2.1 of Ruby on Rails went so far as to integrate memcached support into the framework, making it even easier to use memcached in your applications.

As I mentioned previously, you can think of memcached as a network-accessible hash table. Like a hash table, it has keys and values, with a single value stored per key. Also like a hash table, there aren't a lot of ways to store and retrieve your data. You can set a key-value pair; you can retrieve a value based on a key, and you can delete a key.

This might seem like a limited set of functions. And, it is, if you think of memcached as your primary data store. But, that's exactly the point. Memcached never was designed to be a general-purpose database or to serve as the primary persistent storage mechanism for your application. Rather, it was meant to cache information that you already had retrieved from a relational database and that you probably were going to need to retrieve again in the near future.

In other words, memcached allows you to make your application more scalable, letting you take advantage of the fact that data is fetched repeatedly from the database, often by multiple users. By first querying memcached and accessing the database only when necessary, you reduce the load on your database and increase the effective speed of your Web application.

The main cost to you is the time involved in integrating memcached into your application, the RAM that you allocate to memcached and the server(s) that you dedicate to memcached. How many servers you will want to allocate to memcached depends, of course, on the size and scale of your Web site. You might need only one memcached server when you start out, but you might well need to expand to ten, 100 or even several hundred memcached servers (as I've heard Facebook uses) to maximize application speed and efficiency.

On my Ubuntu system, I was able to install memcached with:

apt-get install memcached

Then, I started memcached with:

/usr/bin/memcached -vv -u reuven

The -vv option turns on very verbose logging, allowing me to see precisely what is happening from the server's perspective. The -u flag lets me set the user under which memcached will run; it cannot be run as root, for security reasons.

Now, let's write a short client program to store and retrieve values. I'm going to write the client program in Ruby, although you can use almost any language (including Perl, Python or PHP) that you like. I used the memcache-client Ruby gem to connect to the memcached server, which I installed by typing:

sudo gem install memcache-client

Here is a short program that connects to the memcached server, stores a value and then retrieves a value:

#!/usr/bin/ruby

# Load necessary libraries
require 'rubygems'
require 'memcache'

# Create the memcached client
CACHE = MemCache.new 'localhost:11211'

# Set a value
CACHE.set('foo', 'bar')

# Retrieve a value
value = CACHE.get('foo')
puts "Value = '#{value}'"

As you can see, the first thing we do is create a client to the memcached server. You can specify one or more servers; in this case, we indicate that there is only one, running on localhost, on port 11211. It might surprise you to learn that although memcached is described as a distributed caching mechanism, the various memcached servers never speak to one another. Rather, it is the client that decides on which server it will store a particular piece of data, and it uses that same algorithm to determine which server should be queried to retrieve that data.

So in this program, we connect to our server, set a value (much as we would set it in a hash table) and then retrieve it. It's nothing very exciting, although the fact that the memcached server might be on another computer already makes things interesting.

Here is a slight variation on the previous program. Notice the third argument to CACHE.set, as well as the invocation of sleep afterward:

#!/usr/bin/ruby

require 'rubygems'
require 'memcache'

CACHE = MemCache.new 'localhost:11211'

CACHE.set('foo', 'bar', 3)

sleep 5

value = CACHE.get('foo')
puts "Value = '#{value}'"

This time, the output looks like this:

Value = ''

Huh? What happened to our value? Didn't we set it? Yes, we did, but we told memcached to expire the value after three seconds. This is one important way that memcached makes it easy to be integrated into a Web application. You can specify how long memcached should continue to see this data as valid. By passing no expiration time, memcached holds onto the value forever. Allowing the data to expire ensures that cached data is valid.

Just how long you should keep data in the cache is a question only you can answer, and it probably depends on the type of object you're storing. Orders from your on-line store probably should expire after a short period, because they likely will change as users visit your site. But, information about users is unlikely to change once they have registered, so it might make sense to hold onto that for a longer period of time.

It might seem strange for me to be describing memcached as a repository for complex objects, such as orders or people. And yet, memcached is fully able to handle such objects, assuming they are marshaled and unmarshaled by the client software. Thus, we can have the following short program:

#!/usr/bin/ruby

require 'rubygems'
require 'memcache'

CACHE = MemCache.new 'localhost:11211'

CACHE.set('foo', [:a, :b, 'c', [1,2,3],
        {:blah => 5, :blahblah => 10}, Time.now])

value = CACHE.get('foo')
puts "Value = '#{value.map{ |i| i.class}.join(', ')}'"

Sure enough, we see that memcached is happy both to set and retrieve values of a variety of classes. This means that even if we create a complex class, we can store it in memcached and retrieve it later.

Memcached is an important part of nearly any Web application's strategy for scaling. It can reduce the time it takes to access certain types of information dramatically, resulting in faster response times for users and freeing up the relational database server for other jobs. Deciding exactly which objects can and should be stored in memcached and determining how long they should be kept in the cache before expiring are issues that must be addressed for each individual application.

Next month, I'll explain how memcached support has been integrated into Ruby on Rails, making it quite easy to take advantage of this technology in your own applications—and, dare I say it, help your applications become truly scalable.