Work the Shell

Holy cow, have we really been working on the movie trivia Twitter stream for almost a year now? This surely must be the longest time-per-line-of-code project in the history of software development.

Previous columns have combined to give us a set of shell scripts that scrape the Internet Movie Database (IMDb, www.imbd.com) for its top 250 list, then pull out the year of release for each film (inconveniently, not available on the same page as the top 250 list), randomly select reasonable incorrect guesses for release year and output the question in the form: “IMDB Top 250 Movie #81: Was 2001: A Space Odyssey released in 1968, 1973 or 1975?”

That's rather more work behind the scenes than you might realize. Of course, if you've been reading all the previous columns, you actually have firsthand knowledge of just how many hoops we've had to jump through (but fortunately, no “hogsheads of real fire”).

This month, we're going to turn our attention to the Twitter microblogging service, as we finally have the ability to produce the desired message. Now, we just need a mechanism to publish it to Twitter.

I could take the long way and actually read through the Twitter API to learn how to make specific calls and interact with the service elegantly and appropriately, but that sounds like work, doesn't it?

It turns out, there's a simple way to invoke Twitter through a command-line Web utility and get access to much of the basic data. My tool of choice? Curl, a slick utility that makes it easier to work with Internet services through the command line.

The URL is twitter.com/statuses/update.json, but the way you pass data is a bit tricky. You need to send it as a name=value pair in the connection stream, not as a GET value or other URL-based technique. The variable it seeks is status, and you can test it by doing this:

$ curl --data-ascii status=testing \
http://twitter.com/statuses/update.json

The problem is immediately obvious in the return message:

Could not authenticate you.

Ah, well, yes, we haven't specified our user ID. Those credentials aren't sent via the URL or --data-ascii, but instead through a basic HTTP auth. So, in fact, you need to do this:

$ curl --user "$user:$pass" --data-ascii \
status=testing "http://twitter.com/statuses/update.json" 

Now, of course, you need a Twitter account to utilize. For this project, I'm going to continue working with FilmBuzz, a Twitter account that I also set up to disseminate interesting film-industry news.

For the purposes of this article, let's assume the password is DeMille, though, of course, that's not really what it is. The user and pass variables can be set, and then we invoke Curl to see what happens:

$ curl --user "$user:$pass" --data-ascii status=testing
http://twitter.com/statuses/update.json
{"truncated":false,"in_reply_to_status_id":null,"text":
↪"testing","favorited":null,"in_reply_to_user_id":
↪null,"source":"web","id":880576363,"user":{"name":
↪"FilmBuzz","followers_count":214,"url":null,
↪"profile_image_url":"http:\/\/s3.amazonaws.com\
↪/twitter_production\/profile_images\/55368346\
↪/FilmReelCloseUp_normal.JPG","description":"Film
↪trivia game, coming soon!","location":"Hollywood, of
↪course","screen_name":"FilmBuzz","id":15097176,"protected":
↪false},"created_at":"Thu Aug 07 16:51:49 +0000 2008"}

The return data gives you a nice sneak peek into the configuration of the FilmBuzz twitter account and tells you that what we sent wasn't too long (truncated:false), but otherwise, it's pretty forgettable stuff, isn't it?

How do you ignore output with a command in the shell? Reroute the output to /dev/null, of course:

$ curl --user "$user:$pass" --data-ascii status=testing
http://twitter.com/ses/update.json > /dev/null
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 505 100 505 0 0 1127 0 --:--:-- --:--:-- --:--:-- 0

Nope, that doesn't work, because then Curl wants to give us some transactional stats. How do we mask that? Use the --silent flag to Curl. Now, we're just about there:

$ curl --silent --user "$user:$pass" --data-ascii status=testing
http://twitter.com/statuses/update.json > /dev/null
$

That's it. Let's put that into a shell script, so we can simply invoke tweet with the message we want to send out to followers of FilmBuzz. (And, if you aren't yet following FilmBuzz on Twitter, why the heck not? Go to twitter.com/FilmBuzz, and click follow.) Actually, we need to take into account one more thing: although our messages on the command line are quite likely to have spaces within them, we can't send a name=value pair with spaces. Instead, a quick invocation of tr lets us convert all spaces into + signs, the commonly accepted HTTP encoding:

#!/bin/sh
# Twitter command line interface
user="FilmBuzz" ; pass="DeMille"
curl="/usr/local/bin/curl"

$curl --silent --user "$user:$pass" --data-ascii \
"status=$(echo $@ | tr ' ' '+')" "http://twitter.com/
↪statuses/update.json"
> /dev/null
exit 0

This isn't a particularly robust solution, because what happens if we have a + character in the message we want to transmit? It gets lost. That can be addressed by first checking to see whether there are + signs and converting them to the safe HTTP-encoding equivalent, %2B. You can't do that with tr, however, because it wants 1:1 substitution, so we'll use sed instead and pull the substitution onto its own line for better style too:

msg=$(echo $@ | sed 's/+/%2B/g;s/ /+/g')

$curl --silent --user "$user:$pass" --data-ascii \
"status=$msg" "http://twitter.com/statuses/update.json" > /dev/null

Problem solved. Now you can do simple things like:

$ tweet 'Tweeting from the command line? Well, sure!'

Nice. Now, to put everything together. Oh. We've run out of space. Again. Okay, next month. We finally have all the building blocks we need. Remember, sign up for Twitter, and then follow @FilmBuzz, so you can see the fruit of this work.