LJ Archive

Work the Shell: Shuffling Letters and Words

You can shuffle your feet and you can shuffle cards, but can you shuffle characters? Dave's latest column explores the possibilities. By Dave Taylor

My last few articles have described building a pretty sophisticated password generator, except for one thing: I never quite got to the point of scrambling the end result to add a second level of randomness. I sidestepped the issue by saying it was an exercise for the reader, but in fact, it's a pretty interesting problem, so let's look at it here.

You can reverse a word with the handy Linux command rev, like so:


$ echo "hello from the other side" | rev
edis rehto eht morf olleh

You also can reverse lines in a file so that the last line is shown first, penultimate line second, and so on:


$ cat -n test.me | sort -rn | cut -f2-
entering along with him. 
enough to prevent a swirl of gritty dust from 
glass doors of Victory Mansions, though not quickly 
escape the vile wind, slipped quickly through the 
chin nuzzled into his breast in an effort to 
clocks were striking thirteen. Winston Smith, his 
It was a bright cold day in April, and the 

You recognize that opening paragraph even though it's backwards, right? "Clocks were striking thirteen" can only be George Orwell's cautionary tale 1984.

Note: there's a Linux command called tac that offers a reverse cat, which would do the job too, but I've always loved sort -rn as a command, so I wanted to demonstrate how to use it in a pipeline to accomplish the same result.

How about getting the lines of this file, but in completely random order? There's a command for that—at least in Linux: shuf. It's not available on the Mac OS X command line, however, so if you're playing along at home with your Mac system, well, you've just hit a road block. Sorry about that. I offer an alternative at the end of this article though, so don't despair!

If you're on a Linux system (and this is Linux Journal after all), then check this out:


$ cat test.me | shuf
clocks were striking thirteen. Winston Smith, his 
entering along with him. 
glass doors of Victory Mansions, though not quickly 
escape the vile wind, slipped quickly through the 
enough to prevent a swirl of gritty dust from 
chin nuzzled into his breast in an effort to 
It was a bright cold day in April, and the 

So those commands are all ready to go, but how about scrambling letters in a line? That can be done with the shuf command as demonstrated previously, but individual lines aren't quite ready for the shuf treatment.

You can break up words by using the under-appreciated fold command, like this:


$ echo "Winston" | fold -w1
W
i
n
s
t
o
n

Now that the word is broken down letter by letter into lines, the shuf command is ready to take on the task and randomize the lines (letters) as needed:


$ echo "Winston" | fold -w1 | shuf
n
t
i
n
o
W
s

Perfect. Now, there's one last step: stitching it all back together so it's a word rather a bunch of really short lines. Surprisingly perhaps, the tr command is up for the task, because all you really need to do is get rid of the newline character at the end of each line. Recall that the -d flag to tr instructs it to delete any occurrence of the specified letter. So, here's the total command to shuffle the letters of a word:


$ echo "Winston" | fold -w1 | shuf | tr -d '\n'
itoWnns$

Why is that $ prompt tacked onto the shuffled word? Because the tr invocation removes all newlines, even the one that otherwise would terminate the final line. There are a couple ways around this, but the easiest is to do something rather script-like. In fact, let's make this a tiny script:


$ cat scramble.sh 
#!/bin/sh

# scramble - scramble whatever's specified on command line
#      usage: scramble word or phrase

echo "$*" | fold -w1 | shuf | tr -d '\n'
echo ""
exit 0

I also could have used some intermediate variables, but as you can see, it's unnecessary in this case. It's easy, really, and now you can get some interesting results:


$ sh scramble.sh Winston Smith
Shnoi tWstmin

Where this becomes particularly interesting is if you invoke it more than once with the same input. It should be different each time, correct? Turns out, it is:


$ sh scramble.sh Winston Smith
nnih ttiWmoSs
$ sh scramble.sh Winston Smith
mnnWiosthS it
$ sh scramble.sh Winston Smith
nsmniott WhiS

That's exactly what was needed for the password generator, so now the script finally is done and ready to go with the addition of this simple script.

And, If You Don't Have shuf

So, what about those of you that aren't running Linux but are using another *nix? There are a couple ways you can get the shuf command or its equivalent, one of which is to install the entire GNU coreutils package. It turns out that Python also can duplicate the basic functionality with a single line. Yes, a single line:


python -c 'import sys, random; L = sys.stdin.readlines(); 
 ↪random.shuffle(L); print "".join(L),'

Now, I don't know anything at all about Python, so I can't explain what's going on, but it's easy to verify that this does indeed work:


$ cat test.me | python -c 'import sys, random; L = 
 ↪sys.stdin.readlines(); random.shuffle(L); print "".join(L),'
entering along with him. 
clocks were striking thirteen. Winston Smith, his 
escape the vile wind, slipped quickly through the 
It was a bright cold day in April, and the 
enough to prevent a swirl of gritty dust from 
chin nuzzled into his breast in an effort to 
glass doors of Victory Mansions, though not quickly 

Nice. Props to Cristian Ciupitu on superuser.com for this code snippet that I'm republishing here too.

And, now with all this randomness and shuffle alternatives, you might like to delve into the question of how random is random? Or maybe not. It turns out that's quite a sticky wicket and a rich area of computer science research.

About the Author

Dave Taylor has been hacking shell scripts on UNIX and Linux systems for a really long time. He's the author of Learning Unix for Mac OS X and Wicked Cool Shell Scripts. You can find him on Twitter as @DaveTaylor, and you can reach him through his tech Q&A site: Ask Dave Taylor.

Dave Taylor
LJ Archive