Storage needs always seem to grow, but when you are ready to upgrade to a new hard drive, how do you transfer all those files? Read on for a tried-and-true method to migrate all your old files to a new drive.
In another article in this issue of Linux Journal [page 84], I talk about my experiences with the new solid state drive (SSD) I installed on my laptop. One of the things I didn't mention in the article was how I transferred all my data and settings to the new drive. There are a number of ways to solve this problem. For instance, you could image the old drive onto the new one and then grow the last partition to fill up the presumably larger disk (which wouldn't work for me, as my new SSD actually was substantially smaller than the old drive). Other people just re-install their OS every time they get a new drive and then transfer their /home directory and other settings, but I've always had just enough custom programs and settings on my laptop for that method to be a pain. You also could use rsync with certain flags to migrate the files, and although I do like that method for network transfers, for local transfers, it can be a hassle, because it first must scan through the entire drive before it begins.
I've done many hard drive migrations during the years with a tried-and-true combination of find piped to cpio. I like this method because it uses common tools that are sure to be installed, it starts immediately and doesn't need to scan the drive, and with the right flags, it correctly can handle (and avoid) special filesystems, such as /proc, /sys and so on. So far, it hasn't failed me, and this migration was no exception. However, this time, I did run into a few gotchas that I will talk about shortly. First, onto the basic steps.
You don't want files to be changed as you are copying them, so you don't want to do this migration from your normal desktop environment. Typically, I boot in to a rescue disk like Knoppix, so that the filesystem stays frozen. Other times, I simply switch to single-user mode, so most files won't change. For desktop systems, I generally just connect both drives directly to the system, and for laptops, I use a USB hard drive adapter, so that both can be connected at the same time. For my last migration, I didn't happen to have a USB adapter for a 1.8" drive, so I transferred the data to an intermediate drive first, then installed the new drive and transferred again.
You can use any partitioning tool that works for you—from fdisk to qtparted. This may sound obvious, but make sure that you allocate plenty of room to fit your existing data, and if you move to a larger hard drive, plenty of room to grow. Once you partition the drive, use mkfs or your preferred formatting tool to write a filesystem to each partition (or mkswap for the swap partition).
Create mountpoints under /mnt for the new partitions you have created. For my example, I have a root partition at /dev/sdb1 and a home partition at /dev/sdb3, so I would type as root:
mkdir /mnt/sdb1 mkdir /mnt/sdb3 mount /dev/sdb1 /mnt/sdb1 mount /dev/sdb3 /mnt/sdb3
If you run this from a rescue disk, you also need to make sure your source partitions are mounted as well.
Now this spell doesn't have a lot to it, but it's funny how you memorize scripts like this over the years after using them and passing them along to friends. First, change to the root level of the partition you want to copy and then execute the command as root. So, to migrate my root partition from single-user mode, I did the following:
cd / find ./ -xdev -print0 | cpio -pa0V /mnt/sdb1
To migrate from a rescue disk, the command is almost identical, but you change to the mountpoint of the source partition instead (I mounted it at /dev/sda1):
cd /mnt/sda1 find ./ -xdev -print0 | cpio -pa0V /mnt/sdb1
The find command searches through the entire root partition for files and directories. The -xdev flag tells find to stay within the current mounted filesystem. Otherwise, when find gets to /home, it would copy the contents of that directory as well and potentially fill up the new partition. It then passes the files to cpio, which places them under my new mountpoint while preserving permissions, symlinks and other settings. The cpio command also outputs one dot for each file it copies, so you can have some sense of its progress. However, what I typically do is go to another terminal and monitor the output of df so I can watch it grow:
Once the first find | cpio command completes, repeat it for each of your other partitions. In my example, if I were in single-user mode, I'd do the following:
cd /home find ./ -xdev -print0 | cpio -pa0V /mnt/sdb3
If I were using a rescue disk, I'd do this:
cd /mnt/sda3 find ./ -xdev -print0 | cpio -pa0V /mnt/sdb3
What you do during this step will vary a bit, depending on how you set up your partitions. If you moved your partition layout around, you need to edit the /etc/fstab file on your new root partition so that it reflects the new drives you have set up.
Traditionally, this has been a simple step for me, because I try to order the partitions the same and generally don't have to touch fstab, but on this last migration, I had to add an extra step due to Ubuntu's use of UUIDs to reference partitions. A lot of modern distributions don't refer to partitions by their device name. Instead, a unique identifier called the UUID is assigned to each partition. If you see UUID=longstringofhex in your /etc/fstab, this means you.
Now, you have two choices here. The first choice is to change all these UUID lines to reference the actual device. This will work, and is less prone to typos that will make the system not boot, but you will lose the advantage of UUIDs. The other method is to reference the UUIDs for your new partitions and put them in place of the old UUIDs. You can find the list of disk-to-UUID mappings under /dev/disk/by-uuid:
greenfly@minimus:~$ ls -l /dev/disk/by-uuid/ total 0 lrwxrwxrwx 1 root root 10 2008-04-06 16:00 ↪634719fd-a6da-4fee-8646-0d485d7681db -> ../../sda2 lrwxrwxrwx 1 root root 10 2008-04-06 16:00 ↪665d7008-fde9-4055-8af9-483697acb005 -> ../../sda1 lrwxrwxrwx 1 root root 10 2008-04-06 16:00 ↪cf3892fd-e3d8-446f-8552-4c633be9c382 -> ../../sda3
Of course, you always could choose a hybrid of the two approaches, and set the hard device names in the fstab for the first boot, and then once you have confirmed the system boots, you then can update fstab with UUIDs.
As with fstab, if you changed your partition layout, you need to update your GRUB configuration under /boot/grub/menu.lst (or on some systems, in /boot/grub/grub.conf) to reflect your changes. Also, GRUB can reference drives by UUID, so if you see references to UUID in the GRUB configuration file, be sure to update it to reflect the new values. Once the file has been updated, chroot into your new root partition's mountpoint and then run grub-install:
chroot /mnt/sdb1 /usr/sbin/grub-install --recheck /dev/sdb
Change /mnt/sdb1 and /dev/sdb to reflect your new mounted root partition and its disk device, respectively. If the chrooted grub-install doesn't work, you typically can use your rescue disk (or single user) grub-install with the --root-directory option:
/usr/sbin/grub-install --recheck --root-directory /mnt/sdb1 /dev/sdb
After I used my new system for some time, I noticed it wouldn't resume correctly from hibernation. It seemed like each time the swap partition would get corrupted. After some troubleshooting, I found that the root cause was a hard-coded resume device based on UUID that is put in the initial ramdisk for the machine. You may or may not run into this issue, depending on your Linux distribution, as each distribution manages its initrd differently. But, here is the fix for my Ubuntu system. I was able to find the offending reference to the old UUID in /etc/initramfs-tools/conf.d/resume. All I needed to do was update that file on the new drive to point to the new UUID for my swap partition, then run update-initramfs from the new system, and reboot.