Disk Utility Fails to Rebuild RAID Mirror

June 9, 2009

disk_utility_icon

I recently had a drive fail in a client’s RAID setup. 3 mirrored drives in a software RAID0 (mirrored) configuration. No big deal, you say! In a 3 drive configuration like that, you can swap the faulty drive without even losing any redundancy! Simple!

Right.

Physically replacing the drive went as planned, dead easy. But the rebuild failed contiuously. It should take hours (they were terabyte drives) but instead would stop after a few minutes, showing the replacement drive status as “Failed”. RAID Status obviously remained “Degraded”.

A bit of digging around on the web turned up this Apple knowledgebase article on rebuilding software RAID mirrors, which pointed out two VERY usefully pieces of information:

  1. You should use the command-line diskutil for rebuilding a RAID. Sometimes Disk Utility will be unable to successfully
    rebuild a degraded RAID mirror.

    • (Not an issue, this is standard practice anyways.)
  2. You should not rebuild the rebuild a mirror while it is the boot volume. Rebuilding a RAID Mirror will sometimes fail if it is the boot volume.
    • (Major issue, as it defeats much of the benefits of running a RAID in the first place! Sure, the data is safe – VERY important – but now I have to take a mail server offline for 12hours while the RAID rebuilds?!?!?)

The successful rebuild of the mirror involved both the above steps. The mail server was booted to an external drive and diskutil was used to rebuild the mirror. Before doing anything, however, I created a backup image file of the server (just to be safe…). The server was offline for approximately 12 hours, but it might have needed less – once I was confident that the rebuild was progressing successfully, I tried to enjoy the rest of my Saturday.

The moral of this story is obviously that software RAID solutions are in no way a substitute of hardware solutions. If you need guaranteed uptime, you need to invest – but we all knew that already… right?

For more information on rebuilding a software RAID in OS X, read Apple’s knowledge base article: How to rebuild a software RAID mirror.

Advertisements

One Response to “Disk Utility Fails to Rebuild RAID Mirror”

  1. Joel Carter said

    You’ve hit the nail on the head with “If you need guaranteed uptime, you need to invest”. The hot-swap SATA/SAS stuff is the way to go! No need to shut the machine down.

    RAID 6 is interesting as well, it’s a reliability increase from RAID 5 so that you’re not compromising the data should another drive fail during the array rebuild.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: