Setting up software RAID on Debian with mdadm

Software RAID has come a long way. Unless you have some very high-rate, high-volume, I/O workloads with SLAs that will otherwise cost you money, for the most part a software RAID will perform just fine. One of the primary benefits of using software RAID is the portability of your disks/data. If a box on which you have a software RAID dies somehow and at least some (depending on your RAID configuration) of drives survive, you can easily resurrect the RAID on another machine without having a duplicate RAID card on hand.

Following is an example of how to setup a RAID 1 array using mdadm on Debian. I have two 2TB drives, /dev/sdb and /dev/sdc.

Install mdadm

apt-get install mdadm

Create the array

mdadm --create --verbose /dev/md0 --level=1 --raid-devices=2 /dev/sdb /dev/sdc

Once you create the RAID device, you can check on the mirroring process (Not sure what it is syncing with a blank disk…)

# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sdc[1] sdb[0]
1953382464 blocks super 1.2 [2/2] [UU]
[>....................] resync = 1.0% (20516736/1953382464) finish=190.1min speed=169401K/sec
bitmap: 15/15 pages [60KB], 65536KB chunk

Create a filesystem

mkfs.xfs /dev/md0

Create a mount point and mount the filesystem

mkdir /data
mount /dev/md0 /data

Ensure that the array is reassembled on boot by adding it to /etc/mdadm/mdadm.conf. We do this by scanning the active array and appending it’s details to the config file.

mdadm --detail --scan /dev/md0 >> /etc/mdadm/mdadm.conf

Upate the initramfs so that the array will be available on boot

update-initramfs -u

Add an entry to /etc/fstab

/dev/md0 /data               xfs     defaults,nofail,discard  0 0

Do a test reboot and make sure that it is mounted and the data accessible.

Replacing a failed disk

Check the array status and identify the failed disk

cat /proc/mdstat

It will output something similar to the following

Personalities : [raid6] [raid5] [raid4] [linear] [multipath] [raid0] [raid1] [raid10] 
md0 : active raid6 sdg[5] sdd[4] sda[0] sdc[3](F) sdb[1] sde[2]
      9376331776 blocks super 1.2 level 6, 512k chunk, algorithm 2 [6/5] [UUU_UU]
      bitmap: 18/18 pages [72KB], 65536KB chunk

In this case the sdc device has failed as denoted by the (F).

Mark the disk as failed (if mdadm hasn’t done so automatically)

mdadm --manage /dev/md0 --fail /dev/sdc

Remove the disk from the mdadm configuration

mdadm --manage /dev/md0 --remove /dev/sdc

If either of those commands fail with No such file or directory it means that there was a total drive failure or the device is no longer connected to the hardware bus.

In that case, use the detached keyword to scrub it from the array

mdadm --manage /dev/md0 --remove detached

Re-running the mdstat command will show the device no longer in the list for the array.

If you system supports it, you can remove the failed drive from the chassis and replace it. Otherwise, you will have to power down the system.

Find the new drive’s identifier

Because Linux assigns drive letters dynamically, the new disk might not be named /dev/sdc. Run the following command to find the unpartitioned new drive

lsblk

Look for a drive that matches the size of your other RAID disks and does not have any mount points or RAID layers (md0) listed under it.

Wipe the existing partition table if there are existing partitions

This command instantly destroys the partition data on sdc so mdadm views it as a clean, blank slate

wipefs -a /dev/sdc

Add the raw drive to the array

Now that the disk is completely blank, add it to your RAID6 array:

mdadm --manage /dev/md0 --add /dev/sdc

Verify the rebuild has started

Check the status to ensure the array has accepted the drive and is currently rebuilding. The following two commands show two different views of the array

cat /proc/mdstat
mdadm --detail /dev/md0

You should see sdc[6] (or a similar number) added to the array string, and a progress bar showing the rebuild percentage and estimated time remaining.

Leave a Reply