Richard's Blog Life Lessons from The Desk


3TB HDD + RAID5 + LVM on CentOS 6.2

So I was asked to create a backup solution for someone in the office who has massive amounts of film footage to store away for a long time. As we don't want to save this to DVD/Blu-ray (think of the cost!) HDD seemed to be the only way. Problem is we don't want to save the files to disk then disconnect them (could have a few issues when we go to start them up again). So we went for a server (not to powerful) that could have the highest sized disks I could find running in a RAID5. I decided to write this blog post as I could find bits to tell me how to do each stage so I though I would put it all together in one nice page. Enjoy!

So to start with I will explain my drive layout. Since I have had issues in the past with Grub and disks over 1TB I decided to install my OS (CentOS 6.2 in this case) onto a separate hard drive. This drive is a 250GB disk which I shall put LVM onto to make it a little easier if I need to extend my storage options later. The other 3 disks I am going to use are new 3TB Seagate drives.

Everything was going well until I used anacondas installed to make the RAID5. As I found out anaconda uses fdisk to create the partitions. Unfortunatly fdisk cannot make partitions over 2TB. So to fix this I made my default system install and hit the command line after to create the RAID5. Thanks to this article, I found a way to create the partitions. There is more details in the article but I shall go over the basics here.

Firstly install parted (I installed a minimal install so this was not installed by default). To do this run:

yum install parted

Once installed we are ready to start!. firstly start parted with the first disk you need to partition. In my case I need to partition sdb, sdc and sdd.

parted /dev/sdb

First we need to create a new GPT disklabel which can be done by running:

(parted) mklabel gpt

You will get a warning about this removing all existing disk labels enter 'yes' (if you are sure). We can now set the default unit to be terabytes to make things easy:

(parted) unit TB

Now onto creating the partition. To do this simply enter:

(parted) mkpart primary 0.00TB 3.00TB

Running the following command will print out the partition table and allow you to confirm that you have created the partition:

(parted) print

Now repeat the above steps till all your disks are done. Now onto creating the RAID!  Firstly you might need to install the raid tools to do this run:

yum install mdadm

Once this is installed you simply run the following command to create the RAID:

mdadm --create /dev/md0 --level=raid5 --raid-devices=3 /dev/sdb1 /dev/sdc1 /dev/sdd1

This then creates the RAID which can be confirmed by viewing the files /proc/mdstat

cat /proc/mdstat

After creating the RAID we then need to put our file system onto the partition. This can be done either by putting a file system like ext4 straight onto the partition or by putting LVM onto it first. I wanted to put LVM onto it first as this means I can extend it if I need to.

Firstly we need to create the physical volume. To do this we run the following command:

pvcreate /dev/md0

We can confirm that this has been created by running pvdisplay. Now that we have our physical volume we can now create our volume group. This is what we use to create our logical volumes (yup, there are a lot of levels to LVM!). To create our volume group called datastore we run the following:

vgcreate datastore /dev/md0

Again this can be confirmed by running vgdisplay. Finally we need to create our logical volume. Now I am going to use all the space available on my volume group for this. To do this simply run:

lvcreate --name datastore -l 100%FREE datastore

Now as you may have noticed I have called my logical volume the same as my volume group. The last datastore is the volume group that I want this space to be taken from. Finally we are onto creating the file system  that linux will mount. The final command to create this file system is:

mkfs.ex4 /dev/datastore/datastore

This will take some time to complete. But once this is done you can then mount your file system using the mount command. Also if you want to have it mount automatically on boot you can put something like this into your /etc/fstab:

/dev/mapper/datastore-datastore /var/data ext4 defaults 1 1

Hopefully this has been useful to you. If so leave a comment!