Reference: https://www.tecmint.com/create-raid-5-in-linux/
Install mdadm and verify drives
yum install mdadm
mdadm -E /dev/sd[b-h]
mdadm: No md superblock detected on /dev/sdc.
mdadm: No md superblock detected on /dev/sdb.
mdadm: No md superblock detected on /dev/sdd.
mdadm: No md superblock detected on /dev/sde.
mdadm: No md superblock detected on /dev/sdf.
mdadm: No md superblock detected on /dev/sdg.
mdadm: No md superblock detected on /dev/sdh.
Partitioning the Disks for RAID
Actually this is optional. If you want to partition first, you should make sure you’re using gpt or others which support larger than 2T.
Create /dev/sdb Partition
Please follow the below instructions to create partition on /dev/sdb drive.
fdisk /dev/sdb
- Press ‘n‘ for creating new partition.
- Then choose ‘P‘ for Primary partition. Here we are choosing Primary because there is no partitions defined yet.
- Then choose ‘1‘ to be the first partition. By default it will be 1.
- Here for cylinder size we don’t have to choose the specified size because we need the whole partition for RAID so just Press Enter two times to choose the default full size.
- Next press ‘p‘ to print the created partition.
- Change the Type, If we need to know the every available types Press ‘L‘.
- Here, we are selecting ‘fd‘ as my type is RAID.
- Next press ‘p‘ to print the defined partition.
- Then again use ‘p‘ to print the changes what we have made.
- Use ‘w‘ to write the changes.
After repeating these for all four disks, check for changes in all four drives sdb, sdc, sdd & sde.
mdadm -E /dev/sd[b-e]
/dev/sdb:
MBR Magic : aa55
Partition[0] : 4294965247 sectors at 2048 (type fd)
/dev/sdc:
MBR Magic : aa55
Partition[0] : 4294965247 sectors at 2048 (type fd)
/dev/sdd:
MBR Magic : aa55
Partition[0] : 4294965247 sectors at 2048 (type fd)
/dev/sde:
MBR Magic : aa55
Partition[0] : 4294965247 sectors at 2048 (type fd)
Now Check for the RAID blocks in newly created partitions. If no super-blocks detected, than we can move forward to create a new RAID 5 setup on these drives.
mdadm --examine /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1
mdadm: No md superblock detected on /dev/sdb1.
mdadm: No md superblock detected on /dev/sdc1.
mdadm: No md superblock detected on /dev/sdd1.
mdadm: No md superblock detected on /dev/sde1.
Creating md device md0
mdadm -C /dev/md0 -l 5 -n 4 /dev/sd[b-h]1
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
After creating raid device, check and verify the RAID, devices included and RAID Level from the mdstat output.
cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sdh[7] sdg[5] sdf[4] sde[3] sdd[2] sdc[1] sdb[0]
46880191488 blocks super 1.2 level 5, 512k chunk, algorithm 2 [7/7] [UUUUUUU]
unused devices: <none>
Watch process
watch -n1 cat /proc/mdstat refresh screen every 1 second.
Verify
After creation of raid, Verify the raid devices using the following command.
mdadm -E /dev/sd[b-h]
/dev/sdb:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 7f86552b:c499fe55:d4be9502:c329d39c
Name : node7:0 (local to host node7)
Creation Time : Sat Sep 8 09:06:24 2018
Raid Level : raid5
Raid Devices : 7
Avail Dev Size : 15627791024 (7451.91 GiB 8001.43 GB)
Array Size : 46880191488 (44708.44 GiB 48005.32 GB)
Used Dev Size : 15626730496 (7451.41 GiB 8000.89 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
State : clean
Device UUID : 61344c1d:281610e6:ce609f08:fd7d9ea7
Update Time : Sun Sep 9 05:24:11 2018
Checksum : e100b486 - correct
Events : 18
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 0
Array State : AAAAAAA ('A' == active, '.' == missing)
/dev/sdc:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 7f86552b:c499fe55:d4be9502:c329d39c
Name : node7:0 (local to host node7)
Creation Time : Sat Sep 8 09:06:24 2018
Raid Level : raid5
Raid Devices : 7
Avail Dev Size : 15627791024 (7451.91 GiB 8001.43 GB)
Array Size : 46880191488 (44708.44 GiB 48005.32 GB)
Used Dev Size : 15626730496 (7451.41 GiB 8000.89 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
State : clean
Device UUID : 4daf07e5:2f98d4e5:36b6870e:20eedf68
Update Time : Sun Sep 9 05:24:11 2018
Checksum : 6faa7706 - correct
Events : 18
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 1
Array State : AAAAAAA ('A' == active, '.' == missing)
/dev/sdd:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 7f86552b:c499fe55:d4be9502:c329d39c
Name : node7:0 (local to host node7)
Creation Time : Sat Sep 8 09:06:24 2018
Raid Level : raid5
Raid Devices : 7
Avail Dev Size : 15627791024 (7451.91 GiB 8001.43 GB)
Array Size : 46880191488 (44708.44 GiB 48005.32 GB)
Used Dev Size : 15626730496 (7451.41 GiB 8000.89 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
State : clean
Device UUID : b86dfb24:06c99d6a:12798289:9da23e0f
Update Time : Sun Sep 9 05:24:11 2018
Checksum : 55c0dda1 - correct
Events : 18
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 2
Array State : AAAAAAA ('A' == active, '.' == missing)
/dev/sde:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 7f86552b:c499fe55:d4be9502:c329d39c
Name : node7:0 (local to host node7)
Creation Time : Sat Sep 8 09:06:24 2018
Raid Level : raid5
Raid Devices : 7
Avail Dev Size : 15626731520 (7451.41 GiB 8000.89 GB)
Array Size : 46880191488 (44708.44 GiB 48005.32 GB)
Used Dev Size : 15626730496 (7451.41 GiB 8000.89 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
State : clean
Device UUID : 54a9a755:6b0d7954:9dbeb452:872b894e
Update Time : Sun Sep 9 05:24:11 2018
Checksum : 78b50168 - correct
Events : 18
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 3
Array State : AAAAAAA ('A' == active, '.' == missing)
/dev/sdf:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 7f86552b:c499fe55:d4be9502:c329d39c
Name : node7:0 (local to host node7)
Creation Time : Sat Sep 8 09:06:24 2018
Raid Level : raid5
Raid Devices : 7
Avail Dev Size : 15626731520 (7451.41 GiB 8000.89 GB)
Array Size : 46880191488 (44708.44 GiB 48005.32 GB)
Used Dev Size : 15626730496 (7451.41 GiB 8000.89 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
State : clean
Device UUID : fb8c7bd7:8e42bb21:0d402531:6db963d4
Update Time : Sun Sep 9 05:24:11 2018
Checksum : 2c16298a - correct
Events : 18
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 4
Array State : AAAAAAA ('A' == active, '.' == missing)
/dev/sdg:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 7f86552b:c499fe55:d4be9502:c329d39c
Name : node7:0 (local to host node7)
Creation Time : Sat Sep 8 09:06:24 2018
Raid Level : raid5
Raid Devices : 7
Avail Dev Size : 15626731520 (7451.41 GiB 8000.89 GB)
Array Size : 46880191488 (44708.44 GiB 48005.32 GB)
Used Dev Size : 15626730496 (7451.41 GiB 8000.89 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
State : clean
Device UUID : 241d579a:b71126c8:0e70c185:024f7ae3
Update Time : Sun Sep 9 05:24:11 2018
Checksum : f90f4e73 - correct
Events : 18
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 5
Array State : AAAAAAA ('A' == active, '.' == missing)
/dev/sdh:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 7f86552b:c499fe55:d4be9502:c329d39c
Name : node7:0 (local to host node7)
Creation Time : Sat Sep 8 09:06:24 2018
Raid Level : raid5
Raid Devices : 7
Avail Dev Size : 15626731520 (7451.41 GiB 8000.89 GB)
Array Size : 46880191488 (44708.44 GiB 48005.32 GB)
Used Dev Size : 15626730496 (7451.41 GiB 8000.89 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
State : clean
Device UUID : bd88463c:51eb855a:ed8d4bab:58e25197
Update Time : Sun Sep 9 05:24:11 2018
Checksum : 6c044dd - correct
Events : 18
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 6
Array State : AAAAAAA ('A' == active, '.' == missing)
Next, verify the RAID array to assume that the devices which we’ve included in the RAID level are running and started to re-sync.
mdadm --detail /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Sat Sep 8 09:06:24 2018
Raid Level : raid5
Array Size : 46880191488 (44708.44 GiB 48005.32 GB)
Used Dev Size : 7813365248 (7451.41 GiB 8000.89 GB)
Raid Devices : 7
Total Devices : 7
Persistence : Superblock is persistent
Update Time : Sun Sep 9 05:24:11 2018
State : clean
Active Devices : 7
Working Devices : 7
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 512K
Name : node7:0 (local to host node7)
UUID : 7f86552b:c499fe55:d4be9502:c329d39c
Events : 18
Number Major Minor RaidDevice State
0 8 16 0 active sync /dev/sdb
1 8 32 1 active sync /dev/sdc
2 8 48 2 active sync /dev/sdd
3 8 64 3 active sync /dev/sde
4 8 80 4 active sync /dev/sdf
5 8 96 5 active sync /dev/sdg
7 8 112 6 active sync /dev/sdh
Creating file system for md0
Create a file system for ‘md0‘ device using ext4 before mounting.
mkfs.ext4 -E lazy_itable_init=0,lazy_journal_init=0 /dev/md0
mke2fs 1.41.9 (22-Aug-2009)
mkfs.ext4: Size of device /dev/md0 too big to be expressed in 32 bits
using a blocksize of 4096.
Because ext4 are still limited to 16 TB volumes, I decided to use xfs
mkfs.xfs /dev/md0
meta-data=/dev/md0 isize=256 agcount=44, agsize=268435328 blks
= sectsz=4096 attr=2, projid32bit=0
data = bsize=4096 blocks=11720047872, imaxpct=5
= sunit=128 swidth=768 blks
naming =version 2 bsize=4096 ascii-ci=0
log =internal log bsize=4096 blocks=521728, version=2
= sectsz=4096 sunit=1 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
Now create a directory under ‘/mnt‘ then mount the created filesystem under /mnt/raid5 and check the files under mount point, you will see lost+found directory. If you use xfs filesystem, there’s no lost+directory directory
mkdir /mnt/raid5
mount /dev/md0 /mnt/raid5/
ls -l /mnt/raid5/
Add entry in fstab
We need to add entry in fstab, else will not display our mount point after system reboot. To add an entry, we should edit the fstab file and append the following line as shown below. The mount point will differ according to your environment.
# vim /etc/fstab
/dev/md0 /mnt/raid5 xfs defaults 0 0
Next, run ‘mount -av‘ command to check whether any errors in fstab entry.
mount: /dev/disk/by-id/scsi-35000cca0545b2b14-part1 already mounted on /boot
mount: /dev/md0 already mounted on /mnt/raid5
mount: proc already mounted on /proc
mount: devpts already mounted on /dev/pts
nothing was mounted
Save Raid 5 Configuration
mdadm --detail --scan --verbose >> /etc/mdadm.conf
cat /etc/mdadm.conf
ARRAY /dev/md0 level=raid5 num-devices=7 metadata=1.2 name=node7:0 UUID=7f86552b:c499fe55:d4be9502:c329d39c
devices=/dev/sdb,/dev/sdc,/dev/sdd,/dev/sde,/dev/sdf,/dev/sdg,/dev/sdh
Say something
Thank you
Your comment has been submitted and will be published once it has been approved.
OOPS!
Your comment has not been submitted. Please go back and try again. Thank You!
If this error persists, please open an issue by clicking here.