RAID 1¶
On the frontend (monolithe.soc.lip6.fr) a RAID 1 of two 16 TB HDDs has
been mounted in the /media/raid1 directory.
Info
RAID 1 consists of data mirroring, without parity or striping. Data is written identically to two or more drives, thereby producing a "mirrored set" of drives. Thus, any read request can be serviced by any drive in the set. If a request is broadcast to every drive in the set, it can be serviced by the drive that accesses the data first (depending on its seek time and rotational latency), improving performance. Sustained read throughput, if the controller or software is optimized for it, approaches the sum of throughputs of every drive in the set, just as for RAID 0. Actual read throughput of most RAID 1 implementations is slower than the fastest drive. Write throughput is always slower because every drive must be updated, and the slowest drive limits the write performance. The array continues to operate as long as at least one drive is functioning.
Meteor Video Sequences¶
To date, the main purpose of this RAID is to store (and to preserve) video
sequences of meteors (in /media/raid1/meteors/).
To upload new video sequences you need write the files in the
/media/raid1/upload/ folder (and to be in the alsoc Unix group). Once the
upload is over, you can ask an administrator to copy the new files in the
/media/raid1/meteors/ folder for you.
Danger
It is forbidden to directly upload files in the /media/raid1/meteors/
folder to avoid mistakes.
NFS Backup¶
A backup of the NFS (/nfs) is automatically made each night at 03:00 AM in the
/media/raid1/backup_nfs directory based on the famous rsync command.
Here is the crontab script used to automatize the backup:
$ sudo crontab -l
# ...
# m h dom mon dow command
0 3 * * * rsync -a --delete /nfs/ /media/raid1/backup_nfs
Note
-a means that the rights and the ownerships will be preserved during the
copy.
--delete means that the files that are in /media/raid1/backup_nfs/ and
not in /nfs/ will be deleted.
Configuration¶
The installed RAID 1 is a software RAID based on the mdadm tool. The
/dev/sdc1 and /dev/sdd1 hardware HDDs are used to expose the /dev/md127
software RAID 1. Then, /dev/md127 is mounted in /media/raid1.
Info
The two 16 TB HDDs used for the RAID 1 are plugged as external HDD over
the USB 3.1 protocol (Type-C connector). The docking station is an
"ICY BOX Cloning & Docking Station USB 3.1 (Gen 2) Type-C (ref:
IB-121CL-C31)" and the HDDs are "Seagate Exos X18 16 TB (ref:
ST16000NM000J)". Two more HDDs are available as spare.
Statistics of the installed RAID:
$ sudo mdadm --detail /dev/md127
/dev/md127:
Version : 1.2
Creation Time : Wed Sep 20 15:47:54 2023
Raid Level : raid1
Array Size : 15625746368 (14901.87 GiB 16000.76 GB)
Used Dev Size : 15625746368 (14901.87 GiB 16000.76 GB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent
Intent Bitmap : Internal
Update Time : Wed Mar 13 16:44:14 2024
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Consistency Policy : bitmap
Name : flalsoc01:0
UUID : 5d71dd1f:c49dce68:d7ec64e5:9024c1b1
Events : 118933
Number Major Minor RaidDevice State
0 8 17 0 active sync /dev/sdb1
1 8 33 1 active sync /dev/sdc1
Source
- Installation guide: https://www.linuxbabe.com/linux-server/linux-software-raid-1-setup
Benchmarks¶
Testing the read throughput:
$ cd $HOME # to move into the $HOME (it is a SSD)
$ rsync -ah --progress /media/raid1/meteors/tau_Herculids_2022/basler/Basler_1920/Basler_acA1920-155um__23278758__20220531_034611281.mkv .
sending incremental file list
Basler_acA1920-155um__23278758__20220531_034611281.mkv
34.04G 100% 226.58MB/s 0:02:23 (xfr#1, to-chk=0/1)
Testing the write throughput:
$ rsync -ah --progress $HOME/Basler_acA1920-155um__23278758__20220531_034611281.mkv /media/raid1/upload/
sending incremental file list
Basler_acA1920-155um__23278758__20220531_034611281.mkv
34.04G 100% 205.71MB/s 0:02:37 (xfr#1, to-chk=0/1)
Clean the copied files:
$ rm $HOME/Basler_acA1920-155um__23278758__20220531_034611281.mkv
$ rm /media/raid1/upload/Basler_acA1920-155um__23278758__20220531_034611281.mkv
Or you can use a the hdparm benchmark: