• jekyll
  • linux

Or an alternate title for this post could be… “LSI, the raid controller I won’t purchase again”… or “MPIO, piss off”

Anyways, in the process of adding 4 new 600GB SAS drives to an x3550 M2 server I encountered a small error. After installing the drives into the machine, creating the raid, and rebooting something was amiss.

fdisk -l

Disk /dev/sda: 1197.9 GB, 1197998080000 bytes
255 heads, 63 sectors/track, 145648 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sdb: 145.9 GB, 145999527936 bytes
255 heads, 63 sectors/track, 17750 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *           1         867     6964146   83  Linux
/dev/sdb2             868       17495   133564410   83  Linux
/dev/sdb3           17496       17750     2048287+  82  Linux swap / Solaris

Disk /dev/dm-0: 1197.9 GB, 1197998080000 bytes
255 heads, 63 sectors/track, 145648 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Now my initial thought was, “what the hell is /dev/dm-0” but i moved ahead and tried fdisk anyways. At first I attempted to create a new primary partition and upon writing it out I got a wonderful error that i forgot to copy. Moral of the story, time to install 26 firmware updates.

This is usually a painful task but the IBM UpdateExpress is in fact the bees knees.

All you need is the following two commands and you are home free with all your Firmware and Driver updates.

./uxspi440.rhel5 ac

./uxspi440.rhel5 co

After some on screen instructions/user input it will do most everything for you. Thank you IBM

After installing the small army of FW and Driver updates I tried again

fdisk /dev/sda

Command (m for help): n
Command action
e extended
p primary partition (1-4)
Partition number (1-4): 1
First cylinder (1-145648, default 1): 
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-145648, default 145648): 
Using default value 145648

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

YAY, successories. The partition has been created lets create the filesystem.

mkfs.ext3 /dev/sda
/dev/sda1 is apparently in use by the system; will not make a filesystem here!

Of course looking at mount or fuser /dev/sda1 resulted in no information the drive wasn’t mounted anywhere or in use.

But after many days of working late I finally came back to that damned /dev/dm-0. Some further google fu resulted in the following two commands.

dmsetup status

dmsetup ls

This showed that there were several mpath devices configured on the system. Since this system is not using software RAID or MPIO this is a problem it should only return

# dmsetup ls
No devices found

# dmsetup status
No devices found

So the final fix this, tell MPIO and Multipath to get the hell out of your system by editing /etc/multipathd.conf and adding the following.

blacklist {
	devnode '*'

Reboot the server, the /dev/dm-0 will be gone.

And you can run mkfs.ext3 /dev/sda1 and then go outside and run in a field.

May be just me, but I believe a driver for a RAID controller should maybe… just maybe check for that during the install. oh well.