Using udev and VPD on XtremIO

A few weeks ago when looking at sdparm output I noticed that EMC XtremIO LUNs report the name of the corresponding volume on the array in their "VPD" (Vital Product Data) output. eg :

# sdparm -i /dev/sdb
    /dev/sdb: XtremIO   XtremApp          3000
Device identification VPD page:
  Addressed logical unit:
    designator type: NAA,  code set: Binary
      0x514f0c5991c001e2
    designator type: vendor specific [0x0],  code set: ASCII
      vendor specific: ScottH-ORA_Data1
    designator type: T10 vendor identification,  code set: ASCII
      vendor id: XtremIO
      vendor specific: 82ec40e65cf94342ac0be6db3bc3de5b
  Target port:
    designator type: Relative target port,  code set: Binary
      Relative target port: 0x2
  Target device that contains addressed lu:
    designator type: vendor specific [0x0],  code set: ASCII
      vendor specific: xio2
    designator type: vendor specific [0x0],  code set: ASCII
      vendor specific: ccda02991dde4d439981c1479396e1c9

The "ScottH-ORA_Data1" under the Addresses logical unit heading refers to the name of the volume, whilst the "xio2" under the "Target device that contains addressed lu" heading is the name of the XtremIO cluster that it's in.

This opens the door for scripts on the host to be able to detect a specific LUN, but we can take this a step further using Linux's udev.

udev is frequently used in database environments to set the owner/permissions on raw devices so that they can be access by the database user, without needing root permissions. The catch with doing this is always that you need to somehow detect which disks are database disks, and which are not, which often requires additional overhead like keeping a list of all serial numbers for your database disks and having udev match against that.

However udev also allows calling an external program during the detection cycle, so we can create a rule like the following (which goes in /etc/udev/rules.d/80-xtremio.rules) :

SUBSYSTEM=="block", ATTRS{vendor}=="XtremIO ", PROGRAM="/bin/xtremio_volname %k", RESULT=="*ORA*", OWNER="oracle", GROUP="dba", MODE="0660"

(Note that's all one line, and there's exactly one space after XtremIO and before the quote!)

The script being referenced, /bin/xtremio_volname is as simple as :

#!/bin/sh
sdparm -i /dev/$1 | grep ' vendor specific:' | head -1 | sed 's/^.*: //'
exit $?

This script will be passed the device being added as a plain device name (eg, "sdb"), and then just grabs and outputs the field from sdparm containing the name of the volume. At some point I'll write a better version of this script, but given that udev only calls this for XtremIO devices, it'll serve it's purpose at this time!

With the config above, udev will only run this script for block devices (SUBSYSTEM=="block") where the vendor name is XtremIO (ATTRS{vendor}=="XtremIO "). It'll then check the output of the command to see if it contains ORA somewhere within it (RESULT=="*ORA*") and only then will it proceed to set the owner, group and permissions on the newly created device to that required for Oracle (OWNER="oracle", GROUP="dba", MODE="0660")

Note the important distinction in udev rules between = and ==. A double equals is an equality check - the rule will only proceed if what's on the left matches what's on the right, so SUBSYSTEM=="block" means that SUBSYSTEM must be equal to "block" or this rule will not apply.

A single equals is a set operation - the value on the left will be set to what's on the right, so OWNER="oracle" means set the owner on the device to be oracle, regardless of what it is currently set to.

We can even take this a step further by adding a symlink to our new device, based on the name of the volume on the array, by adding a SYMLINK option :

SUBSYSTEM=="block", ATTRS{vendor}=="XtremIO ", PROGRAM="/bin/xtremio_volname %k", RESULT=="*ORA*", OWNER="oracle", GROUP="dba", MODE="0660", SYMLINK+="oracle/%c"

We've used the += operator to SYMLINK, which means that this will be added in addition to any other symlinks that the system would have automatically created. The name of the symlink will be oracle/, followed by %c which is the output from the PROGRAM command we ran - and it'll be put in the /dev directory.

For the the volume ScottH-ORA-Data1 on sdb we'll end up with a symlink /dev/oracle/ScottH-ORA-Data1 pointing to the device /dev/sdb

We can now configure Oracle to use the devices in /dev/oracle/*, either using their specific names or just by letting ASM look for all disks in that directory, knowing that the symlinks there will be automatically created, an the permissions will always be correct!

# ls -l /dev/oracle/*
lrwxrwxrwx. 1 root root 6 Dec  1 14:03 /dev/oracle/ScottH-ORA_Data1 -> ../sdb
lrwxrwxrwx. 1 root root 6 Dec  1 13:37 /dev/oracle/ScottH-ORA_Data2 -> ../sdc