Making permissions on EMC PowerPath devices persistent for CRS on Enterprise Linux

 

PowerPath Device Permissions

It is possible to set permissions for EMC PowerPath devices to be persistent across server reboots and EMC PowerPath restarts.

This can be achieved through UDEV Rules.

For Oracle Cluster Ready Services (CRS) the permissions on devices supporting the OCR and Voting disk are essential for CRS to function.

Edit the /etc/udev/rules.d/50-udev.rules file and add the text below (where “emcpowerb” is the Voting disk and “emcpowerc” is the OCR)

# emc block devices
# Voting
SUBSYSTEM==”block”, KERNEL==”emcpowerb”, GROUP=”oinstall”, OWNER=”oracle”, MODE=”0640″

# OCR
SUBSYSTEM==”block”, KERNEL==”emcpowerc”, GROUP=”oinstall”, OWNER=”root”, MODE=”0640″

The voting disk is a partition that Oracle Clusterware uses to verify cluster node membership and status.

The voting disk must be owned by the oracle user, must be in the dba or oinstall group, and must have permissions set to 644

The location of the Voting disk may be found by executing the following command as root user:

<CRS_HOME>/bin/crsctl query css votedisk

E.g.

/u01/app/oracle/product/11.1.0/crs/bin/crsctl query css votedisk

The OCR disk supports the Oracle Clusterware Registry, which holds the complete configuaration of the cluster.

This includes metadata for nodeapps, ASM instances, database instances, database listeners, databases etc.

The OCR location may be found in /etc/oracle/ocr.loc

emcpower* devices are pseudo device names created by EMC PowerPath.

It is also possible to add the following text to /etc/init.d/init.crs file to enable persistence.

chown root:oinstall /dev/emcpowerc

chown oracle:oinstall /dev/emcpowerb

However, root.sh will overwrite this file when performing a CRS upgrade. Therefore the /etc/init.d/init.crs file should not be modified.

Pseudo Device Name Persistence

 

It is essential that EMC PowerPath pseudo device names are consistent between nodes in a cluster. Also the name must be persistent across server reboots. Below are commands with example output that help to illustrate the ease of device management from the Linux command line.

 

The following EMC PowerPath Management command displays the state of all the HBAs configured for PowerPath:

 

[root@#####01A ~]# powermt display dev=all

Pseudo name=emcpowerh

Symmetrix ID=000290104687

Logical device ID=04C5

state=alive; policy=SymmOpt; priority=0; queued-IOs=0

==============================================================================

---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---

###  HW Path                I/O Paths    Interf. Mode    State  Q-IOs Errors

==============================================================================

   2 lpfc                      sdd       FA 3cB   active  alive 0      0

   0 lpfc                      sdm       FA 14cB active  alive      0 0

 

Pseudo name=emcpowerg

Symmetrix ID=000290104687

Logical device ID=04CD

state=alive; policy=SymmOpt; priority=0; queued-IOs=0

==============================================================================

---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---

###  HW Path                I/O Paths    Interf. Mode    State  Q-IOs Errors

==============================================================================

   2 lpfc                      sde       FA 3cB   active  alive 0      0

   0 lpfc                      sdn       FA 14cB active  alive      0 0

 

Pseudo name=emcpowerf

Symmetrix ID=000290104687

Logical device ID=04D5

state=alive; policy=SymmOpt; priority=0; queued-IOs=0

==============================================================================

---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---

###  HW Path                I/O Paths    Interf. Mode    State  Q-IOs Errors

==============================================================================

   2 lpfc                      sdf       FA 3cB   active  alive 0      0

   0 lpfc                      sdo       FA 14cB active  alive      0 0

 

Pseudo name=emcpowerc

Symmetrix ID=000290104687

Logical device ID=04DD

state=alive; policy=SymmOpt; priority=0; queued-IOs=0

==============================================================================

---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---

###  HW Path                I/O Paths    Interf. Mode    State  Q-IOs Errors

==============================================================================

   2 lpfc                      sdg       FA 3cB   active  alive 0      0

   0 lpfc                      sdp       FA 14cB active  alive      0 0

 

Pseudo name=emcpowerb

Symmetrix ID=000290104687

Logical device ID=04E5

state=alive; policy=SymmOpt; priority=0; queued-IOs=0

==============================================================================

---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---

###  HW Path                I/O Paths    Interf. Mode    State  Q-IOs Errors

==============================================================================

   2 lpfc                      sdh       FA 3cB   active  alive 0      0

   0 lpfc                      sdq       FA 14cB active  alive      0 0

 

Pseudo name=emcpowera

Symmetrix ID=000290104687

Logical device ID=04ED

state=alive; policy=SymmOpt; priority=0; queued-IOs=0

==============================================================================

---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---

###  HW Path                I/O Paths    Interf. Mode    State  Q-IOs Errors

==============================================================================

   2 lpfc                      sdi       FA 3cB   active  alive 0      0

   0 lpfc                      sdr       FA 14cB active  alive      0 0

 

Pseudo name=emcpowere

Symmetrix ID=000290104687

Logical device ID=05FD

state=alive; policy=SymmOpt; priority=0; queued-IOs=0

==============================================================================

---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---

###  HW Path                I/O Paths    Interf. Mode    State  Q-IOs Errors

==============================================================================

   2 lpfc                      sdb       FA 3cB   active  alive 0      0

   0 lpfc                      sdk       FA 14cB active  alive      0 0

 

Pseudo name=emcpowerd

Symmetrix ID=000290104687

Logical device ID=05FE

state=alive; policy=SymmOpt; priority=0; queued-IOs=0

==============================================================================

---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---

###  HW Path                I/O Paths    Interf. Mode    State  Q-IOs Errors

==============================================================================

   2 lpfc                      sdc       FA 3cB   active  alive 0      0

   0 lpfc                      sdl FA 14cB   active  alive 0      0

 

 

The following command lists all used PowerPath pseudo device names:

 

[root@#####01A ~]# emcpadm getusedpseudos

 

PowerPath pseudo device names in use:

 

Pseudo Device Name      Major# Minor#

        emcpowera         120      0

        emcpowerb         120 16

        emcpowerc         120 32

        emcpowerd         120 48

        emcpowere         120 64

        emcpowerf         120 80

        emcpowerg         120 96

        emcpowerh         120 112

 

The following command lists available PowerPath pseudo device names:

 

[root@#####01A ~]# emcpadm getfreepseudos

 

Next free pseudo device name(s) from emcpowera are:

 

Pseudo Device Name      Major# Minor#

        emcpoweri         120 128

 

 

On Linux, EMC PowerPath configuration settings are stored in config file /etc/powermt.custom

 

To rename an existing PowerPath pseudo device name to an available pseudo device name, issue the following command with source and target names.

 

[root@#####01A ~]# emcpadm renamepseudo -s emcpowerg -t emcpoweri

 

To save changes made issue the following command:

 

[root@#####01A ~]# powermt save

 

By default, powermt save saves the PowerPath settings in the file powermt.custom

 

 

_______________________________________________________________________________

Did you find the article useful?

Please provide your feedback by voting now.

If you have a comment or question, please complete and submit the form below.

_______________________________________________________________________________