Boot from SAN and SharedClusterwide

[raidzero]

I'm having a bit of an issue with hosts that are booting from SAN and marking the boot device as shared clusterwide.  The boot LUN is unique per host.

I can set the shared clusterwide setting to false either through powercli, esxcli, or host profile, but on reboot the device returns to shared clusterwide true.  This causes a compliance failure in host profiles that we have to remediate every time.

This is 6.0 U3 in case that is relevant.  I can choose to ignore the shared clusterwide setting in host profile but this doesn't really seem to be the best way to do things.

[ashwin_prakash]

For SAN boot LUN devices shared across the cluster but logically local to the host, disable the Is Shared Clusterwide on the reference host. You must set the value to False before extracting the host profile from the reference host. When applying the host profile to the target host, the boot device settings for the remote boot LUN device are copied from the reference host into the target host. Select Storage configuration > Pluggable Storage Architecture configuration > Host boot device configuration and verify that this LUN is correctly captured.

Remediate the profile to the reference host for the changes in the sharing state to take effect on the reference host

Below mentioned is the Host Profile doc for your reference.

https://docs.vmware.com/en/VMware-vSphere/6.0/vsphere-esxi-vcenter-server-601-host-profiles-guide.pd...

[raidzero]

The setting, whether set by esxcli, powercli, or host profiles is not persistent across reboots.

If the setting is set to false, and then extracted to a host profile, and then that profile is applied to the same host, then the host will be compliant with the profile (obviously, being the reference host).

If that host is rebooted, it will no longer be in compliance because the device will return to IsSharedClusterwide = true on reboot.

[ashwin_prakash]

Could you please shared the output from the command "esxcli storage core device list"

[raidzero]

Sure.  Here is before and after a reboot.  You can see the Is Shared Clusterwide setting (line 27) go from false to true.

Before:

naa.624a93704d8e2dbf5aa3428700011021
   Display Name: PURE Fibre Channel Disk (naa.624a93704d8e2dbf5aa3428700011021)
   Has Settable Display Name: true
   Size: 10240
   Device Type: Direct-Access
   Multipath Plugin: NMP
   Devfs Path: /vmfs/devices/disks/naa.624a93704d8e2dbf5aa3428700011021
   Vendor: PURE
   Model: FlashArray
   Revision: 8888
   SCSI Level: 6
   Is Pseudo: false
   Status: on
   Is RDM Capable: true
   Is Local: false
   Is Removable: false
   Is SSD: true
   Is VVOL PE: false
   Is Offline: false
   Is Perennially Reserved: false
   Queue Full Sample Size: 0
   Queue Full Threshold: 0
   Thin Provisioning Status: yes
   Attached Filters:
   VAAI Status: supported
   Other UIDs: vml.0200010000624a93704d8e2dbf5aa3428700011021466c61736841
   Is Shared Clusterwide: false
   Is Local SAS Device: false
   Is SAS: false
   Is USB: false
   Is Boot USB Device: false
   Is Boot Device: true
   Device Max Queue Depth: 32
   No of outstanding IOs with competing worlds: 32
   Drive Type: unknown
   RAID Level: unknown
   Number of Physical Drives: unknown
   Protection Enabled: false
   PI Activated: false
   PI Type: 0
   PI Protection Mask: NO PROTECTION
   Supported Guard Types: NO GUARD SUPPORT
   DIX Enabled: false
   DIX Guard Type: NO GUARD SUPPORT
   Emulated DIX/DIF Enabled: false

And after:

naa.624a93704d8e2dbf5aa3428700011021
   Display Name: PURE Fibre Channel Disk (naa.624a93704d8e2dbf5aa3428700011021)
   Has Settable Display Name: true
   Size: 10240
   Device Type: Direct-Access
   Multipath Plugin: NMP
   Devfs Path: /vmfs/devices/disks/naa.624a93704d8e2dbf5aa3428700011021
   Vendor: PURE
   Model: FlashArray
   Revision: 8888
   SCSI Level: 6
   Is Pseudo: false
   Status: on
   Is RDM Capable: true
   Is Local: false
   Is Removable: false
   Is SSD: true
   Is VVOL PE: false
   Is Offline: false
   Is Perennially Reserved: false
   Queue Full Sample Size: 0
   Queue Full Threshold: 0
   Thin Provisioning Status: yes
   Attached Filters:
   VAAI Status: supported
   Other UIDs: vml.0200010000624a93704d8e2dbf5aa3428700011021466c61736841
   Is Shared Clusterwide: true
   Is Local SAS Device: false
   Is SAS: false
   Is USB: false
   Is Boot USB Device: false
   Is Boot Device: true
   Device Max Queue Depth: 32
   No of outstanding IOs with competing worlds: 32
   Drive Type: unknown
   RAID Level: unknown
   Number of Physical Drives: unknown
   Protection Enabled: false
   PI Activated: false
   PI Type: 0
   PI Protection Mask: NO PROTECTION
   Supported Guard Types: NO GUARD SUPPORT
   DIX Enabled: false
   DIX Guard Type: NO GUARD SUPPORT
   Emulated DIX/DIF Enabled: false

[ashwin_prakash]

Manully set the isLocal value to true:

Steps to be followed to mark the device as local:

# esxcli storage nmp satp rule add --satp=VMW_SATP_LOCAL -d NAA.ID_of_affected_datastore -o "enable_local"

# esxcli storage core claiming unclaim --type=device -d NAA.ID_of_affected_datastore

# esxcli storage core claimrule load

# esxcli storage core claimrule run

# esxcli storage core claiming reclaim -d NAA.ID_of_affected_datastore

Effect of above steps:

The above will set the PSP policy for that device to VMW_PSP_FIXED.

Reboot the ESXi host

esxcli storage core device setconfig --device <Local SAS device id> --shared-clusterwide=false

Reboot the ESXi host