PowerCLI esxcli errors when attempting to reclaim path

[Fourmica]

I am writing a script that masks LUNs on all hosts in a cluster, so that they may be removed on the storage side without causing an All Paths Down error.

I have been working to implement this entirely with Powershell and PowerCLI, avoiding the use of scripted SSH to execute esxcli commands, instead using a get-esxcli object to issue all of the appropriate commands.

I am able to apply the rules, run them, confirm they are set to runtime, etc. However, the last step I am supposed to perform - reclaiming the LUN path - gives me a 500 Internal Server Error, InvalidOperation and spams /var/log/vmkernel with SCSI errors.

Note: I am running this against ESX(i) 4.1.

The command I am issuing looks like this:

$esxCli.corestorage.claiming.reclaim("naa." + $lunObject.LunUID.replace(":","").toLower())

This command is the equivalent of "esxcli corestorage claiming reclaim -d wwn" in the service console. The $lunObject.LunUID property is the WWN of the target LUN in the colon-separated format, thus adding the naa., replacing the colons, and making the WWN alpha characters lower-case.

After issuing the command, Powershell sits and thinks about it for a while. /var/log/vmkernel displays the following log information:

Apr 19 13:32:34 Arc2App02 vmkernel: 120:15:35:26.616 cpu7:4118)PowerPath: EmcpEs
xLogEvent: Info:emcp:MpxEsxPathUnclaim: Unclaimed path to volume naa.60060160f6802e004611898f122ce111. Setting state to ON.
Apr 19 13:32:34 Arc2App02 vmkernel: 120:15:35:26.636 cpu7:4118)PowerPath: EmcpEsxLogEvent: Info:emcp:MpxEsxPathUnclaim: Unclaiming path vmhba1:C0:T1:L48.
Apr 19 13:32:34 Arc2App02 vmkernel:
Apr 19 13:32:34 Arc2App02 vmkernel: 120:15:35:26.637 cpu7:4118)VMWARE SCSI Id: Id for vmhba2:C0:T0:L48
Apr 19 13:32:34 Arc2App02 vmkernel: 0x60 0x06 0x01 0x60 0xf6 0x80 0x2e 0x00 0x46 0x11 0x89 0x8f 0x12 0x2c 0xe1 0x11 0x56 0x52 0x41 0x49 0x44 0x20
Apr 19 13:32:34 Arc2App02 vmkernel:
Apr 19 13:32:34 Arc2App02 vmkernel: 120:15:35:26.637 cpu7:4118)VMWARE SCSI Id: Id for vmhba2:C0:T1:L48
Apr 19 13:32:34 Arc2App02 vmkernel: 0x60 0x06 0x01 0x60 0xf6 0x80 0x2e 0x00 0x46 0x11 0x89 0x8f 0x12 0x2c 0xe1 0x11 0x56 0x52 0x41 0x49 0x44 0x20
Apr 19 13:32:34 Arc2App02 vmkernel: 120:15:35:26.637 cpu7:4118)ScsiDeviceIO: 4510: QErr is correctly set to 0x0 for device naa.60060160f6802e004611898f122ce111.
Apr 19 13:32:34 Arc2App02 vmkernel: 120:15:35:26.641 cpu5:4313)ScsiDeviceIO: 1672: Command 0x28 to device "naa.60060160f6802e004611898f122ce111" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
Apr 19 13:32:34 Arc2App02 vmkernel:
Apr 19 13:32:34 Arc2App02 vmkernel: 120:15:35:26.637 cpu7:4118)VMWARE SCSI Id: Id for vmhba2:C0:T1:L48
Apr 19 13:32:34 Arc2App02 vmkernel: 0x60 0x06 0x01 0x60 0xf6 0x80 0x2e 0x00 0x46 0x11 0x89 0x8f 0x12 0x2c 0xe1 0x11 0x56 0x52 0x41 0x49 0x44 0x20
Apr 19 13:32:34 Arc2App02 vmkernel: 120:15:35:26.637 cpu7:4118)ScsiDeviceIO: 4510: QErr is correctly set to 0x0 for device naa.60060160f6802e004611898f122ce111.

The log then spams the following:

Apr 19 13:32:34 Arc2App02 vmkernel: 120:15:35:26.641 cpu5:4313)ScsiDeviceIO: 1672: Command 0x28 to device "naa.60060160f6802e004611898f122ce111" failed H:0x2 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.

... for about a minute and a half, and then does this:

Apr 19 13:33:54 Arc2App02 vmkernel: 120:15:36:46.690 cpu7:4118)WARNING: Partition: 801: Partition table read from device naa.60060160f6802e004611898f122ce111 failed: I/O error

Then Powershell returns this:

The remote server returned an error: (500) Internal Server Error.
At C:\powershell\remove-lun_rewrite.ps1:1002 char:44
+             $esxCli.corestorage.claiming.reclaim <<<< ("naa." + $lunObject.LunUID.replace(":","").toLower())
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : MethodInvocationException

If I issue the command "esxcli corestorage claiming reclaim -d wwn" from the service console, it works fine, which leads me to believe the SCSI errors and Partition table read errors are erroneous.

Any idea why there are errors being generated when I use PowerCLI, but not from the service console? I know get-esxcli is "experimental", so if the answer is "sometimes it's buggy", well, that's just the way it is. But I wonder if I am doing something wrong in the way I'm using $esxCli from PowerCLI that's causing the problem.

Thanks,

Josh