VMware Cloud Community

After re-running an incomplete recovery, SRM is showing VM Powered Off even though it failed to power off on 1st Recovery

I have a VPLEX DR1 Metro volume as our datastore in Vsphere and is replicating to a remote site using EMC RecoverPoint. The VPLEX DR1 is a mirrored stretched volume between two clusters. In my production site my VC has two ESXs which each are zoned to one of the two VPLEX clusters. 

1. I caused a storage disaster on the VPLEX winner site

2. Ran recovery in SRM with Disk.AutoremoveOnPDL=0 set on both production ESXs. My production site placeholder is also on another vplex DR1 datastore that is seen by both production site ESXs. Currently in this step, only the ESX that is zoned with the VPLEX loser cluster has access to the DR1 datastore since the winner is down. This causes fracture on the DR1 volume.

This step ends with an incomplete recovery due to the fact that our placeholder, which also resides on a stretched Metro volume, is also fractured.

In the ‘Recovery Plan History’ it shows that my single VM is powered off, but in the VM events I am seeing this:

Initiate guest OS shutdown

SRM <----- This is the name of my VM

The operation cannot be allowed at the current time because the virtual machine has a question pending: 'msg.hbacommon.as-konpermanent-deviceloss:The storage backing virtual disk SRM_2.vmdk has permanent device loss. You may be able to hot remove this virtual device from the virtual machine and continue after clicking Retry. Click Cancel to terminate this session. '.



5/12/2014 9:23:17 AM

5/12/2014 9:23:17 AM

5/12/2014 9:23:17 AM

After fixing the disaster, I re-run the ‘Recovery’ and fail with this message:

Error - The attempted operation cannot be performed in the current state (Powered on). Failed to unregister protected VMs.

The Recovery Plan History Report for the 2nd Recovery is showing that the VM is already Powered Off when in fact it is not.

Attached Logs

Production VC: drpk103

Remote VC: drpk101

Production Winner VPLEX side ESX: drpk111

Production Loser VPLEX side ESX: drpk110

Remote site ESX: drpk111

0 Kudos
0 Replies