Storage vmotion fails, stuck with delta files
Argyle Jul 17, 2008 7:29 AMI'm migrating VMs from a HP EVA 5000 SAN to an HP EVA 8000 SAN with storage vmotion using the Remote CLI tool. It works for most part but sometimes it fails midway and I'm stuck with:
vmdk files created on new LUNs (sometimes just one disk, sometimes all disks)
swap and .vmx files moved to new LUNs
delta files left on old LUNs
VM running but using delta files on old LUNs and swap on new LUNs
I could use some help completing or restarting the process. I was thinking about creating a new snapshot and committing it to get rid of the storage vmotion delta files but the option "Take snapshot" is gayed out in the Virtual Center for the VMs that fail this way.
Error this time showed up after about 10 minutes. Sys disk is 15 GB and data disk is 20 GB so no large files involved. Had same problem with a VM with 100 GB data disk, it failed after 20-30 minutes. The server running Remote CLI is on the SC network.
Writing down some system data and log info below:
Command and error message:
-
C:\Program Files\VMware\VMware VI Remote CLI\bin>svmotion.pl
--server=vc.mydomain.com --username=some_name --password=some_pass --datacenter="My Data Center"
--vm="[Sys-Disk-306] SERVER01/SERVER01.vmx: Sys-Disk-327"
--disks="[Sys-Disk-306] SERVER01/SERVER01.vmdk:Sys-Disk-327, SERVER01/SERVER01.vmdk:Data-Disk-328"
--verbose
Attempting to connect to service url.
Connected to server.
Resolving the input arguments.
Performing Storage VMotion.
Received an error from the server: An error occurred while communicating with the remote host.
C:\Program Files\VMware\VMware VI Remote CLI\bin>
-
Interesting info in the VMs wmare.log:
Jul 17 15:04:40.904: vcpu-0| HBACommon: First write on scsi0:0.fileName='/vmfs/volumes/47500341-572986f8-bf6e-001cc478a0d4/SERVER01//DMotion-scsi0:00_SERVER01.vmdk'
Jul 17 15:04:40.918: vcpu-0| DISKLIB-CHAIN : UpdateContentID: old = 0xc4692175, new = 0x3b52f9e2
Jul 17 15:12:40.901: vmx| vmdbPipe_Streams Couldn't read: OVL_STATUS_EOF
Jul 17 15:12:40.955: vmx| SOCKET 1 client closed connection
-
Currently the old LUNs look like this:
ll /vmfs/volumes/Sys-Disk-306/SERVER01/
-rw------- 1 root root 67141632 Jul 17 15:57 DMotion-scsi0:00_SERVER01-delta.vmdk
-rw------- 1 root root 326 Jul 17 15:04 DMotion-scsi0:00_SERVER01.vmdk
-rw------- 1 root root 16106127360 Jul 17 15:04 SERVER01-flat.vmdk
-rw------- 1 root root 342 Jun 14 13:15 SERVER01.vmdk
ll /vmfs/volumes/Data-Disk-308/SERVER01/
-rw------- 1 root root 16820224 Jul 17 15:28 DMotion-scsi0:01_SERVER01-delta.vmdk
-rw------- 1 root root 326 Jul 17 15:28 DMotion-scsi0:01_SERVER01.vmdk
-rw------- 1 root root 21474836480 Jul 17 14:28 SERVER01-flat.vmdk
-rw------- 1 root root 342 Jun 14 13:21 SERVER01.vmdk
Currently the new LUNs look like this:
ll /vmfs/volumes/Sys-Disk-327/SERVER01/
-rw------- 1 root root 805306368 Jul 17 15:04 SERVER01-a0151293.vswp
-rw------- 1 root root 16106127360 Jul 17 15:12 SERVER01-flat.vmdk
-rw------- 1 root root 8684 Jul 17 15:04 SERVER01.nvram
-rw------- 1 root root 403 Jul 17 15:08 SERVER01.vmdk
-rw------- 1 root root 0 Jul 17 15:03 SERVER01.vmsd
-rwxr-xr-x 1 root root 2789 Jul 17 15:17 SERVER01.vmx
-rw------- 1 root root 265 Jul 17 15:17 SERVER01.vmxf
-rw-rr 1 root root 22802 Jul 17 15:03 vmware-31.log
-rw-rr 1 root root 28840 Jul 17 15:03 vmware-32.log
-rw-rr 1 root root 31444 Jul 17 15:03 vmware-33.log
-rw-rr 1 root root 112306 Jul 17 15:03 vmware-34.log
-rw-rr 1 root root 30702 Jul 17 15:03 vmware-35.log
-rw-rr 1 root root 1479819 Jul 17 15:03 vmware-36.log
-rw-rr 1 root root 48791 Jul 17 15:28 vmware.log
ll /vmfs/volumes/Data-Disk-328/SERVER01/
-rw------- 1 root root 0 Jul 17 15:08 SERVER01.vmdk
-
The .vmx files show that its using vmdk and delta files on old LUNs and swap on new LUN.
Anyone experienced the same thing and is there a safe way to complete or rollback the process?