Highlighted
Contributor
Contributor

Can't Migrate VM

I am having trouble migrating a VM from a datastore.  I am trying to decommission this SAN and I have been able to move all other VM's off except one.  I am not able to move it powered on on or off.  When I try to move it using vcenter I get the attached error.  If I use vmkfstools -i to clone it I get a "Function Not Implemented" error and if I try a straight cp copy of the VM I get a "read:   Function Not Implemented"  error.  The VM can be powered on and the OS and Service functions fine.

I have narrowed the problem to one of the two vmdk's the other vmdk cp's and " vmkfstools's" just fine.  I am a bit stuck on how to proceded.  I am hoping someone out there has had a problem like this and a quick fix. I would rather not rebuild the VM or restore backups to new VM if possible.  Any idea's would be a great help.  Thanks.

quick enviroment info

Current Datastore:  Windows Storage Server.

Datastore to migrate to: Dell Equallogic

Vmware 4.1 environment. 

0 Kudos
28 Replies
Highlighted
Immortal
Immortal

i'm not saying this will fix it, but it may be worth a shot.  First thing I would do is restart the management agents of both the source and destination host

service mgmt-vmware restart

Also, you may try powering off the guest, making a note of the datastore it's registered in, remove the guest from inventory, browse the datastore find the vmx file, right click and add to inventory.  From there power on the guest and see if you can Storage vMotion again.

0 Kudos
Highlighted
Immortal
Immortal

I think you would get a different response if this was the case but just in case. Is the block size of the of the destination datastore large enough to accomodate the size of the VMDK you are trying to copy?

-- David -- VMware Communities Moderator
0 Kudos
Highlighted
Expert
Expert

Have u tried doing a storage vMotion with the VM turned off?

Follow me @ Cloud-Buddy.com

Blog: www.Cloud-Buddy.com | Follow me @hashmibilal
0 Kudos
Highlighted
Enthusiast
Enthusiast

I would check block size as advised by DSTRAVERT. Aditionnaly what type of disk format did you use (thick/thin are good for esx)?

If nothing works you cold also do a disk copy from the guest. For example I have used gparted several times.

0 Kudos
Highlighted
Contributor
Contributor

Thanks everyone for your responses.

I checked the block size and both datastores are set to 1MB block sizes.  I was able to move all the other VM from the this datastore to the new one except for the problem VM.  Also I have tried powering down the VM and Migrating.  Also just a straight cp of the VM's files fail.  The disk is thin now.   It also fails when I just cp the files on the same datastore.  I have narrowed the problem down to one of the two vmdk's of the VM.  I will probably try to P2V (V2V in this case) if I cant figure anything else out I hope that P2V doesn't fail because rebuilding the service on this VM will be a political mess Smiley Sad .  This is a production VM that I am trying to move to better disk storage, so it is hard for me to power down the VM when I need to.

I haven't tried to deregister the VM and move it yet, when I get a chance to power it down again it, I will give that a try.  Thanks again for all the idea's any other leads or idea are greatly appreceated

0 Kudos
Highlighted
Immortal
Immortal

How big are the disks in this VM?

With 1 MB of block size you can have only max 256 GB of each vmdk (also if is it thin).

Andre

Andre | http://about.me/amauro | http://vinfrastructure.it/ | @Andrea_Mauro
0 Kudos
Highlighted
Contributor
Contributor

100G each drive.  So 200G total.

Both drives are using very little actual space, around 20G between the both of them.

0 Kudos
Highlighted
Immortal
Immortal

The VM is already v7 hardware?

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=101250...

Another try could be create a snapshot and then delete all snapshots before make the Storage vMotion.

Andre

Andre | http://about.me/amauro | http://vinfrastructure.it/ | @Andrea_Mauro
0 Kudos
Highlighted
Contributor
Contributor

Yes the VM is hardware v7, and I did try to take a snapshot and delete all before migratingwith no luck.

0 Kudos
Highlighted
Contributor
Contributor

When I did shut down the VM and cp the files over the only file to fail was flat.vmdk.  it would try to cp and then error with something like read: function not implemented.

Could that flat file just be corrupt?  is there some tool to check flat.vmdk consistancy?

*note flat_1.vmdk worked but flat.vmdk did not so it looks like the second drive is fine..

0 Kudos
Highlighted
Immortal
Immortal

A vmdk corruption sound strange (which firmware are you using on Eql?).

Have you tried to use vMotion to move the VM on another host (to see if could be a host problem)?

Have you also tried to use datastore browser to take a copy of the vmdk?

Andre

Andre | http://about.me/amauro | http://vinfrastructure.it/ | @Andrea_Mauro
0 Kudos
Highlighted
Contributor
Contributor

EQ Firmware is latest ( just put it in ) the VM is on a Windows Storage Server Lun right now and I am trying to move it off of there though.  I can vmotion the VM to another host and the vomtion of the host works fine but the storage vmotion fails on either host.  I havent tried using just the datastore browser but I will have to schedule a time to power down the vm this week to try it.

0 Kudos
Highlighted
Immortal
Immortal

So it's on a NFS server? Or are you using iSCSI with the Windows Storage Server?

Maybe could be a problem on storage side.

I suggest to try a hot clone to see if at least you can copy the files.

If it works then also a vmdk copy with datastore browser or a cold clone will work.

If not, go to Converter and make a V2V.

Andre

Andre | http://about.me/amauro | http://vinfrastructure.it/ | @Andrea_Mauro
0 Kudos
Highlighted
Contributor
Contributor

Using the WSS iscsi target.

When you say hot clone.  you mean on the storage side.  like a snapshot? 

regardless, the more I get into this, the more I think I will need to v2v the server this week.  Smiley Sad

If I can get the v2v to work.  I think I may keep this lun/vm up and play around with it.  see if I can come up with something.

0 Kudos
Highlighted
Immortal
Immortal

Hot-clone is simple a clone with the VM running Smiley Happy

Just to see if the files are readble.

Andre

Andre | http://about.me/amauro | http://vinfrastructure.it/ | @Andrea_Mauro
0 Kudos
Highlighted
Contributor
Contributor

huh duh, yes I have tried to hot-clone the vm and it fails too. 

I found this in the hostd.log files on the host

VMotionLastStatusCb: Failed with error 536870915: Failed waiting for data.  Error bad0003. Not found.

This was right before it throughs the error.  This would point me to the storage but I can move any other vm in and out of this lun.

0 Kudos
Highlighted
Contributor
Contributor

Just as an update, I was able to v2v the machine and move it to another store.  I did how ever get an "Unable to read the block at offset XXXXXX" Warning when I converted.  The warning came when vmware convert was cloning the C: which was the same vmdk that I was having problems with.  I plan to save the old VM and play around with it and try to find a better cause.

0 Kudos
Highlighted
Hot Shot
Hot Shot

Why not backup the VM and restore it elsewhere?? You do backup your VMs, right??

0 Kudos
Highlighted
Contributor
Contributor

We do backup our applications and servers but we do not backup at the vmdk level yet.  So restoring would a pain because I would have to rebuild the OS and the features that support our applications.  So, v2v'ing the VM was the easier way for us because I didn't not have to re-project manage the building of the application with the vendor and the customer.

0 Kudos