paradoxxaz
Enthusiast
Enthusiast

Machines wont migrate off host

So this is going to seem pretty broad, but I have a 3 node cluster. One of the nodes was throwing HA errors all over the place last week. I was able to finesse it enough (restarting services etc) to get it running. There are about 12 VM's on it.

Now, whenever I try to migrate a machine off, it just hangs at 10%, never moves.. ulitimately i think it errors with a time out...

Any ideas? One node is on update 4, the others, including this one, arent even on update 4, thats where I am trying to get. This Friday is maintenance, so I need to get all off so I can update. How the *&% am I going to get these machines off without just killing the hosts and letting HA take over? if it even will.

Thanks guys..

0 Kudos
48 Replies
Troy_Clavell
Immortal
Immortal

I think the best thing I can do to help you troubleshoot is point you to this KB article.

Diagnosing VMware VMotion failure at 10%

0 Kudos
max2479
Enthusiast
Enthusiast

What update are they on?

run esxupdate -l query

also take a look in the logs for the esx you are migrating from

cat /var/log/vmware/hostd.log | grep error

> if you found this or any other answer useful please consider allocating points for helpful or correct answers <

> if you found this or any other answer useful please consider allocating points for helpful or correct answers <
0 Kudos
paradoxxaz
Enthusiast
Enthusiast

I have hostd-0 thru 9.log

which is the current?

0 Kudos
paradoxxaz
Enthusiast
Enthusiast

Invalid transition req uested (VM_STATE_EMIGRATING -&gt; VM_STATE_RECONFIGURING): Invalid state

Invalid transition re quested (VM_STATE_EMIGRATING -&gt; VM_STATE_RECONFIGURING): Invalid state

Invalid transition r equested (VM_STATE_EMIGRATING -&gt; VM_STATE_RECONFIGURING): Invalid state

Invalid transition re quested (VM_STATE_EMIGRATING -&gt; VM_STATE_RECONFIGURING): Invalid state

Invalid transition re quested (VM_STATE_EMIGRATING -&gt; VM_STATE_RECONFIGURING): Invalid state

Invalid transition req uested (VM_STATE_EMIGRATING -&gt; VM_STATE_RECONFIGURING): Invalid state

Invalid transiti on requested (VM_STATE_EMIGRATING -&gt; VM_STATE_EMIGRATING): Invalid state

Method vim.host.VMotionManager.p repareSource threw undeclared fault of type vim.fault.InvalidState

Invalid transiti on requested (VM_STATE_EMIGRATING -&gt; VM_STATE_UNREGISTERING): Invalid state

Invalid tra nsition requested (VM_STATE_EMIGRATING -&gt; VM_STATE_EMIGRATING): Invalid state

Method vim.host.VMotionManager. prepareSource threw undeclared fault of type vim.fault.InvalidState

Invalid tran sition requested (VM_STATE_EMIGRATING -&gt; VM_STATE_EMIGRATING): Invalid state

Method vim.host.VMotionManager.p repareSource threw undeclared fault of type vim.fault.InvalidState

Invalid transit ion requested (VM_STATE_EMIGRATING -&gt; VM_STATE_EMIGRATING): Invalid state

Method vim.host.VMotionManager. prepareSource threw undeclared fault of type vim.fault.InvalidState

Invalid transiti on requested (VM_STATE_EMIGRATING -&gt; VM_STATE_EMIGRATING): Invalid state

Method vim.host.VMotionManager.p repareSource threw undeclared fault of type vim.fault.InvalidState

Invalid transiti on requested (VM_STATE_EMIGRATING -&gt; VM_STATE_EMIGRATING): Invalid state

Method vim.host.VMotionManager.p repareSource threw undeclared fault of type vim.fault.InvalidState

Invalid tran sition requested (VM_STATE_EMIGRATING -&gt; VM_STATE_EMIGRATING): Invalid state

Method vim.host.VMotionManager.p repareSource threw undeclared fault of type vim.fault.InvalidState

Invalid tran sition requested (VM_STATE_EMIGRATING -&gt; VM_STATE_EMIGRATING): Invalid state

Method vim.host.VMotionManager.p repareSource threw undeclared fault of type vim.fault.InvalidState

Invalid transition requested (VM_S TATE_EMIGRATING -&gt; VM_STATE_UNREGISTERING): Invalid state

Invalid transition requested (VM_STA TE_EMIGRATING -&gt; VM_STATE_UNREGISTERING): Invalid state

0 Kudos
paradoxxaz
Enthusiast
Enthusiast

# vmkping exc-scd-esxp09

PING exc-scd-esxp09 (172.17.0.24): 56 data bytes

64 bytes from 172.17.0.24: icmp_seq=0 ttl=64 time=0.672 ms

64 bytes from 172.17.0.24: icmp_seq=1 ttl=64 time=0.359 ms

64 bytes from 172.17.0.24: icmp_seq=2 ttl=64 time=0.286 ms

ping to other host trying to migrate to

0 Kudos
paradoxxaz
Enthusiast
Enthusiast

3 packets transmitted, 3 packets received, 0% packet loss

round-trip min/avg/max = 0.286/0.439/0.672 ms

0 Kudos
paradoxxaz
Enthusiast
Enthusiast

its been alomost 45 minutes, the migration is @ 13%

0 Kudos
paradoxxaz
Enthusiast
Enthusiast

what is the diff between these 2 commands:

service mgmt-vmware restart

service vmware-vpxa restart

0 Kudos
max2479
Enthusiast
Enthusiast

Invalid transiti on requested (VM_STATE_EMIGRATING -&gt; VM_STATE_UNREGISTERING): Invalid state

above is the problem. this particular VM, can you power on and off OK? Can you migrate any other VMs?

edit vmx and see if the configuration OK.

&gt; if you found this or any other answer useful please consider allocating points for helpful or correct answers &lt;

what is the ESX version you are migrating to?

esxupdate -l query

> if you found this or any other answer useful please consider allocating points for helpful or correct answers <
0 Kudos
max2479
Enthusiast
Enthusiast

Management agent on the host: service mgmt-vmware restart

Virtualcenter agent: service vmware-vpxa restart

&gt; if you found this or any other answer useful please consider allocating points for helpful or correct answers &lt;

> if you found this or any other answer useful please consider allocating points for helpful or correct answers <
0 Kudos
paradoxxaz
Enthusiast
Enthusiast

this is happening with any VM on the host... only 1 node in the 3 node cluster is on update 4, other 2 arent even on update 1

0 Kudos
max2479
Enthusiast
Enthusiast

just for rtesting purposes of course if you have the laxury of doing it.... can you power down 1 vm and migrate?

you may also try disable vmotion and enable it back. restart Management agent on the host and try it again

&gt; if you found this or any other answer useful please consider allocating points for helpful or correct answers &lt;

> if you found this or any other answer useful please consider allocating points for helpful or correct answers <
0 Kudos
paradoxxaz
Enthusiast
Enthusiast

if I shut down the VM it migrates

0 Kudos
max2479
Enthusiast
Enthusiast

Than in this case i would not spend any more time but cold migrate. After you will upgrade your ESX to 3.5 u4 there should not be any problems.

&gt; if you found this or any other answer useful please consider allocating points for helpful or correct answers &lt;

> if you found this or any other answer useful please consider allocating points for helpful or correct answers <
0 Kudos
paradoxxaz
Enthusiast
Enthusiast

HAHAHAHAH

Can YOU call my customers and tell them?

0 Kudos
max2479
Enthusiast
Enthusiast

Smiley Happy it was a long shot.....

have you tried to disable vmotion, enable it back? restart Management agent on the host?

&gt; if you found this or any other answer useful please consider allocating points for helpful or correct answers &lt;

> if you found this or any other answer useful please consider allocating points for helpful or correct answers <
0 Kudos
paradoxxaz
Enthusiast
Enthusiast

i see settings for disabling HA and one for DRS, where do I dis-re-eneable vmotion?

0 Kudos
Troy_Clavell
Immortal
Immortal

you have reviewed the KB I posted? Can you go to your ESX Service console and vmkping all of the vmkernel ports of all of your ESX Hosts in the cluster?

0 Kudos
paradoxxaz
Enthusiast
Enthusiast

found it

0 Kudos