Hello community! I've just started to have issues with a host in my cluster. There are 3 nodes in which 1 node will not allow a vm to vmotion off the server nor will it let the server vmotion to it. The vmotion network is fine. I've tested dns, vmkping and ping as which are were successful. I also restarted the management agents as well. I've taken a look at esxtop and don't see anything that is jumping out at me.
When I attempt to vmotion whether "high priority" or "standard" vmotion starts at 3 percent and then to at 9 percent. Shortly thereafter I get the following message:
Migrate virtual machine
Failed to initialize migration at source.
VMotion failed to start due to lack of cpu or memory resources.
The vmkerrcode 0xbad00a4 which reports " vmk_preemptive_fail"
I have plenty of cpu and memory shown on each server and they are balanced as far as the cluster is shown with 3 connected. each having 98gb and the cpu's barely are working. I'd say maybe 2% at most on each. When I power off a machine, I can vmotion it to another host and vice versa but not powered on. When I try to vmotion back with a powered on machine as a test, I get the following below message. Please remember, this is only 1 host having the problem. I can vmotion fine between the other two, both servers are identical and vmotion worked fine before. The resource pools have never changed and I have no reservations whatsoever.
Failed to initialize migration at source. Error 195887268. VMotion failed to start due to lack of cpu or memory resources.
Failed to allocate migration heap.
Failed to create a migrate heap of size 30879984: Not found
Failed to reserve a migration memory slice. This host has reached its concurrent migration limit. Please wait for another migration to complete before retrying.
The vMotion failed because the destination host did not receive data from the source host on the vMotion network. Please check your vMotion network settings and physical network configuration and ensure they are correct.
Thanks in advance for any advice on how to resolve.