Hi, I have a script that sets up VMware hosts networking, but for some reason when it's configuring the VMkernel to be enabled for vMotion, the script sits there waiting, and looking in the vCenter task, it always takes 60 seconds for this action to complete.
Is this normal, or is it some kind of fault,
Here is my command line that's causing the wait
New-VMHostNetworkAdapter -VMhost $vmhost -virtualswitch vSwitch1 -portgroup vMotion_02 -ip $vmotionip1 -subnetmask 255.255.255.0 -Mtu 1500 | Set-VMHostNetworkAdapter -VMotionEnabled $true -Confirm:$False
vCenter Task details
Select vMotion virtual NIC
start time - 18/07/2016 14:22:27
Finish time - 18/07/2016 14:23:28
Cheers in advance
You will need to have a look in the vpxd logs to see what is actually taking so long.
As a sidenote, why the separate Set-NetworkAdapter? You could do this all in the same cmdlet.
For example (btw I used splatting to make it a bit more readable)
$sNIC = @{
VMHost = $vmhost
VirtualSwitch = 'vSwitch1'
PortGroup = 'vMotion_02'
IP = $vmotionip1
SubnetMask = '255.255.255.0'
VMotionEnabled = $true
Confirm = $False
}
New-VMHostNetworkAdapter @sNIC
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
I'm pretty new to powershell/powerCLI extra
so I'm getting there with it, cheers for the advice
No problem :smileycool:
Is it still relatively slow when you do it with the one cmdlet?
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
yes,
So the commands are run as part of a wider range of commands for setting up the host, but essentially, this one is the command that takes the longest.
If I manually do each command once, the setting the vMotion as enabled on the VMkernel still takes 60 seconds, I'm just digging the VPXD logs now
So here is what I can see
2016-07-18T13:59:15.068Z info vpxd[7F12A1A5A700] [Originator@6876 sub=vpxLro opID=25df7774] [VpxLRO] -- BEGIN task-450 -- vmotionSystem-85 -- vim.host.VMotionSystem.selectVnic -- 528cbacf-8653-2a11-5a19-e11ca1677dd8(52c203eb-d209-034b-aae2-1ff5dd20f4ab)
2016-07-18T13:59:16.742Z info vpxd[7F12A1A5A700] [Originator@6876 sub=MoHostConfig opID=25df7774] [VpxdMoVMotionSystem::SelectVnic] Wait for another host sync to get the updated config.vmotion.
2016-07-18T13:59:20.787Z info vpxd[7F12A1A5A700] [Originator@6876 sub=MoHostConfig opID=25df7774] [VpxdMoVMotionSystem::SelectVnic] Wait for another host sync to get the updated config.vmotion.
2016-07-18T13:59:23.806Z info vpxd[7F129A37E700] [Originator@6876 sub=vpxLro opID=opId-4a7a6049-bb9e-4091-a288-cfc79dbcc093-0b-ba] [VpxLRO] -- BEGIN task-internal-37101 -- ServiceInstance -- vim.ServiceInstance.GetServerClock -- 52eec56b-d324-0482-9082-f321031163b9(52476de5-cefd-0c10-3358-c4372a62441a)
2016-07-18T13:59:23.806Z info vpxd[7F129A37E700] [Originator@6876 sub=vpxLro opID=opId-4a7a6049-bb9e-4091-a288-cfc79dbcc093-0b-ba] [VpxLRO] -- FINISH task-internal-37101
2016-07-18T13:59:28.824Z info vpxd[7F12A1A5A700] [Originator@6876 sub=MoHostConfig opID=25df7774] [VpxdMoVMotionSystem::SelectVnic] Wait for another host sync to get the updated config.vmotion.
2016-07-18T13:59:29.323Z info vpxd[7F129A37E700] [Originator@6876 sub=vpxLro opID=41e9774b] [VpxLRO] -- BEGIN task-internal-37105 -- ServiceInstance -- vim.ServiceInstance.GetContent -- 528fdae4-c7cd-a4a6-dc1d-afe447fd456b(522069fc-1b04-64ea-231c-19d4016386fb)
2016-07-18T13:59:29.323Z info vpxd[7F129A37E700] [Originator@6876 sub=vpxLro opID=41e9774b] [VpxLRO] -- FINISH task-internal-37105
2016-07-18T13:59:31.734Z info vpxd[7F129A37E700] [Originator@6876 sub=vpxLro opID=opId-7105e296-69ab-4f0d-a47a-6a6e8a55a452-eb-31] [VpxLRO] -- BEGIN task-internal-37106 -- ServiceInstance -- vim.ServiceInstance.GetServerClock -- 52148c8c-8030-f970-4851-fabee8a687b0(52098b41-f179-96ae-4b0d-be6d4a910baf)
2016-07-18T13:59:31.734Z info vpxd[7F129A37E700] [Originator@6876 sub=vpxLro opID=opId-7105e296-69ab-4f0d-a47a-6a6e8a55a452-eb-31] [VpxLRO] -- FINISH task-internal-37106
2016-07-18T13:59:35.831Z info vpxd[7F129A480700] [Originator@6876 sub=vpxLro opID=opId-0b605910-2141-497f-a7ee-cc6fcd95061d-ef-c2] [VpxLRO] -- BEGIN task-internal-37107 -- ServiceInstance -- vim.ServiceInstance.GetServerClock -- 521491ed-d696-cb3c-2d5d-3d7f894ff072(52fbe230-e55d-fc25-346e-deb3a644a76a)
2016-07-18T13:59:35.831Z info vpxd[7F129A480700] [Originator@6876 sub=vpxLro opID=opId-0b605910-2141-497f-a7ee-cc6fcd95061d-ef-c2] [VpxLRO] -- FINISH task-internal-37107
2016-07-18T13:59:40.871Z info vpxd[7F12A1A5A700] [Originator@6876 sub=MoHostConfig opID=25df7774] [VpxdMoVMotionSystem::SelectVnic] Wait for another host sync to get the updated config.vmotion.
2016-07-18T13:59:48.289Z info vpxd[7F1299E74700] [Originator@6876 sub=vpxLro opID=5acc69e6] [VpxLRO] -- BEGIN task-internal-37162 -- SessionManager -- vim.SessionManager.sessionIsActive -- 524c82a0-2ea9-292a-734c-712bbd3bd276(5202344d-52fa-94b4-6231-6b59aafcef2c)
2016-07-18T13:59:48.289Z info vpxd[7F1299E74700] [Originator@6876 sub=vpxLro opID=5acc69e6] [VpxLRO] -- FINISH task-internal-37162
2016-07-18T13:59:52.556Z info vpxd[7F1299E74700] [Originator@6876 sub=vpxLro opID=671e63f] [VpxLRO] -- BEGIN task-internal-37163 -- PerfMgr -- vim.PerformanceManager.queryStats -- 52af8b9e-72c1-a9af-521f-963d0a2f2023(52cfcca5-d355-a6fe-5ae7-97c659b72312)
2016-07-18T13:59:52.572Z info vpxd[7F1299E74700] [Originator@6876 sub=vpxLro opID=671e63f] [VpxLRO] -- FINISH task-internal-37163
2016-07-18T13:59:53.806Z info vpxd[7F1299E74700] [Originator@6876 sub=vpxLro opID=opId-4679ee64-5961-4f96-aae5-eb60fcce1da1-88-56] [VpxLRO] -- BEGIN task-internal-37164 -- ServiceInstance -- vim.ServiceInstance.GetServerClock -- 52eec56b-d324-0482-9082-f321031163b9(52476de5-cefd-0c10-3358-c4372a62441a)
2016-07-18T13:59:53.806Z info vpxd[7F1299E74700] [Originator@6876 sub=vpxLro opID=opId-4679ee64-5961-4f96-aae5-eb60fcce1da1-88-56] [VpxLRO] -- FINISH task-internal-37164
2016-07-18T13:59:56.912Z info vpxd[7F12A1A5A700] [Originator@6876 sub=MoHostConfig opID=25df7774] [VpxdMoVMotionSystem::SelectVnic] Wait for another host sync to get the updated config.vmotion.
2016-07-18T14:00:01.734Z info vpxd[7F129A603700] [Originator@6876 sub=vpxLro opID=opId-61dbc410-6e68-4890-a028-5913dbdd223e-52-d0] [VpxLRO] -- BEGIN task-internal-37166 -- ServiceInstance -- vim.ServiceInstance.GetServerClock -- 52148c8c-8030-f970-4851-fabee8a687b0(52098b41-f179-96ae-4b0d-be6d4a910baf)
2016-07-18T14:00:01.734Z info vpxd[7F129A603700] [Originator@6876 sub=vpxLro opID=opId-61dbc410-6e68-4890-a028-5913dbdd223e-52-d0] [VpxLRO] -- FINISH task-internal-37166
2016-07-18T14:00:02.584Z info vpxd[7F1299E74700] [Originator@6876 sub=vpxLro opID=2d915ede] [VpxLRO] -- BEGIN task-internal-37167 -- ServiceInstance -- vim.ServiceInstance.GetContent -- 52a1ca47-5f96-cdaf-3d5c-2aa8ffd935d2
2016-07-18T14:00:02.585Z info vpxd[7F1299E74700] [Originator@6876 sub=vpxLro opID=2d915ede] [VpxLRO] -- FINISH task-internal-37167
2016-07-18T14:00:02.605Z info vpxd[7F129ACFD700] [Originator@6876 sub=vpxLro opID=8eb0ae0] [VpxLRO] -- BEGIN task-internal-37168 -- SessionManager -- vim.SessionManager.GetCurrentSession -- 522140cc-051c-287c-0658-c08b9402afbf
2016-07-18T14:00:02.605Z info vpxd[7F129ACFD700] [Originator@6876 sub=vpxLro opID=8eb0ae0] [VpxLRO] -- FINISH task-internal-37168
2016-07-18T14:00:02.607Z info vpxd[7F1299E74700] [Originator@6876 sub=vpxLro opID=3e284b4d] [VpxLRO] -- BEGIN task-internal-37169 -- ServiceInstance -- vim.ServiceInstance.GetContent -- 527d1458-9eef-ad47-4684-f45c664e9fce
2016-07-18T14:00:02.607Z info vpxd[7F1299E74700] [Originator@6876 sub=vpxLro opID=3e284b4d] [VpxLRO] -- FINISH task-internal-37169
2016-07-18T14:00:02.615Z info vpxd[7F129ACFD700] [Originator@6876 sub=vpxLro opID=6f8966e3] [VpxLRO] -- BEGIN task-internal-37170 -- SessionManager -- vim.SessionManager.loginExtensionByCertificate -- 52e5c65b-0958-652b-119d-015b3029bf70
2016-07-18T14:00:02.616Z info vpxd[7F129ACFD700] [Originator@6876 sub=vpxLro opID=6f8966e3] [VpxLRO] -- FINISH task-internal-37170
2016-07-18T14:00:02.619Z info vpxd[7F1299E74700] [Originator@6876 sub=vpxLro opID=621723a3] [VpxLRO] -- BEGIN task-internal-37171 -- ServiceInstance -- vim.ServiceInstance.GetContent -- 52e5c65b-0958-652b-119d-015b3029bf70(52638661-c07f-71c1-3b82-bde745d6c7e6)
2016-07-18T14:00:02.619Z info vpxd[7F1299E74700] [Originator@6876 sub=vpxLro opID=621723a3] [VpxLRO] -- FINISH task-internal-37171
2016-07-18T14:00:02.631Z info vpxd[7F1299FF7700] [Originator@6876 sub=vpxLro opID=7ab3b6ee] [VpxLRO] -- BEGIN task-internal-37172 -- group-d1 -- vim.ManagedEntity.GetTag -- 52e5c65b-0958-652b-119d-015b3029bf70(52638661-c07f-71c1-3b82-bde745d6c7e6)
2016-07-18T14:00:02.631Z info vpxd[7F1299FF7700] [Originator@6876 sub=vpxLro opID=7ab3b6ee] [VpxLRO] -- FINISH task-internal-37172
2016-07-18T14:00:02.633Z info vpxd[7F129ACFD700] [Originator@6876 sub=vpxLro opID=15b8f5e2] [VpxLRO] -- BEGIN task-internal-37173 -- ServiceInstance -- vim.ServiceInstance.retrieveContent -- 52e5c65b-0958-652b-119d-015b3029bf70(52638661-c07f-71c1-3b82-bde745d6c7e6)
2016-07-18T14:00:02.633Z info vpxd[7F129ACFD700] [Originator@6876 sub=vpxLro opID=15b8f5e2] [VpxLRO] -- FINISH task-internal-37173
2016-07-18T14:00:02.640Z info vpxd[7F129A603700] [Originator@6876 sub=vpxLro opID=89c27ba] [VpxLRO] -- BEGIN task-internal-37174 -- SessionManager -- vim.SessionManager.logout -- 52e5c65b-0958-652b-119d-015b3029bf70(52638661-c07f-71c1-3b82-bde745d6c7e6)
2016-07-18T14:00:02.640Z info vpxd[7F129A603700] [Originator@6876 sub=vpxLro opID=89c27ba] [VpxLRO] -- FINISH task-internal-37174
2016-07-18T14:00:05.831Z info vpxd[7F129A786700] [Originator@6876 sub=vpxLro opID=opId-0b605910-2141-497f-a7ee-cc6fcd95061d-09-30] [VpxLRO] -- BEGIN task-internal-37175 -- ServiceInstance -- vim.ServiceInstance.GetServerClock -- 521491ed-d696-cb3c-2d5d-3d7f894ff072(52fbe230-e55d-fc25-346e-deb3a644a76a)
2016-07-18T14:00:05.831Z info vpxd[7F129A786700] [Originator@6876 sub=vpxLro opID=opId-0b605910-2141-497f-a7ee-cc6fcd95061d-09-30] [VpxLRO] -- FINISH task-internal-37175
2016-07-18T14:00:16.955Z error vpxd[7F12A1A5A700] [Originator@6876 sub=MoHostConfig opID=25df7774] [VpxdMoVMotionSystem::SelectVnic] Host sync doesn't send the updated config.vmotion after 60 seconds.
2016-07-18T14:00:16.955Z info vpxd[7F12A1A5A700] [Originator@6876 sub=vpxLro opID=25df7774] [VpxLRO] -- FINISH task-450
It looks as if the last line is the most interesting "Host sync doesnt send the updated config.vmotion after 60 seconds, so I presume that the host takes the vMotion config fine straight away, but doesnt tell vCenter
Looks more like there might be an issue with the sync of one or more of the vmkernels over which vMotion was defined.
Are you usiing VDS for all vmkernels, or are there VSS based ones?
Are all the nodes in the cluster fully configured?
Is HA showing warnings/errors?
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
using standard switches for everything
all hosts are configured and working with HA and DRS with no warnings or errors
Did you check if the time on all ESXi nodes is synchronised?
Do you have multiple multiple vmnic on the switch? Are they in an active/standby config?
Try use only one vmnic on the switch (move the 2nd one to Unused Adapters), see if that makes a difference.
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
Hi LucD,
great to read from you on this - that's raising hopes 😃
We are facing the same issue in our environment - no matter whether we try it on vDS's or vSS's, always on HA+DRS powered Clusters without warnings.
Indeedd, we have multiple vmnics on the vswitches but those 60 seconds are affecting us in active/active and active/standby configurations.
On vDS we are only able to trigger active/unused on portgroup basis - afaik there is no possibility to configure "teaming and failover" on vDS directly. But we get that delay even if we set the second adapter to unsused...
Did you tried that in your (test) environment and are you sure that this is not a general issue?
Thanks and regards!
Could be a general issue.
But not a global one, otherwise I would have expected more complaints.
In which vSphere version (vCenter/VCSA, ESXi) do you experience the issue?
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
Ah yes, we had similiar thoughts on this but didn't find a way to fix / work around this issue :smileyconfused:
We're on VCSA 6.0.0 3634749 / ESXi 6.0.0 3825889 and tested on vDS 5.5.0 and 6.0.0...