VMware Cloud Community
CAE0621
Enthusiast
Enthusiast

Section of ESXi Deploy code runs in segments, but not when ran as whole

I've got an ESXi Deploy script that will run absolutely fine when I do it section-by-section with the F8 button, but when I go to the file and right click > run with Powershell 7 it will skip over a section. It doesn't throw any errors, it just skips it.  I've attached the offending section, plus the one immediately before it along with a screenshot of the code skipping over the section

 

 

...
write-host "Finished adding host to VDS" -ForegroundColor Green
Start-Sleep -Seconds 10
###########################################################################
########################Add VMNIC1 to UPLINK################################
###########################################################################
write-host "Moving VMNIC1 to VDS Uplink" -ForegroundColor White
$vmnic1 = Get-VMHost $HOSTNAME | Get-VMHostNetworkAdapter -Physical -Name 'vmnic1'
Get-VDSwitch $vds.Name | Add-VDSwitchPhysicalNetworkAdapter -VMHostPhysicalNic $vmnic1 -Confirm:$false 
write-host "Finished amoving VMNIC1 to VDS Uplink" -ForegroundColor Green
#Start-Sleep -Seconds 10
###########################################################################
##############Add vmnic1 to LAG, must be in UPLINK already#################
###########################################################################
write-host "Moving VMNIC1 to LAG" -ForegroundColor White
$phynic1 = 1
$lagName = 'lag1'
$vmnic1 = Get-VMHost $HOSTNAME | Get-VMHostNetworkAdapter -Physical -Name vmnic$phynic1
$tgtDevs = $vmnic1

$uplPg = Get-VDPortgroup -VDSwitch $vds -Name *Uplinks*

Get-View -Id $vds.ExtensionData.Summary.HostMember | 
Where-Object{$_.Name -eq $HOSTNAME} |
ForEach-Object -Process {
    $netSys = Get-View -Id $_.ConfigManager.NetworkSystem
    $lagPorts = (($netSys.NetworkInfo.ProxySwitch | Where-Object { $_.DvsUuid -eq $vds.Key }).HostLag |
        Where-Object { $_.LagName -eq $lagName }).UplinkPort.Key
    $i = 0
    
    $spec = New-Object VMware.Vim.HostNetworkConfig
    $proxy = New-Object VMware.Vim.HostProxySwitchConfig
    $proxy.ChangeOperation = [VMware.Vim.HostConfigChangeOperation]::edit
    $proxy.Uuid = $vds.Key
    $proxy.Spec = New-Object VMware.Vim.HostProxySwitchSpec
    $proxy.Spec.Backing = New-Object VMware.Vim.DistributedVirtualSwitchHostMemberPnicBacking
    
    ($netSys.NetworkInfo.ProxySwitch | Where-Object { $_.DvsUuid -eq $vds.Key }).Spec.Backing.PnicSpec |
    ForEach-Object -Process {
        if ($tgtDevs.Name -contains $_.PnicDevice)
        {
            $pNicSpec = New-Object VMware.Vim.DistributedVirtualSwitchHostMemberPnicSpec
            $pNicSpec.PnicDevice = $_.PnicDevice
            $pNicSpec.UplinkPortKey = $lagPorts[$i]
            $pNicSpec.UplinkPortgroupKey = $uplPg.Key
            $proxy.Spec.Backing.PnicSpec += $pNicSpec
            $i++
			}
        else
        {
            $proxy.Spec.Backing.PnicSpec += $_
        }
    }

    $spec.ProxySwitch += $proxy
    $netSys.UpdateNetworkConfig($spec, [VMware.Vim.HostConfigChangeMode]::modify)
   #Write-Host "This takes some time to be reflected, please wait"
    #Start-Sleep -s 15
    write-host "Finished adding VMNIC1 to LAG" -ForegroundColor Green
	}
###########################################################################
########################Move VMK to VDS###############################
###########################################################################
write-host "Moving VMK over to VDS" -ForegroundColor White
...

 

 

2021-04-29 13_27_50-ucw01camellis01 - Remote Desktop Connection.png

 

 

 

 

 

 

 

 

 

 

As you can see from the write-hosts, it successfully ads the host to VDS, moves the nic to the VDS, and STARTS the LAG section, but then it just skips it without an error and starts the next section.

But when I run it in Studio it works completely fine:

 

2021-04-29 13_36_18-ucw01camellis01 - Remote Desktop Connection.png

2021-04-29 13_37_15-Window.png

 

 

 

 

 

 

 

 

 

 

If anyone has any ideas on why this is happening, please let me know!

Labels (3)
Reply
0 Kudos
14 Replies
LucD
Leadership
Leadership

This looks a lot like a timing issue.
Did you check if this produces an object?

$netSys.NetworkInfo.ProxySwitch | Where-Object { $_.DvsUuid -eq $vds.Key }

Perhaps add an extra Write-Host to check

Write-Host "vdSwitch $(($netSys.NetworkInfo.ProxySwitch | Where-Object { $_.DvsUuid -eq $vds.Key }).dvsName)"

 


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
CAE0621
Enthusiast
Enthusiast

Skipped right over the section again, didn't do either write-host. I've tried putting in start-sleeps here and there, but it never seems to work. It's like it just skips over everything after the "Get-View" pipeline starts

2021-04-29 13_57_50-ucw01camellis01 - Remote Desktop Connection.png2021-04-29 13_57_28-ucw01camellis01 - Remote Desktop Connection.png

Reply
0 Kudos
LucD
Leadership
Leadership

Did you already try adding the Verbose switch when you call your function?
That should show which PowerCLI cmdlet is executed or not


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
CAE0621
Enthusiast
Enthusiast

I called them at the start of each ForEach loop, but it didn't show anything.

I did test something that was odd. If I run the script up until it adds the host to the VDS, then run a separate script the just adds the uplink and moves it, that works no problem. So I have you imagine you're right about it being a timing thing, I'm just not sure where it needs more time!

Reply
0 Kudos
LucD
Leadership
Leadership

After adding vmnic1 to the Uplinks, you could insert a loop that waits till that vmnic1 is actually shown in the Uplinks for that ESXi node.

Since VDS are managed on the vCenter level there might be a delay there.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
CAE0621
Enthusiast
Enthusiast

Unless you see an issue with my loop, that didn't do it 😕

2021-04-29 15_19_20-ucw01camellis01 - Remote Desktop Connection.png2021-04-29 15_20_05-ucw01camellis01 - Remote Desktop Connection.png

Reply
0 Kudos
LucD
Leadership
Leadership

I would do a test like this.
To avoid overwriting any of your variables, I added the suffix 'Test'

$esxTest = Get-VMHost $HOSTNAME
$netSysTest = Get-View -Id $esxTest.ExtensionData.ConfigManager.NetworkSystem
$vdsTest = $netSysTest.NetworkInfo.ProxySwitch.where{$_.DvsName -match $vds.Name}

while($vdsTest.pnic.foreach{($_.split('.')[-1]).Split('-')[1]} -notcontains $vmnic){
    sleep 1
    $netSysTest.UpdateViewData()
    $vdsTest = $netSysTest.NetworkInfo.ProxySwitch.where{$_.DvsName -match $vds.Name}
}

 


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
CAE0621
Enthusiast
Enthusiast

Afraid that didn't kick it into gear either. I changed $vmnic to $vmnic1 since $vmnic was empty and $vmnic1 has the vmnic we moved in it, I hope that was okay.

I'm going to try to write up something similar to put at the end of the LAG move section that will just sit there sleeping until it reads the LAG is moved

Reply
0 Kudos
CAE0621
Enthusiast
Enthusiast

Yeah, that didn't do it. It definitely looks like it just won't run that section of code in total. I'm going to beat my head against the desk for a while...

 

2021-04-30 08_21_02-ucw01camellis01 - Remote Desktop Connection.png

Realized the image cut off part of the script, here it is. It's pretty dirty but I just wanted something to try real quick then I would make something better. There is one other host in the environment, so there are 2 existing LAGs. Made it wait until there were 3 before exiting the loop

$Sabre = Get-VDPort -VDSwitch $vds -Uplink -ConnectedOnly | Where-Object -Property Name -like "lag*" 
write-host "Confirming Uplink is live"
while ($Sabre.count -lt 3){
    write-host "Please hold"
    Start-Sleep -Seconds 5
}
Reply
0 Kudos
CAE0621
Enthusiast
Enthusiast

Hope you're ready for head-scratcher on this one. So I split the script into two parts, put a pause prompt in the main script and copy/pasted the Uplink, LAG, VMK, Uplink2, LAG2 migration sections to a separate script. I run the network migration script during the pause, and everything goes off without a hitch! I have ZERO idea why this would be happening.

2021-04-30 08_49_01-ucw01camellis01 - Remote Desktop Connection.png

Any thoughts on this voodoo?

Reply
0 Kudos
LucD
Leadership
Leadership

I suspect the network system is doing stuff that is not completed when the 2nd part is started.

The problem is how to determine what it is doing and how to check if it completed


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
CAE0621
Enthusiast
Enthusiast

I threw in some code to wait a full minute throwing in a few $netSysTest.UpdateViewData() 's as well in three sections 

  1. VDS add (this is where I split the scripts and it magically works)
  2. Uplink add
  3. LAG move

No luck. I guess I'm just going to have to have a part 1 and part 2 to this script! I wonder if I can just call part 2 of the script from part 1 and have it run "automatically" that way...

Let me know if you have any more thoughts, otherwise I'm just going to call it good enough

Reply
0 Kudos
LucD
Leadership
Leadership

One option I would perhaps look at is to create a kind of time table.
First add timestamp on the Write-Host text.
Then retrieve events from what happened with Get-VIEvent. For TaskEvents that would be the start time and the duration.

Map those together on a timeline and see if that might shed a light on where the issue is.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
CAE0621
Enthusiast
Enthusiast

There's a lot of data, so I dumped it into two tabs in an excel doc to comb through along with the commands and powershell output (as an image)

I'm going to spend some time comparing between the two and finding the differences, but thought I'd give you the full data just in case you were curious.

Reply
0 Kudos