VMware Cloud Community
barnette08
Expert
Expert
Jump to solution

TCP/IP Stack

I wanted to start a separate thread since an old thread I had about this was two questions and marked as answered already.  Basically I'm having issues setting the TCP/IP stack for the vmotion vmk port in a script I have to setup the vmk ports.  Below is the snippet and vpxd log output.

$csv = Import-Csv C:\vmk.csv -UseCulture

foreach ($line in $csv){

Write-Host "Creating vMotion VMKernel Port for" $($line.Host)"..."

$vmk = New-VMHostNetworkAdapter -VMHost $line.Host -VirtualSwitch $line.dvSwitch -PortGroup $line.vMoPG -IP $line.vMoIP -SubnetMask $line.vMoMask -Mtu 9000 -VMotionEnabled:$true -Confirm:$false | Out-Null

Write-Host "Creating NFS VMKernel Port for" $line.Host"..."

New-VMHostNetworkAdapter -VMHost $line.Host -VirtualSwitch $line.dvSwitch -PortGroup $line.NFSPG -IP $line.NFSIP -SubnetMask $line.NFSMask -Mtu 9000 -Confirm:$false | Out-Null

Write-Host "Creating FT VMKernel Port for" $line.Host"..."

New-VMHostNetworkAdapter -VMHost $line.Host -VirtualSwitch $line.dvSwitch -PortGroup $line.FTPG -IP $line.FTIP -SubnetMask $line.FTMask -Mtu 9000 -FaultToleranceLoggingEnabled:$true -Confirm:$false | Out-Null

   $esx = Get-VMHost -Name $line.Host

   $netSys = Get-View -Id $esx.ExtensionData.ConfigManager.NetworkSystem

   $stack = $esx.ExtensionData.Config.Network.NetStackInstance | where{$_.Key -eq 'vmotion'}

  

   $config = New-Object VMware.Vim.HostNetworkConfig

   $spec = New-Object VMware.Vim.HostNetworkConfigNetStackSpec

   $spec.Operation = [VMware.Vim.ConfigSpecOperation]::edit

   $spec.NetStackInstance = $stack

   $spec.NetStackInstance.RouteTableConfig = New-Object VMware.Vim.HostIpRouteTableConfig

   $route = New-Object VMware.vim.HostIpRouteConfig

   $route.gatewayDevice = $vmk.DeviceName

   $spec.NetStackInstance.ipRouteConfig = $route

   $config.NetStackSpec += $spec

   $netsys.UpdateNetworkConfig($config,[VMware.Vim.HostConfigChangeMode]::modify)

}

Write-Host "***VMKernelConfiguration Section Complete***"  

2017-12-05T14:11:47.374Z info vpxd[7FD0CAEB5700] [Originator@6876 sub=Default opID=37816d5f] [VpxLRO] -- ERROR task-3365 -- networkSystem-434 -- vim.host.NetworkSystem.updateNetworkConfig: vim.fault.NotFound:

--> Result:

--> (vim.fault.NotFound) {

-->    faultCause = (vmodl.MethodFault) null,

-->    faultMessage = <unset>

-->    msg = "Received SOAP response fault from [<cs p:00007fd034139f30, TCP:host02.com:443>]: invokeHostTransactionCall

--> Received SOAP response fault from [<cs p:02081f38, TCP:localhost:8307>]: invokeHostTransactionCall

--> The object or item referred to could not be found."

--> }

--> Args:

-->

--> Arg config:

--> (vim.host.NetworkConfig) {

-->    vswitch = <unset>,

-->    proxySwitch = <unset>,

-->    portgroup = <unset>,

-->    pnic = <unset>,

-->    vnic = <unset>,

-->    consoleVnic = <unset>,

-->    dnsConfig = (vim.host.DnsConfig) null,

-->    ipRouteConfig = (vim.host.IpRouteConfig) null,

-->    consoleIpRouteConfig = (vim.host.IpRouteConfig) null,

-->    routeTableConfig = (vim.host.IpRouteTableConfig) null,

-->    dhcp = <unset>,

-->    nat = <unset>,

-->    ipV6Enabled = <unset>,

-->    netStackSpec = (vim.host.NetworkConfig.NetStackSpec) [

-->       (vim.host.NetworkConfig.NetStackSpec) {

-->          netStackInstance = (vim.host.NetStackInstance) {

-->             key = "vmotion",

-->             name = "vmotion",

-->             dnsConfig = (vim.host.DnsConfig) {

-->                dhcp = false,

-->                virtualNicDevice = <unset>,

-->                hostName = "",

-->                domainName = "",

-->                address = <unset>,

-->                searchDomain = <unset>

-->             },

-->             ipRouteConfig = (vim.host.IpRouteConfig) {

-->                defaultGateway = <unset>,

-->                gatewayDevice = "",

-->                ipV6DefaultGateway = <unset>,

-->                ipV6GatewayDevice = <unset>

-->             },

-->             requestedMaxNumberOfConnections = 11000,

-->             congestionControlAlgorithm = "newreno",

-->             ipV6Enabled = true,

-->             routeTableConfig = (vim.host.IpRouteTableConfig) {

-->                ipRoute = <unset>,

-->                ipv6Route = <unset>

-->             }

-->          },

-->          operation = "edit"

-->       }

-->    ]

--> }

--> Arg changeMode:

--> "modify"

Reply
0 Kudos
1 Solution

Accepted Solutions
LucD
Leadership
Leadership
Jump to solution

I can't seem to find a cmdlet or API to change the stack on an existing VMKernel, but you can create one with esxcli, directly with the correct stack.

Something like this.
Note that the PortGroup has to exist.

$esxName = 'esx1.local.lab'

$esx = Get-VMHost -Name $esxName

$esxcli = Get-EsxCli -VMHost $esx -V2

$sVmk = @{

    interfacename = 'vmk2'

    portgroupname = 'PG1'

    netstack = 'vmotion'

}

$esxcli.network.ip.interface.add.Invoke($sVmk)


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

View solution in original post

Reply
0 Kudos
10 Replies
LucD
Leadership
Leadership
Jump to solution

You did check that you have a vMotion netstack present on that ESXi node?

Is there an object in the $stack variable?


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
barnette08
Expert
Expert
Jump to solution

When I view the vmk adapters in the Web Client, I can see that vMotion is enabled but the TCP/IP stack is still set to default.  As for the $stack, it's set to as recommended before.

$stack = $esx.ExtensionData.Config.Network.NetStackInstance | where{$_.Key -eq 'vmotion'}

Reply
0 Kudos
nicholas1982
Hot Shot
Hot Shot
Jump to solution

Credit to LucD

This works nicely for me on dvSwitch

$esxiserver = Get-VMHost "esx.vsphere.local"

        $subnetmask = '255.255.255.0'

        $ipaddress = '10.10.10.10'

        $mtu = '9000'

        $stack = 'vmotion'

       

       

        ## Create vMotion Netstack ##

        $esxcli = Get-EsxCli -VMHost $esxiserver -V2

        $result = $esxcli.network.ip.netstack.add.invoke(@{netstack = $stack})

       

        ## Create vMotion VMKernel Portgroup ##

        $portgoup = $VmotionPg

        $nic = New-Object VMware.Vim.HostVirtualNicSpec

$distributedVirtualPort = New-Object VMware.Vim.DistributedVirtualSwitchPortConnection

        $nic.distributedVirtualPort = $distributedVirtualPort

        $nic.distributedVirtualPort.portgroupKey = $portgoup.key

        $nic.distributedVirtualPort.switchUuid = $portgoup.VirtualSwitch.key

        $nic.netStackInstanceKey = $stack

        $nic.mtu = $mtu

        $ip = New-Object VMware.Vim.HostIpConfig

        $ip.subnetMask = $subnetmask

        $ip.ipAddress = $ipaddress

        $ip.dhcp = $false

        $ip.ipV6Config = New-Object VMware.Vim.HostIpConfigIpV6AddressConfiguration

        $ip.ipV6Config.dhcpV6Enabled = $false

        $ip.ipV6Config.autoConfigurationEnabled = $false

        #$ip.IpV6Config = $ipV6Config

        $nic.Ip = $ip

        $networkSystem = $esxiserver.ExtensionData.configManager.NetworkSystem

        $_this = Get-view -Id ($networkSystem.Type + "-" + $networkSystem.Value)

        $result = $_this.AddVirtualNic('', $nic)

Nicholas
Reply
0 Kudos
barnette08
Expert
Expert
Jump to solution

I played around with it a little but get more errors.  Since I don't fully understand these objects, I'm sure I've put things in the wrong places.

Message: Unable to add Netstack Instance. Error was : Netstack instance 'vmotion' is already found in kernel;

InnerText: Unable to add Netstack Instance. Error was : Netstack instance 'vmotion' is already found in kernelEsxCLI.CLIFault.summary

At C:\vmk_create_vds2.ps1:11 char:9

+         $result = $esxcli.network.ip.netstack.add.invoke(@{netstack = ...

+         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : OperationStopped: (:) [], ViError

    + FullyQualifiedErrorId : VMware.VimAutomation.Sdk.Types.V1.ErrorHandling.VimException.ViError

Exception calling "AddVirtualNic" with "2" argument(s): "A specified parameter was not correct: PortConnection.portgroupKey. DVPortgroup  cannot be found"

At C:\vmk_create_vds2.ps1:33 char:9

+         $result = $_this.AddVirtualNic('', $nic)

+         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException

    + FullyQualifiedErrorId : VimException

foreach ($line in $csv){

Write-Host "Creating vMotion VMKernel Port for" $($line.Host)"..."

$esxiserver = Get-VMHost $line.Host

        $mtu = '9000'

        $stack = 'vmotion'

             

        ## Create vMotion Netstack ##

        $esxcli = Get-EsxCli -VMHost $esxiserver -V2

        $result = $esxcli.network.ip.netstack.add.invoke(@{netstack = $stack})

      

        ## Create vMotion VMKernel Portgroup ##

        $portgoup = $line.vMoPG

        $nic = New-Object VMware.Vim.HostVirtualNicSpec

$distributedVirtualPort = New-Object VMware.Vim.DistributedVirtualSwitchPortConnection

        $nic.distributedVirtualPort = $distributedVirtualPort

        $nic.distributedVirtualPort.portgroupKey = $portgoup.key

        $nic.distributedVirtualPort.switchUuid = $portgoup.VirtualSwitch.key

        $nic.netStackInstanceKey = $stack

        $nic.mtu = $mtu

        $ip = New-Object VMware.Vim.HostIpConfig

        $ip.subnetMask = $line.vMoMask

        $ip.ipAddress = $line.vMoIP

        $ip.dhcp = $false

        $ip.ipV6Config = New-Object VMware.Vim.HostIpConfigIpV6AddressConfiguration

        $ip.ipV6Config.dhcpV6Enabled = $false

        $ip.ipV6Config.autoConfigurationEnabled = $false

        #$ip.IpV6Config = $ipV6Config

        $nic.Ip = $ip

        $networkSystem = $esxiserver.ExtensionData.configManager.NetworkSystem

        $_this = Get-view -Id ($networkSystem.Type + "-" + $networkSystem.Value)

        $result = $_this.AddVirtualNic('', $nic)

}

btw, I'm sure it's driving you nuts - but how do I make my replied formatted in PS instead of plain? :smileyconfused:

Reply
0 Kudos
LucD
Leadership
Leadership
Jump to solution

Just to make sure I get the question correctly, you want to assign the vMotion netstack to the new VMKernel you created?


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
barnette08
Expert
Expert
Jump to solution

Correct

Reply
0 Kudos
LucD
Leadership
Leadership
Jump to solution

I can't seem to find a cmdlet or API to change the stack on an existing VMKernel, but you can create one with esxcli, directly with the correct stack.

Something like this.
Note that the PortGroup has to exist.

$esxName = 'esx1.local.lab'

$esx = Get-VMHost -Name $esxName

$esxcli = Get-EsxCli -VMHost $esx -V2

$sVmk = @{

    interfacename = 'vmk2'

    portgroupname = 'PG1'

    netstack = 'vmotion'

}

$esxcli.network.ip.interface.add.Invoke($sVmk)


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
barnette08
Expert
Expert
Jump to solution

This assumes vswitch not vds I assume?  Here is the error I get, but I think it's because I'm giving it a dvPortgroup.

Message: Invalid vswitch port group name esx_vmotion. Error was: Unable to get node: Sysinfo error: Not foundSee VMkernel log for details.;

InnerText: Invalid vswitch port group name esx_vmotion. Error was: Unable to get node: Sysinfo error: Not foundSee VMkernel log for details.EsxCLI.CLIFault.summary

At C:\vmk_create_vds2-testing.ps1:11 char:1

+ $esxcli.network.ip.interface.add.Invoke($sVmk)

+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : OperationStopped: (:) [], ViError

    + FullyQualifiedErrorId : VMware.VimAutomation.Sdk.Types.V1.ErrorHandling.VimException.ViError

Reply
0 Kudos
LucD
Leadership
Leadership
Jump to solution

The example was indeed for a VSS, but it should also work with a VDS.

You'll have to add additional parameter dvsname, and possibly dvportid.

Name                           Value                                                                                                               -----                           -----                                                                                                             interfacename                  Unset, ([string], optional)                                                                                         portgroupname                  Unset, ([string], optional)                                                                                         dvportid                       Unset, ([string], optional)                                                                                         mtu                            Unset, ([long], optional)                                                                                           macaddress                     Unset, ([string], optional)                                                                                         netstack                       Unset, ([string], optional)                                                                                         dvsname                        Unset, ([string], optional)


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
barnette08
Expert
Expert
Jump to solution

I'll dig into that, thanks!

Reply
0 Kudos