10 Replies Latest reply on Dec 6, 2017 11:12 AM by barnette08

    TCP/IP Stack

    barnette08 Hot Shot

      I wanted to start a separate thread since an old thread I had about this was two questions and marked as answered already.  Basically I'm having issues setting the TCP/IP stack for the vmotion vmk port in a script I have to setup the vmk ports.  Below is the snippet and vpxd log output.

       

      $csv = Import-Csv C:\vmk.csv -UseCulture
      
      foreach ($line in $csv){
      Write-Host "Creating vMotion VMKernel Port for" $($line.Host)"..."
      $vmk = New-VMHostNetworkAdapter -VMHost $line.Host -VirtualSwitch $line.dvSwitch -PortGroup $line.vMoPG -IP $line.vMoIP -SubnetMask $line.vMoMask -Mtu 9000 -VMotionEnabled:$true -Confirm:$false | Out-Null
      Write-Host "Creating NFS VMKernel Port for" $line.Host"..."
      New-VMHostNetworkAdapter -VMHost $line.Host -VirtualSwitch $line.dvSwitch -PortGroup $line.NFSPG -IP $line.NFSIP -SubnetMask $line.NFSMask -Mtu 9000 -Confirm:$false | Out-Null
      Write-Host "Creating FT VMKernel Port for" $line.Host"..."
      New-VMHostNetworkAdapter -VMHost $line.Host -VirtualSwitch $line.dvSwitch -PortGroup $line.FTPG -IP $line.FTIP -SubnetMask $line.FTMask -Mtu 9000 -FaultToleranceLoggingEnabled:$true -Confirm:$false | Out-Null
      
         $esx = Get-VMHost -Name $line.Host
         $netSys = Get-View -Id $esx.ExtensionData.ConfigManager.NetworkSystem
         $stack = $esx.ExtensionData.Config.Network.NetStackInstance | where{$_.Key -eq 'vmotion'}
         
         $config = New-Object VMware.Vim.HostNetworkConfig
         $spec = New-Object VMware.Vim.HostNetworkConfigNetStackSpec
         $spec.Operation = [VMware.Vim.ConfigSpecOperation]::edit
         $spec.NetStackInstance = $stack
         $spec.NetStackInstance.RouteTableConfig = New-Object VMware.Vim.HostIpRouteTableConfig
         $route = New-Object VMware.vim.HostIpRouteConfig
         $route.gatewayDevice = $vmk.DeviceName
         $spec.NetStackInstance.ipRouteConfig = $route
         $config.NetStackSpec += $spec
         $netsys.UpdateNetworkConfig($config,[VMware.Vim.HostConfigChangeMode]::modify)
      }
      
      Write-Host "***VMKernelConfiguration Section Complete***"   
      

       

      2017-12-05T14:11:47.374Z info vpxd[7FD0CAEB5700] [Originator@6876 sub=Default opID=37816d5f] [VpxLRO] -- ERROR task-3365 -- networkSystem-434 -- vim.host.NetworkSystem.updateNetworkConfig: vim.fault.NotFound:

      --> Result:

      --> (vim.fault.NotFound) {

      -->    faultCause = (vmodl.MethodFault) null,

      -->    faultMessage = <unset>

      -->    msg = "Received SOAP response fault from [<cs p:00007fd034139f30, TCP:host02.com:443>]: invokeHostTransactionCall

      --> Received SOAP response fault from [<cs p:02081f38, TCP:localhost:8307>]: invokeHostTransactionCall

      --> The object or item referred to could not be found."

      --> }

      --> Args:

      -->

      --> Arg config:

      --> (vim.host.NetworkConfig) {

      -->    vswitch = <unset>,

      -->    proxySwitch = <unset>,

      -->    portgroup = <unset>,

      -->    pnic = <unset>,

      -->    vnic = <unset>,

      -->    consoleVnic = <unset>,

      -->    dnsConfig = (vim.host.DnsConfig) null,

      -->    ipRouteConfig = (vim.host.IpRouteConfig) null,

      -->    consoleIpRouteConfig = (vim.host.IpRouteConfig) null,

      -->    routeTableConfig = (vim.host.IpRouteTableConfig) null,

      -->    dhcp = <unset>,

      -->    nat = <unset>,

      -->    ipV6Enabled = <unset>,

      -->    netStackSpec = (vim.host.NetworkConfig.NetStackSpec) [

      -->       (vim.host.NetworkConfig.NetStackSpec) {

      -->          netStackInstance = (vim.host.NetStackInstance) {

      -->             key = "vmotion",

      -->             name = "vmotion",

      -->             dnsConfig = (vim.host.DnsConfig) {

      -->                dhcp = false,

      -->                virtualNicDevice = <unset>,

      -->                hostName = "",

      -->                domainName = "",

      -->                address = <unset>,

      -->                searchDomain = <unset>

      -->             },

      -->             ipRouteConfig = (vim.host.IpRouteConfig) {

      -->                defaultGateway = <unset>,

      -->                gatewayDevice = "",

      -->                ipV6DefaultGateway = <unset>,

      -->                ipV6GatewayDevice = <unset>

      -->             },

      -->             requestedMaxNumberOfConnections = 11000,

      -->             congestionControlAlgorithm = "newreno",

      -->             ipV6Enabled = true,

      -->             routeTableConfig = (vim.host.IpRouteTableConfig) {

      -->                ipRoute = <unset>,

      -->                ipv6Route = <unset>

      -->             }

      -->          },

      -->          operation = "edit"

      -->       }

      -->    ]

      --> }

      --> Arg changeMode:

      --> "modify"

        • 1. Re: TCP/IP Stack
          LucD Guru
          vExpertCommunity WarriorsUser Moderators

          You did check that you have a vMotion netstack present on that ESXi node?

          Is there an object in the $stack variable?

          • 2. Re: TCP/IP Stack
            barnette08 Hot Shot

            When I view the vmk adapters in the Web Client, I can see that vMotion is enabled but the TCP/IP stack is still set to default.  As for the $stack, it's set to as recommended before.

             

            $stack = $esx.ExtensionData.Config.Network.NetStackInstance | where{$_.Key -eq 'vmotion'}

            • 3. Re: TCP/IP Stack
              nicholas1982 Hot Shot

              Credit to LucD

               

              This works nicely for me on dvSwitch

               

              $esxiserver = Get-VMHost "esx.vsphere.local"

                      $subnetmask = '255.255.255.0'

                      $ipaddress = '10.10.10.10'

                      $mtu = '9000'

                      $stack = 'vmotion'

                     

                     

                      ## Create vMotion Netstack ##

                      $esxcli = Get-EsxCli -VMHost $esxiserver -V2

                      $result = $esxcli.network.ip.netstack.add.invoke(@{netstack = $stack})

                     

                      ## Create vMotion VMKernel Portgroup ##

                      $portgoup = $VmotionPg

                      $nic = New-Object VMware.Vim.HostVirtualNicSpec

              $distributedVirtualPort = New-Object VMware.Vim.DistributedVirtualSwitchPortConnection

                      $nic.distributedVirtualPort = $distributedVirtualPort

                      $nic.distributedVirtualPort.portgroupKey = $portgoup.key

                      $nic.distributedVirtualPort.switchUuid = $portgoup.VirtualSwitch.key

                      $nic.netStackInstanceKey = $stack

                      $nic.mtu = $mtu

                      $ip = New-Object VMware.Vim.HostIpConfig

                      $ip.subnetMask = $subnetmask

                      $ip.ipAddress = $ipaddress

                      $ip.dhcp = $false

                      $ip.ipV6Config = New-Object VMware.Vim.HostIpConfigIpV6AddressConfiguration

                      $ip.ipV6Config.dhcpV6Enabled = $false

                      $ip.ipV6Config.autoConfigurationEnabled = $false

                      #$ip.IpV6Config = $ipV6Config

                      $nic.Ip = $ip

                      $networkSystem = $esxiserver.ExtensionData.configManager.NetworkSystem

                      $_this = Get-view -Id ($networkSystem.Type + "-" + $networkSystem.Value)

                      $result = $_this.AddVirtualNic('', $nic)

              • 4. Re: TCP/IP Stack
                barnette08 Hot Shot

                I played around with it a little but get more errors.  Since I don't fully understand these objects, I'm sure I've put things in the wrong places.

                 

                Message: Unable to add Netstack Instance. Error was : Netstack instance 'vmotion' is already found in kernel;

                InnerText: Unable to add Netstack Instance. Error was : Netstack instance 'vmotion' is already found in kernelEsxCLI.CLIFault.summary

                At C:\vmk_create_vds2.ps1:11 char:9

                +         $result = $esxcli.network.ip.netstack.add.invoke(@{netstack = ...

                +         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                    + CategoryInfo          : OperationStopped: (:) [], ViError

                    + FullyQualifiedErrorId : VMware.VimAutomation.Sdk.Types.V1.ErrorHandling.VimException.ViError

                 

                Exception calling "AddVirtualNic" with "2" argument(s): "A specified parameter was not correct: PortConnection.portgroupKey. DVPortgroup  cannot be found"

                At C:\vmk_create_vds2.ps1:33 char:9

                +         $result = $_this.AddVirtualNic('', $nic)

                +         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException

                    + FullyQualifiedErrorId : VimException

                 

                foreach ($line in $csv){
                Write-Host "Creating vMotion VMKernel Port for" $($line.Host)"..."
                $esxiserver = Get-VMHost $line.Host
                        $mtu = '9000'
                        $stack = 'vmotion'
                              
                        ## Create vMotion Netstack ##
                        $esxcli = Get-EsxCli -VMHost $esxiserver -V2
                        $result = $esxcli.network.ip.netstack.add.invoke(@{netstack = $stack})
                       
                        ## Create vMotion VMKernel Portgroup ##
                        $portgoup = $line.vMoPG
                        $nic = New-Object VMware.Vim.HostVirtualNicSpec
                $distributedVirtualPort = New-Object VMware.Vim.DistributedVirtualSwitchPortConnection
                        $nic.distributedVirtualPort = $distributedVirtualPort
                        $nic.distributedVirtualPort.portgroupKey = $portgoup.key
                        $nic.distributedVirtualPort.switchUuid = $portgoup.VirtualSwitch.key
                        $nic.netStackInstanceKey = $stack
                        $nic.mtu = $mtu
                        $ip = New-Object VMware.Vim.HostIpConfig
                        $ip.subnetMask = $line.vMoMask
                        $ip.ipAddress = $line.vMoIP
                        $ip.dhcp = $false
                        $ip.ipV6Config = New-Object VMware.Vim.HostIpConfigIpV6AddressConfiguration
                        $ip.ipV6Config.dhcpV6Enabled = $false
                        $ip.ipV6Config.autoConfigurationEnabled = $false
                        #$ip.IpV6Config = $ipV6Config
                        $nic.Ip = $ip
                        $networkSystem = $esxiserver.ExtensionData.configManager.NetworkSystem
                        $_this = Get-view -Id ($networkSystem.Type + "-" + $networkSystem.Value)
                        $result = $_this.AddVirtualNic('', $nic)
                }
                

                 

                 

                btw, I'm sure it's driving you nuts - but how do I make my replied formatted in PS instead of plain?

                • 5. Re: TCP/IP Stack
                  LucD Guru
                  User ModeratorsvExpertCommunity Warriors

                  Just to make sure I get the question correctly, you want to assign the vMotion netstack to the new VMKernel you created?

                  • 6. Re: TCP/IP Stack
                    barnette08 Hot Shot

                    Correct

                    • 7. Re: TCP/IP Stack
                      LucD Guru
                      Community WarriorsvExpertUser Moderators

                      I can't seem to find a cmdlet or API to change the stack on an existing VMKernel, but you can create one with esxcli, directly with the correct stack.

                      Something like this.
                      Note that the PortGroup has to exist.

                       

                       

                      $esxName = 'esx1.local.lab'

                       

                      $esx = Get-VMHost -Name $esxName

                      $esxcli = Get-EsxCli -VMHost $esx -V2

                       

                      $sVmk = @{

                          interfacename = 'vmk2'

                          portgroupname = 'PG1'

                          netstack = 'vmotion'

                      }

                      $esxcli.network.ip.interface.add.Invoke($sVmk)

                       

                      • 8. Re: TCP/IP Stack
                        barnette08 Hot Shot

                        This assumes vswitch not vds I assume?  Here is the error I get, but I think it's because I'm giving it a dvPortgroup.

                         

                        Message: Invalid vswitch port group name esx_vmotion. Error was: Unable to get node: Sysinfo error: Not foundSee VMkernel log for details.;

                        InnerText: Invalid vswitch port group name esx_vmotion. Error was: Unable to get node: Sysinfo error: Not foundSee VMkernel log for details.EsxCLI.CLIFault.summary

                        At C:\vmk_create_vds2-testing.ps1:11 char:1

                        + $esxcli.network.ip.interface.add.Invoke($sVmk)

                        + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                            + CategoryInfo          : OperationStopped: (:) [], ViError

                            + FullyQualifiedErrorId : VMware.VimAutomation.Sdk.Types.V1.ErrorHandling.VimException.ViError

                        • 9. Re: TCP/IP Stack
                          LucD Guru
                          vExpertUser ModeratorsCommunity Warriors

                          The example was indeed for a VSS, but it should also work with a VDS.

                          You'll have to add additional parameter dvsname, and possibly dvportid.

                           

                          Name                           Value                                                                                                               -----                           -----                                                                                                             interfacename                  Unset, ([string], optional)                                                                                         portgroupname                  Unset, ([string], optional)                                                                                         dvportid                       Unset, ([string], optional)                                                                                         mtu                            Unset, ([long], optional)                                                                                           macaddress                     Unset, ([string], optional)                                                                                         netstack                       Unset, ([string], optional)                                                                                         dvsname                        Unset, ([string], optional)

                          • 10. Re: TCP/IP Stack
                            barnette08 Hot Shot

                            I'll dig into that, thanks!