VM -> small buffer / rx ring buffer size

[VM_Niels]

Same idea as ESXi - buffers / ring size but for VM's.

Any idea how I can find/see the data of buffer and ring size of network card of VM with powershell?

Same idea as vsish but I would run on all of ESXi-hosts from 1 script.

[LucD]

Isn't that something that would need to be done inside the Guest OS?

[VM_Niels]

No, i don't want to know the settings within the Guest OS.

I want to know which VM's have running out of buffers.

See also: https://vmzoneblog.com/2020/04/02/monitoring-vmxnet3-ring-buffer-full-condition/ ->

Monitoring  if a VM is hitting buffer full condition

[LucD]

Why are you shouting?

[VM_Niels]

I am shouting?

Sorry? I didn't mean that.

I am trying:
https://communities.vmware.com/t5/VMware-PowerCLI-Discussions/PowerCLI-equivalent-of-net-stats-l-in-...

But I must first update the Powershell to newer version (SSH implemented)

[LucD]

In the same way (Posh-Ssh module) you could use the vsish command.
I'm testing some code, hold on.

PS: I thought the bigger font size in your last line in the previous reply indicated some aggravation 😉 

[LucD]

With a combination of Get-EsxCli and the Posh-SSH module I seem to be able to get the information.
Something like this

$esxName = 'MyEsx'
$user = 'root'
$pswd = 'VMware1!'

$cmd = @'
vsish -e cat /net/portsets/$($_.vSwitch)/ports/$($_.PortID)/vmxnet3/rxSummary
'@

$esx = Get-VMHost -Name $esxName
$cred = New-Object -TypeName PSCredential -ArgumentList $user,(ConvertTo-SecureString -String $pswd -AsPlainText -Force)

$ssh = Get-VMHostService -VMHost $esx | Where-Object { $_.Label -eq 'SSH' }
if (-not $ssh.Running) {
  Start-VMHostService -HostService $ssh -Confirm:$false | Out-Null
}

$session = New-SSHSession -ComputerName $esx.Name -Credential $cred -AcceptKey

$esxcli = Get-EsxCli -VMHost $esx -V2
$esxcli.network.vm.list.Invoke() |
ForEach-Object -Process {
  $vm = $_
  $nics = Get-NetworkAdapter -VM $vm.Name
  $esxcli.network.vm.port.list.Invoke(@{worldid = "$($vm.WorldID)"}) |
  ForEach-Object -Process {
    $port = $_
    $nic = $nics | where{$_.MacAddress -eq $port.MACAddress}
    if($nic.Type -eq 'vmxnet3'){
      $result = Invoke-SSHCommand -SSHSession $session -Command $ExecutionContext.InvokeCommand.ExpandString($cmd)
      if($result.ExitStatus -eq 0){
        New-Object -TypeName PSObject -Property ([ordered]@{
          VM = $vm.Name
          Nic = $nic.Name
          OutOfBuffers = ($result.Output | Where-Object { $_ -match 'running out of buffers:' }).Split(':')[1]
          Ring1 = ($result.Output | Where-Object { $_ -match '1st ring is full:' }).Split(':')[1]
          Ring2 = ($result.Output | Where-Object { $_ -match '2nd ring is full:' }).Split(':')[1]
        })
      }
      else{
        Write-Host "vsish call on $($vm.Name) for vNIC $($nic.Name) failed with exitcode $($result.ExitStatus)"
      }
    }
    else{
      Write-Host "$($nic.Name) on $($vm.Name) is type $($nic.Type)"
    }
  }
}

Remove-SSHSession -SSHSession $session | Out-Null

if (-not $ssh.Running) {
  Stop-VMHostService -HostService $ssh -Confirm:$false | Out-Null
}

[VM_Niels]

Thank you.

After I installed the Posh-SSH module, I am investigate/testing the script. (in progress)

I see that the output is maybe incorrect.
- 1 VM have 1 networkcard, but the output of script show 2 network cards

$nics = Get-NetworkAdapter -VM $vm.Name

Output:
PS C:\WINDOWS\system32> $vm

Name Networks NumPorts WorldID
---- -------- -------- -------
VM-NAME {dvportgroup-1814} 1 4343181

PS C:\WINDOWS\system32> $nics = Get-NetworkAdapter -VM $vm.Name

PS C:\WINDOWS\system32> $nics

Name Type NetworkName MacAddress WakeOnLan
Enabled
---- ---- ----------- ---------- ---------
Network adapter 1 Vmxnet3 XXXX I... 00:50:56:9f:7d:11 True
Network adapter 1 Vmxnet3 XXXX ... 00:50:56:9f:7d:11 True

[LucD]

Can't replicate that I'm afraid.

Could it be that you have 2 VCSA connections open?
Check what is in $global:defaultVIServers

[VM_Niels]

Correct.
I have fixed this. I have now 1 connection.
But I have now this errors (I am investigate this):

vsish call on <VM-NAME> for vNIC Network adapter 1 failed with exitcode 2

I get that with all VMs.

Edit: I found command.

I missed $($_.vSwitch)
$cmd = @'
vsish -e cat /net/portsets/$($_.vSwitch)/ports/$($_.PortID)/vmxnet3/rxSummary


So, I have changed temporarily to correct name of switch.
It works.
I will investigate this issue.



I am now investigate for next steps:
- Output to file (Name / NIC / Outofbuffers / Ring1 / Ring2)

[LucD]

You could export to a CSV.

} | Export-Csv -Path .\report.csv -NoTypeInformation -UseCulture

Remove-SSHSession -SSHSession $session | Out-Null

[VM_Niels]

Thanks.
I am investigate to change the output. I want to notice all good information and errors in 1 output (report.csv).

And I am trying to change the script, so I want to connect the vcenter, and connect every ESXi-host to get the informaton about buffers.

[LucD]

The errors are currently written to the console.
To have that information in the CSV you would need to create an extra property in the object.

[VM_Niels]

Hello,

About this, special -> $_.vSwitch :

vsish -e cat /net/portsets/$($_.vSwitch)/ports/$($_.PortID)/vmxnet3/rxSummary

 
If I run that code with vsish

My output is different:

/net/portsets/> ls
DvsPortset-2/
DvsPortset-0/
DvsPortset-1/
DvsPortset-3/

 
The output of esxli shows this:

esxcli network vm list
World ID  Name          Num Ports  Networks
--------  ------------  ---------  --------
 5711222  xxxxxx              1  dvportgroup-864756
 5710184  xxxxxx              2  dvportgroup-152, dvportgroup-126
 5709958  xxxxxx              1  dvportgroup-115
 5709638  xxxxxx              1  dvportgroup-1820
 5707147  xxxxxx              3  dvportgroup-115, dvportgroup-115, dvportgroup-115
 5706943  xxxxxx              1  dvportgroup-35527

I haven't found a relationship yet between DvsPortset-<x> and dvportgroup-<x>

Maybe it is better to work with net-stats -l on esxi-host?

[LucD]

That variable $_.vSwitch is replaced with the actual vSwitch name through

$ExecutionContext.InvokeCommand.ExpandString($cmd)

If you want to tackle your original question differently that is your choice.
I only provided what you asked.

[VCPJOHN20111014]

Great stuff here, but there seems to be an issue?

$port.vSwitch  returns the name shown in vcenter like VDS_Switch1

the vsish  command wants the switch name as the internal name like DvsPortset-0

Am I missing something here?  We're on VSphere 7