ESXI 6.5 help adding new NIC in new vmware machine

[0p3nS0urc3]

Hi Guys

i am having a strange issue, i have installed Esxi 6.5 Dell EMC version on a Dell R900

initially it was installed the system on a Quadro Intell 1000Mbps NIC card, and its running fine on all the VMs

But we had the need to install a new Fiber NIC SFP+ which we accquired long time ago from Myricom manufacturer.

We have installed the manufacturer VIB files on ESXI via ssh and everything went ok, so we can see the New NIC on the system, and we have managed to add this new NIC to the vswitch.

our problem is we cannot select this new NIC when we try to create a new VM, it only displays the old nic model from intel gigabit..

so i tried the following..

1- create new port group

2- created a new switch called vSwitch1

3- inside the new vSwitch1 i have added a new uplink card, and i have selected from the list of available nics the 10gbps SFP+ myricom NIC available which was already UP connected with teh SFP+ cable to our SFP+ switch it linked ok as 10000mbps UP card.

4- created a new vmKernelNic1 and inside this vmKernelSwitch1 i have set the static IP xxxxxxxx, with vlan id 0,  after setting this Up , we have connection, i can ping the ipv4 xxxx gateway, and from the gateway i can ping back to the NIC ipv4 static ip setup ok.

5- on the TCP/IP stack i left it default, i can see it has both vmkernelswitch inside with both static IPs.. aldo the first main intel NIC.. it shows the gateway , but on the new NIC card, displays gateway 0.0.0.0

6- now when i click on create new VMWARE machine,  on the Network adapter options i have the following cenario  2 options, first and initial  VM network (which contains the intel  E1000 and E1000E and VMXNET3 (card types available in the options)

and i also have the new VM network2 (which is the portgroup and new vswitch1 and vkmnic1 configurations i have just done for the new 10g myricom NIC card) aldo when i click on the adavanced tab from the Network adapter options it only displays the same exact options on the first NIC which was used from intel when the system was installed.. (which contains the intel  E1000 and E1000E and VMXNET3 (card types available in the options)

i cannot make it display the new NIC card in order to work from the new NIC or from a Virtual adapter nic pulling connections out of the 10Gbps SFP+ network card

and after installed the OS.. it has not internet conectivity, aldo i can ping on the IPV4  IP from outside our network because it is a Public IPV4 static setup on the interface.. so it means that Esxi has the NIC working as i can ping to it and outside.. but i cannot force it to use this new NIC on the new VMWARE machines we are trying to install

Shurely i am missing a step somewhere in order to make it work properly.

If anyone has any suggestions or had the same problem before, i would appreciate some comments

Thanks in advance

0p3nS0urc3

[amohammadimir]

Virtual network adapter available to attach to the virtual machines are virtual hardware which emulates real existing physical network adapters. Network adapter choices depend on the version number and the guest operating system running on the virtual machine such as E1000, E1000e, VMXNET3 (Note that the physical network cards in the physical ESXi host is unrelated.)

So, if you've added the newly installed physical adapters to a new virtual switch,created a port group on that virtual switch and your virtual machine is connected to that you are just fine.

[scott28tt]

Foto4 is the key to your problem.

Is it only the VMs that you wanted to use the new pNIC, or do you need some vmkernel function (eg. management, vMotion) to use it too?

If only VMs, you did not need to create the new vmk interface - doing that meant that the port group “VM Network 2” could not be used by VMs as a port group can only be used either by a vmk OR VMs, not both. Delete the vmk interface and you will be able to give the VM NICs a connection to “VM Network 2”.

If you need the vmkernel to also use that pNIC, you need to create an additional port group on the vSwitch which you can use with VM NICs, I would probably also rename “VM Network 2” to something else seeing as you would not be using it for VMs anyway.

You also have a port group called ”Management Network 2” to which you have a VM NIC connected.

[0p3nS0urc3]

Hi Amir

Thanks for your input i will mais further testing today

[0p3nS0urc3]

hi scott28tt

Thanks for your input, indeed i only need to add, the New physical interface, to the vm machine.

Só um that case de vkmnic1 where i have setup the New ipv4 i van delete It? And do i need to add the New physical nic on the old vswitch0 or do i leave the New vswitch1 and New port group?

I Will make further testing based on your reply and Will post the outcome result here.

Thanks a Lot for your input

[scott28tt]

I can’t answer both of your questions as you don’t say what the pNICs are physically connected into - the same L2 network or different L2 networks. That would influence whether you should be adding vmnic1 to your first vSwitch or keeping it as the uplink for the current vSwitch.

If it is right that vmnic1 remains on the current vSwitch, then yes you can delete the additional vmknic - that would mean “VM Network 2” would revert to being available for VM NICs.

This section of the documentation should help you: vSphere Standard Switches

[0p3nS0urc3]

Hi scott28tt

initially we only had the VM Network and Management Network, when system was pre-installed OS.. and setup with the Intel Quadro NIC

all our vms are running on this NIC card wich in the system created a Portgroup called Management Network as described on PIC1 attached

i can see that vmk0 is attached to it..

then it created the VMnetwork as attached on PIC2 which i believe is the network that is being used to connect all the vms to the internet , which uses vmnic0 physical card.

when i added the new 10G fiber NIC.. and installed the drivers i tried adding this new 10g NIC into the vswitch0 as an UPLINK card,  but when i was creating a new VM machine, i could not see on the adapter type the new NIC.. name.

So i tought i would need to create a New "Port group, vmknic, vSwitch1, and so on...."   but i will try to remove everything and test again

[0p3nS0urc3]

Ok so i did further testing

i have deleted the vkmnic1 which was pointed on IP 190.8.xxxxxx

just left the new portgroup VM NETWORK 2 attached to the new vSwitch1 and on the vSwitch1 left the physical NIC 1000mbps selected..

on the VM machine when i select adpter settings it displays both port groups againa  VM network and VM Network 2 , i have selected VM Network2  but on the adapter type inside the menu of VM Network 2, it only displays the same old Adapters from VM Network also..

does not show the new NIC

as attached on the pic below.

[0p3nS0urc3]

Hi

i forgot to add on previous reply,  indeed all the Physical nics are on separate networks..

NIC copper  eth 1gbps are on one IP network 10.xx.xx.xx.x

and NIC fiber 10gbps are connected on another switch L2  network ip 190.8.xxxxxx on differnt ports not the same switch.

[0p3nS0urc3]

Hi amirmohammadimir

indeed i have added the new physical NIC to a new vswitch1 and new portgroup, but i cannot select the nic on the adapter type when trying to install a new VM it only displays the adapter types from the first vswitch.

in the adpter settings i can see the new port group and select new port group´, byt inside adapter type options.. i cannot see the new NIC 10gbps in order to select it.

[IRIX201110141]

I dont have read all posts but for me it sounds like a Layer8 problem You think way to complicated......

1. You create a vSwitch with a VM Portgroup called Foobar
2. From the right you add a VMNIC (the phys. NIC you installed together with the driver(vib)) to the vSwitch as a uplink
3. From the left you connect a VM with its vNIC to the Portgroup called Foobar
4. The packets flow from the GuestOS -> vNIC -> PortGroup -> vSwitch -> Uplink -> VMNIC

A VM always use a virtual Intel or VMware VMXNET3 from the GuestOS perspective. This is a virtual 10G nic when choosing VMXNET3. The VM never see or use the phys. VMNIC directly and this is the reason why you dont see it when configuring the VM.

The Portgroup "works" as abstraction level to decouple the VM from the pyhs. Network devices.

Spend some time and take a look to The Great vSwitch Debate – Part 1 | Ken's Virtual Reality

Regards,
Joerg

[0p3nS0urc3 · Accepted solution]

Hi guys..

problem fixed it turned out that VMNX3 device does recognize the 10gbps card after it boots windows with the net card adapter selected. i have configured the ipv4 and now we have it working with good results now i need to check my ISP connection as i am only getting 2gbps down and 4gbps upload, its meant to be 10gbps dedicated.