in NSX-T there's "bare-metal" option to choose for the edge's form.
the installation guide mentioned about the networking of bare-metal edge, as shown below:
the questions are:
1. if I want to use VLAN tagged traffic for the NIC eth0 as management network, should I configure the tag directly on eth0 or the internal device fp-eth0(e.g. vlan tag 1234 device: fp-eth0.1234)?
2. will the fp-eth0 be used for both management and overlay traffic? it's a little be confusing for me because each fp-ethx interface maps to a physical NIC therefore the fp-eth0 should be mapped to eth0.
3. the installation guide 2.2 said "up to four" of the internal interfaces can be allocated (depends on phy. NICs);
does that mean the maximum physical NIC for an edge is totally 4 (excluding the management NIC)?
Accepted Solutions
Hi niceguy (again 
),
I will be talking mostly about NSX-T DC 2.4 version but as I see you mentioned 2.2.
1. if I want to use VLAN tagged traffic for the NIC eth0 as management network, should I configure the tag directly on eth0 or the internal device fp-eth0(e.g. vlan tag 1234 device: fp-eth0.1234)?
By default the eth0 and the fp-eth0 are not the same. The eth0 is for management and the fp-ethXX interfaces belong to the DPDK "domain" (fp = fastpath) to be used by the host-switch (N-VDS).
The mapping would look like (by default):
| The order of physical interfaces | What the edge sees internally |
|---|---|
| NIC0 | eth0 |
| NIC1 | fp-eth0 |
| NIC2 | fp-eth1 |
| NIC3 | fp-eth2 |
| ..and so on ... |
So yes, you can tag the eth0 from the Linux command line for whatever VLAN you want, but there is a difference using the managment traffic over the fp-ethXX (see below).
2. will the fp-eth0 be used for both management and overlay traffic? it's a little be confusing for me because each fp-ethx interface maps to a physical NIC therefore the fp-eth0 should be mapped to eth0.
Can use fp-ethXX for management if you want. See: VMware NSX-T Data Center 2.4 Release Notes
Bare Metal Edge Node Enhancements
- NSX-T 2.4 introduces the option on Bare Metal Edge node to support the management on the fast path NICs, no longer requiring a dedicated management NIC.
But in this case the "management" interface will pass through the fp-ethX interfaces, with whatever configuration they have. In case there is no available free interface for only management purposes.
3. the installation guide 2.2 said "up to four" of the internal interfaces can be allocated (depends on phy. NICs);
does that mean the maximum physical NIC for an edge is totally 4 (excluding the management NIC)?
That is the amount of the fp-ethX interfaces (excluding the management eth0) and as I see that's the recommended limit in 2.2. I am not sure now what is - if any - in 2.4. Seems it is really limited by the underlying OS/HW, but I'll check it for any public info about it.
Hi niceguy (again ),
I will be talking mostly about NSX-T DC 2.4 version but as I see you mentioned 2.2.
1. if I want to use VLAN tagged traffic for the NIC eth0 as management network, should I configure the tag directly on eth0 or the internal device fp-eth0(e.g. vlan tag 1234 device: fp-eth0.1234)?
By default the eth0 and the fp-eth0 are not the same. The eth0 is for management and the fp-ethXX interfaces belong to the DPDK "domain" (fp = fastpath) to be used by the host-switch (N-VDS).
The mapping would look like (by default):
| The order of physical interfaces | What the edge sees internally |
|---|---|
| NIC0 | eth0 |
| NIC1 | fp-eth0 |
| NIC2 | fp-eth1 |
| NIC3 | fp-eth2 |
| ..and so on ... |
So yes, you can tag the eth0 from the Linux command line for whatever VLAN you want, but there is a difference using the managment traffic over the fp-ethXX (see below).
2. will the fp-eth0 be used for both management and overlay traffic? it's a little be confusing for me because each fp-ethx interface maps to a physical NIC therefore the fp-eth0 should be mapped to eth0.
Can use fp-ethXX for management if you want. See: VMware NSX-T Data Center 2.4 Release Notes
Bare Metal Edge Node Enhancements
- NSX-T 2.4 introduces the option on Bare Metal Edge node to support the management on the fast path NICs, no longer requiring a dedicated management NIC.
But in this case the "management" interface will pass through the fp-ethX interfaces, with whatever configuration they have. In case there is no available free interface for only management purposes.
3. the installation guide 2.2 said "up to four" of the internal interfaces can be allocated (depends on phy. NICs);
does that mean the maximum physical NIC for an edge is totally 4 (excluding the management NIC)?
That is the amount of the fp-ethX interfaces (excluding the management eth0) and as I see that's the recommended limit in 2.2. I am not sure now what is - if any - in 2.4. Seems it is really limited by the underlying OS/HW, but I'll check it for any public info about it.