witch switches must use 1550 MTU ?

[hamidja]

Hi Dears 

I know we  must setup switches with at least 1550 MTU because of VXLan. But we cannot setup all switches in our network. What happen if we setup some switches only not all of them. Does they works together. Is it possible to setup MTU just for some Vlans or ports?

[shank89]

Because you mentioned VXLAN, I assume you are running NSXv?  The recommended configuration is setting jumbo frames to 9k and at minimum 1600.

This will at minimum need to be configured on all ToR's and switches that your encapsulated traffic will pass.  The answer to this question is dependant on your design.  IE, wherever your host transport nodes are connected to and where your edges sit.

Depending on what hardware you are running, MTU can be specified globally, on a per interface level or VLAN.

[hamidja]

Thank you Shank

what are TORs?

what happen if change MTU to 9000 in my core switch (that all hosts and floor switches are connected to) and VDS but servers and access switches use 1500 MTU?

[FengyuanLiu]

TOP switch is the switch at the top of the cabinet, that is, the switch that connects all servers in the cabinet Modify the MTU of the switch specifically, depending on the switch you use for the VDS connection of NSXV to connect to the network card. You need to modify the MTU of the switch interconnected with the uplink of the VDS of NSXV. It is recommended to change it to 9K, at least 1650 or more. can

[hamidja]

Thank you FengyuanLiu

does it need to set on Trunk port? i saw somewhere trunk port use jumbo frames

[shank89]

In general yes they should be trunked, as the links are usually passing VLANs for physical uplink for edges, host teps and management.

As for the MTU question, the reason 9k or a consistent MTU configuration across all switches is recommended is to avoid packet fragmentation.  For example, if your host TEPs need to communicate with Edge TEPs (at least in NSX-T).  In your case they would have to go past the top of rack switch (set to MTU 1500) and would be fragmented before reaching the core.  GEVENE and i'm pretty sure VXLAN packets do not play well with being fragmented.  Ontop of that you will also have to think about if you are using jumbo frames for storage and vmotion.

[hamidja]

Hi dear Shank

i understood why we need to use jumbo frame. I can not change just some Interface MTU and i have to change it in system global level. Now my question is what happened for my devices that connected to this switch but didnot use jumbo frame. in other word device managment ports ,floor access switches trunk ports that use MTU 1500 but connected to a port with MTU 9000. what happened to them?do they work correctly? what about routing MTU does it need to be changed?

Thank you

[shank89 · Accepted solution]

Spoiler

They will still work fine,  its perfectly fine to have smaller frame sizes passing a switch that supports a large mtu