The rule is that it must be set to both at the same value.
1500 - 1500
9000 - 9000
The MTU size can be very important because it determines packet fragments on the network. If your chunk size is 8KB and the MTU is 1500, it will take six Ethernet frames to transmit the 8KB. If you increase the MTU to 9000 (9,000 bytes), the number of Ethernet frames drops to one.
The most common recommendation for better NFS performance is to set the MTU on both the NFS server and the NFS client to 9000 if the underlying network can accommodate it.
The MTU is whatever is set on the vmkernel and vswitch.