Yes, setting "Notify Switches" to No is a bad thing.
Here's why... Your VMware hosts are connected to physical network switches (hopefully at least two different ones for redundancy). Those switches have MAC address forwarding tables. The tables associate each MAC address to a physical switch port. When a frame comes in, the switch will look up the destination MAC address in the table and decide which physical port to send the frame to.
If a NIC failover occurs, your VMs will be talking through a different physical NIC. That also means a different port on the physical network switches. If you don't notify the network switches that something has changed they will continue to look up the old information from the tables and send the frames to the old port. Because the virtual machine is no longer at the old port, it won't get the frames. You will lose network connectivity.
That can continue for a little while - anywhere from a couple of seconds to a minute or two, until the switches update their tables. For more details, look up gratuitous ARP.
If you are running NLB, you need to do one of two things:
1) Set aside two physical NICs in each and every host, and dedicate those to NLB. Create a new vSwitch, assign the dedicated NLB NICs to this new vSwitch, and make sure your NLB servers use that switch only. This vSwitch using NLB will the only one where Notify Swtiches is set to No.
2) For large installs, if you have many NLB servers, you can just set aside two or three hosts and makes them a separate cluster. You can turn off HA in that cluster if you want. Then you can set Notify Switches to No and not worry about affecting the rest of your VMs.
Hopefully this helps!
Awesome information. Thank you for the quick answer.
There you go again PacketRacer giving incomplete answers. Just create a port group for NLB and you will be half way there.
and set notify switches to NO. Put your NLB vm in that port group.