That's what I did.
vlan 1 - default vlan, Untag (U) on all 24 ports
vlan 100 - Tag (T) on Port 11 and Port 12 that connect to ESX Host for outgoing VM to Internet, Port 1 is the common port which should be set to UnTag as it's used to go out to Internet.
So say for a VM is leaving the port group vlan 100, it will be tagged, and the receiving Netgear Switch Port 11 & 12 will understand this vlanid=100 as both ports are tagged, then the tagged frame will go to Port 1 untagged (stripped off by untag Port 1) and then it should be able to go out.
If you look my vlan.doc, you will see normal VLAN Mode EST (ie, without tag) is on vlan 3, all those VM on vlan 3 has no problem going out.
I am pretty sure it's a configure error on Netgear that blocks the VST mode 802.1q VLAN from working.
For example, consider the organization whose servers plug into distribution layer switches. These distribution layer switches then connect to a core switch. If the connections between the core switch and the distribution switch are not already configured as VLAN trunks, i.e., are capable of carrying multiple VLANs simultaneously, then using VST is impossible. Each of the distribution switches only carries a single VLAN and is only capable of carrying a single VLAN.
I found the above during google, could this be the reason why my VST doesn't work?
I thought I don't need to get uplink Port 1 (connect to data center's core switch) to have the capability of VLAN trunks.
ie, 802.1Q VLAN only happens in Port 11 and Port 12, after the traffice leaves Port 11 & 12, then it will go to Port 1 which is untagged, then go out to core switch then to the Internet, no?
Sorry, seemed I am answering my own question, but I just want to post my finding to help others who may encounter this strange problem in the future.
I did a test by setting a private IP 10.0.18.10 on VM1 on ESX Host 1 on vlan 10, then do the same for VM2 on ESX Host 2 on vlan 10.
Guess what? They can poing each other!
To future prove my original Netgear VLAN setting is correct, I did the following tests
test 1. Change vlan 10 to vlan 20 on ESX Host 1, now VM1 cannot ping VM2, so original VLAN tagging or 802.1q is working indeed!
test 2. Change Netgear Port 11 & Port 12 (both on ESX Host 1) to Untag, now VM1 cannot ping VM2, so original VLAN tagging or 802.1q is working indeed!
So why public IP doesn't work? I am pretty sure it's the link between core switch and my Netgear ISN'T SET TO VLAN TRUNK, so I will ask them to do so, but I suspect they won't allow me. (During google, I saw a topic regarding VLAN Trunk outweights security? and vice vrsa, I think that's the reason my data center may not allow me to do so)
If you're using VST, I believe what you need to do is trunk the Netgear ports on your switch. They should not be tagged as access ports.
So, for example your Switch will have ports 11 and 12 if I understand trunked. Then you'll set the VLAN ID on your port group to be VLAN 100, or whatever it needs to be. This is all that is necessary for VST.