Hi all,
we are using linux debian guest on vmware ESX, and this linux guest is setup to be a router for other linux servers with iptables. Now the prolem is as follows: when we try to do scp of ftp file transfer from this linux guest to any other host, the transfer is working fine. But when we try to do a transfer from some other linux machine through the linux router (and through iptables) the connection is dropped as soon as the transfer starts. SSH connection from other linux machines through the linux router also drops if you try to less a large file with error "connection reset by peer".
I know that this sounds like MTU issue but all network cards have MTU set to 1500, and we tried iptables option clamp-mss-to-pmtu with no luck. iptables does not have any other entries other then masquerade on outgoing interface.
We are quite sure that this problem is somehow connected to ESX, because this is a new setup. Before this we had exactly the same setup but with router installed on phisycal machine. When we migrated this router to ESX, the problems started to apear. ESX network card is connected to the switch with trunk, and router's virtual network card is set to vlanXX.
Right before the connection is dropped, tcpdump shows a lot of "TCP dup ACK" packages, and on the other side "TCP ACKed lost segment". Any help is appreciated.
Please elaborate on how you have things configured. What does the virtual network look like?
What I find very surprising is that no guests seems to be connected to 2 differents portgroups.
In your screenshot, what's the name of your Linux router ? Which one is you "public" VLAN ? Which one is the private ?
The more information you give, the more chance we have to help you solve your problem.
All traffic goes through the same virtual network card. The machine that is the router is called ldap on the screenshot.
So, your router is connecter only to one VLAN. But, it uses 2 differents IP addess on 2 differents on the same VLAN (obvously as it's a router) ...
On your screenshot, which VM are supposed to be on the private VLAN and must be routed through your router "ldap" ?
Other VM guests on this ESX go directly out on the net, but other "physical" servers use "ldap" to send traffic through it.