Issue with DNS in Windows 2008 domain and ESX 3.5

[jseda]

Hello!

We have a big troble with running DNS on ESX in Windows 2008 domain. We have installed a clear and new DC in ESX (from iso files) - Win2k8 Standard edition. Domain was created by dcpromo as usually. Then we have installed 2 other ones windows 2008 standard servers into ESX too. Those were added into first server. There were no configuration changes till this moment, just adding those servers into domain.

We are able to ping between those servers, share files, run authentication and run IIS when using IP addressing. But nslookup doesn't work at all (we are doing tcpdump and we can see some calls to m.server-names.net servers - but we don't know where do they come from). So when running ping from any other server then DC, then it's fine. When running nslookup to any record in DNS then we'll get timeouts.

When removing those servers from domains then we'll get first answer from DNS as timeout (there are those m.server-names.net packets) and then we'll get the answer from DC DNS - the correct one. But ONLY when this server is not in a domain. using telnet we can connect to DC DNS from outside and it works fast and fine.

1:1 the same configuration as I wrote about above is working in physical configuration, in hyper-v and even in vmware server. But we have to run it in ESX environment and not it seems there is something hidden in ESX server and we are not able to configure it at all.

Anyone has any idea or experience with Windows 2008 DC in ESX? Thanks a lot for any idea and advice.

[jseda]

Just to make it clear:

1) Configuration: DC + win2k8A + win2k8B all in one domain - ping works, netbios works, nslookup doesnt work

2) Configuration: (Domain = DC + win2k8IISA) + Win2k8B - from server win2k8A i can ping but not able to nslookup, server win2k8B is able to ping and nslookup but with one timeout when searching for some m.server-names.net servers and then i'll get answer from DC DNS).

Is there any internal routing of DNS calls? How does vswitch is working with those requests?

Thanks a lot.

[Lightbulb]

1.Are all systems on same subnet?

2. Are all systems set with static IPs?

3. Are systems set with IP of DC for DNS?

4. Are all systems on same vswitch?

[jseda]

Hello!

1) yes, they are. But it doesn't matter. We tested it on 192.168.6.0 and 7.0 networks. with the same results I wrote about.

2) Yes, all servers.

3) Yes, correct one - the DC DNS.

4) Yes, we tested it on the same one vswitch and different vswitches too. The same as point 1.

[jseda]

Now we are testing it to install whole environment without VMWare network driver. We'll install VMWare tools but without a driver. Maybe the driver is not well tested with Win2k8 domains.

Update: After the tests we've got the same results. Servers outside of domain are getting the answer after one timeout. Servers inside of a domain are not able to nslookup at all. All were default installs with static IPs.