VMware Communities
Manouchehri
Contributor
Contributor
Jump to solution

No DNS in NAT (vmnet-natd broken)

As of glibc 2.22, vmnet-natd is unable to look up external results with the default settings.

Temporary workaround

Open /etc/vmware/vmnet8/dhcpd/dhcpd.conf and change:

option domain-name-servers 172.20.142.2;

to

option domain-name-servers 8.8.8.8;

Bug Reports

Debugging

The issue seems to be with vmnet-natd not using res_init.c properly.

https://sourceware.org/git/gitweb.cgi?p=glibc.git;a=history;f=resolv/res_init.c;hb=5e7fdabd7df1fc6c5...

My guess is that these are the changes that broke it.

https://sourceware.org/git/gitweb.cgi?p=glibc.git;a=blobdiff;f=resolv/res_init.c;h=02590fb1fdbb15c9f...

There's roughly only a dozen calls where res_init.c is being used, so it shouldn't be too difficult to debug. The only issue is that vmnet-natd is stripped; if someone could send me the symbols or a non-stripped binary, that would be great.

ltrace -f -p 1337 # vmmnet-natd PID here

2020 strncpy(0x7ffff83451e0, "ctftime.org", 64) = 0x7ffff83451e0

2020 malloc(16) = 0x2644cb0

2020 malloc(168) = 0x2644380

2020 pipe(0x7ffff83451d0) = 0

2020 fcntl(6, 3, 0x429280, 0x7ff3f92482e7) = 0

2020 fcntl(6, 4, 2048, 0x7ff3f97221a2) = 0

2020 malloc(65884) = 0x2654640

2020 strncpy(0x26443e8, "ctftime.org", 64) = 0x26443e8

2020 pthread_create(0x7ffff83451c8, 0, 0x428ba0, 0x2644cb0) = 0

10054 mempcpy(0x7ff3f2aeebb0, 0x26449c6, 8, 7) = 0x7ff3f2aeebb8

2020 pthread_detach(0x7ff3f2af0700, 0x7ff3f2aeffb0, 0, 0x800000 <unfinished ...>

10054 mempcpy(0x7ff3f2aeebb8, 0x26449ce, 4, 3 <unfinished ...>

2020 <... pthread_detach resumed> ) = 0

10054 <... mempcpy resumed> ) = 0x7ff3f2aeebbc

2020 poll(0x26121b0, 6, 1000, 0x80412 <unfinished ...>

10054 __res_ninit(0x7ff3f2aefca0, 103, 11, 0 <unfinished ...>

2020 <... poll resumed> ) = 1

2020 gettimeofday(0x7ffff834d390, 0) = 0

2020 read(3, "", 32768) = 71

2020 malloc(199) = 0x2644430

2020 memcpy(0x26444b0, "\0PV\3604\330\0\f)\335\0049\b\0E\0\09]\241\0\0\200\021\327:\300\250B\205\300\250"..., 71) = 0x26444b0

2020 mempcpy(0x7ffff8344b80, 0x26444e6, 8, 7 <unfinished ...>

10054 <... __res_ninit resumed> ) = 0

2020 <... mempcpy resumed> ) = 0x7ffff8344b88

10054 __res_iclose(0x7ff3f2aefca0, 0, 2, 0 <unfinished ...>

2020 mempcpy(0x7ffff8344b88, 0x26444ee, 4, 3 <unfinished ...>

10054 <... __res_iclose resumed> ) = 0

10054 __errno_location() = 0x7ff3f2af0680

10054 __res_nclose(0x7ff3f2aefca0, 0, 0, 0) = 0xffffffff

10054 __errno_location() = 0x7ff3f2af0680

10054 memcpy(0x265473c, "o\0\0\0E\0\09]\240\0\0\200\021\327;\300\250B\205\300\250B\002\344j\05\0%\210\237"..., 61) = 0x265473c

10054 write(10, "\001", 1) = 1

10054 close(10) = 0

10054 free(0x2644cb0) = <void>

10054 +++ exited (status 0) +++

2020 <... mempcpy resumed> ) = 0x7ffff8344b8c

0 Kudos
1 Solution

Accepted Solutions
dariusd
VMware Employee
VMware Employee
Jump to solution

Workstation 12.1.1 has just been released, and the problem you've encountered seems to be listed as a resolved issue.  Try installing Workstation 12.1.1 and let us know if the problem is resolved for you.

VMware Workstation 12 Pro Version 12.1.1 Release Notes

Download VMware Workstation Pro

Cheers,

--

Darius

View solution in original post

0 Kudos
11 Replies
raiford
Enthusiast
Enthusiast
Jump to solution

I am experiencing this same problem.  I have found many other reports of this online, but no solution yet.  Using an external DNS works, but for those of us who move between networks, this is not a good option.  Using a Google DNS server will prevent me from accessing local servers.  Hardcoding my local DNS server at work will not work at home, etc.

I have not found any reference to VMware acknowledging that the issue exists.

EDIT:

I'm running Ubuntu 16.04 LTS and VMware Workstation Pro 12.1.0 build-3272444.  When the new 16.04 LTS is officially released at the end of this month this issue will likely affect a lot more people.  Hopefully VMware can fix it beforehand.

0 Kudos
Manouchehri
Contributor
Contributor
Jump to solution

Sooner or later this is going to hit all Workstation users on Linux. I think it would be wise to get on top of this before it becomes a widespread critical issue.

I'd be quite happy to assist VMware in troubleshooting more, but I'm not interested in paying for a support license to give free work. If any developers are reading this, shoot me an email and I'll send my number (Google'ing my name will give contact info).

0 Kudos
www23456
Contributor
Contributor
Jump to solution

Are there any entries in /etc/resolv.conf? I tested the xenial.iso u1604 hsot with WS12.1.0, VM is linux, and there is 127.0.1.1 added automatically by

network manager: netstat -an |grep 127; 127.0.1.1:53 LISTEN

0 Kudos
Manouchehri
Contributor
Contributor
Jump to solution

127.0.1.1:53 inside the guest is correct. I'm using that in other systems and it's fine.

From the ltrace I posted, it appears that the vmnet driver on the host might be the issue. Changing domain-name-servers on the host only fixes the bug for me, which implies it's the host.

Have you tried using my suggested temporary fix?

I'm tempted to try to use VirtualBox's NAT server with VMware, not sure how well that's going to work though.

0 Kudos
www23456
Contributor
Contributor
Jump to solution

actually the ubuntu1604 beta2 host nat working fine with linux VM.  On my host, the 127.0.1.1 was added

automatically by network manager and there is 127.0.1.1:53 tcp in 'listen'state. Basically  the nat will

forward the dns query to 127.0.1.1:53 and get response from the local 127.0.1.1 resolver and forward back

to VM.

on my host: # ldd --version showing glibc 2.23,

Are you using window VM?  what is your glibc version?

0 Kudos
Manouchehri
Contributor
Contributor
Jump to solution

All of us are using Linux as our host (Windows does not use glibc). What is your /etc/resolv.conf and /etc/vmware/vmnet8/dhcpd/dhcpd.conf on your host machine?

Both Linux & Windows guests are broken for most of us. With Linux guests, attempting a ping will eventually result in VMware returning the correct IP (but by appending .localdomain). Using dig or nslookup without .localdomain will always fail.

0 Kudos
raiford
Enthusiast
Enthusiast
Jump to solution

I was hoping VMWare would acknowledge this issue but it seems they either don't recognize it or have decided it is not important.  I went ahead and bit the bullet and logged an official support issue for this.  I have referenced this thread since it has the best description of the error.  Hopefully others who have additional information will also post here rather in other threads so we can keep the information gathered in one place.

0 Kudos
raiford
Enthusiast
Enthusiast
Jump to solution

FYI, my ticket has been archived awaiting the next maintenance release.  I've been told "The information is already there with Engineering team and they are looking into the issue.", so hopefully we will see this resolved sooner than later.

0 Kudos
dariusd
VMware Employee
VMware Employee
Jump to solution

Workstation 12.1.1 has just been released, and the problem you've encountered seems to be listed as a resolved issue.  Try installing Workstation 12.1.1 and let us know if the problem is resolved for you.

VMware Workstation 12 Pro Version 12.1.1 Release Notes

Download VMware Workstation Pro

Cheers,

--

Darius

0 Kudos
Tanker_Bob
Contributor
Contributor
Jump to solution

Thanks! Version 12.1.1 fixed the NAT problem for me in a Windows 10 guest on Ubuntu 16.04 64-bit host. Nice work!

0 Kudos
raiford
Enthusiast
Enthusiast
Jump to solution

I can also confirm this update resolved my problem.  Thanks!

0 Kudos