VMware Cloud Community
dbray925
Contributor
Contributor

Vulnerability Scans Will Crash VCSA 6.5.0.5200

Recently we have discovered an issue where a simple vulnerability scan (we are using latest OpenVAS + Full and fast ultimate scan all TCP ports) will crash the VCSA. It is killing this new service called: vmdnsd (VMware Domain Name Service)

Our setup is fairly small, 3 hosts and about 40 guests, highly under utilized, with plenty of free memory and CPU. Each night the scans kick off, we noticed the following start scrolling by in the log files:

vmdnsd t@139675744290560: dirsync failed with 9127.

vmdnsd t@139675744290560: dirsync failed with 9127.

vmdnsd t@139675744290560: dirsync failed with 9127.

vmdnsd t@139675744290560: dirsync failed with 9127.

This happens over and over continuously until the service is restarted. If we try to login to the Web client, we are presented with a web SSO error, and are unable to login. The only solution we've found to this, is SSH into the VCSA and run the following:

service-control --stop vmdnsd

service-control --start vmdnsd

This will quickly stop the "dirsync failed with 9127." from scrolling, and allow logins again. We have opened a ticket with VMware on this issue, but so far they have been unable to help with this, and basically have told us "just stop scanning the appliance". Not really a good fix, as we have basically identified a DOS on the VCSA, and they are unable to fix it.

Just wanted to give everybody else a heads up, in case your security team (like ours) performs regular scans on the network (OpenVAS, Nessus, etc.) For now, just put the VCSA into the exception list :smileyangry:

Tags (1)
5 Replies
RMPeter
Contributor
Contributor

We discovered this too. Was a solution ever found? It seems credential testing crashes the VMware manager. The guests continue to run, but they are not accessible by console anymore.

Reply
0 Kudos
jprovine7
Expert
Expert

I know someone asked this but I will ask again since it was not answered, was this issue ever resolved? Are you now able to run your scans or are they still disabled? Was it an issue with vmware or the scanning software?

Reply
0 Kudos
RMPeter
Contributor
Contributor

I'm the one who asked on Feb 7th, but did not reach a conclusion. It doesn't seem to be terribly widespread. Maybe not enough people are doing internal scans with OpenVAS.

jprovine7
Expert
Expert

It caused us big issues, two of our esxi host disconnected and our root account got locked out. We had to move all the guests off and reboot the esxi host to get things to work again. So I am trying to find out if it is cause by vmware or by the scanning software. I have a ticket open with vmware but they keep saying it was the backup software that wasn't even running at that time.

Reply
0 Kudos
sjesse
Leadership
Leadership

We use nessus which I'm pretty sure openvas is an earlier fork of, and so far I haven't seen any problems with the lastest versions of the appliance.We have only been on 6.5 since September 2018.