Many of this error per second on ESXi hosts (<11>ERROR vsanperfsvc)

[ChrisJ615]

Hey Gang,

My ESXi 6u2 logs are filling up with this message.

<14>INFO vsanperfsvc[MainThr MYSERVER01.DOMAIN.COM VSANMGMTSVC: ad] [statsdaemon::start] Now force cleanup pyCMMDS

<11>esx/release/vmvisor/sys- MYSERVER01.DOMAIN.COM VSANMGMTSVC: oot/lib/python2.7/SocketServer.py", line 434, in server_bind

  File "/build/mts/release/bora-3620759/bora/build/esx/release/vmvisor/sys-boot/lib/python2.7/socket.py", line 224, in meth

error: [Errno 98] Address already in use

<11>ERROR vsanperfsvc[MainTh MYSERVER01.DOMAIN.COM VSANMGMTSVC: ead] [statsdaemon::start] Failed to run() daemon: [Errno 98] Address already in use

Traceback (most recent call last):

  File "/build/mts/release/bora-3544323/bora/build/esx/release/vsanhealth/usr/lib/vmware/vsan/perfsvc/statsdaemon.py", line 456, in start

  File "/build/mts/release/bora-3544323/bora/build/esx/release/vsanhealth/usr/lib/vmware/vsan/perfsvc/statsdaemon.py", line 524, in run

  File "/build/mts/release/bora-3544323/bora/build/esx/release/vsanhealth/usr/lib/vmware/vsan/perfsvc/statsdaemon.py", line 230, in InitVmomiServer

  File "/build/mts/release/bora-3620759/bora/build/esx/release/vmvisor/sys-boot/lib/python2.7/SocketServer.py", line 420, in __init__

  File "/build/mts/release/bora-3620759/bora/build/esx/release/vmvisor/sys-boot/lib/python2.7/BaseHTTPServer.py", line 108, in server_bind

  File "/build/mts/release/bora-3620759/bora/build/

  <14>INFO vsanperfsvc[MainThr MYSERVER01.DOMAIN.COM VSANMGMTSVC: ad] [statsdaemon::InitVmomiServer] Listening on ('localhost', 9095)

I have found these on every node, all 15 of them, in one of my vSAN clusters.  I don't seem to have this problem everywhere and it appears to be isolated. 

As a side note I do not have the performance service running.  As a trouble shooting step I have enabled and disabled the service, but I'm not sure if the scope of the issue is with the performance service even though it appears to be that way. Any help would be great.

[ChrisJ615]

And just to be clear about the frequency of this message.  I have received it 165,657 times across 15 nodes over the last 24 hours.

[ChrisJ615]

For everyone that is following this thread I have opened a SR: 16232466409

[zdickinson]

Good morning, perhaps I missed it, are you running vSAN?  Thank you, Zach.

[hennish]

Hi. Did you ever get this resolved? I'm seeing the same error on one host in my customer's VSAN cluster.

[zdickinson]

Good morning, I'm having trouble finding the article I'm thinking of... but I think this might be a bug the shows up when you have other traditional datastores attached to host that are also in a vSAN cluster.  You can disable the health and performance alerts.  If you try and disable it, without success it might be due to the stats master not being found.  If that's the case...  let us know and we'll go from there   Thank you, Zach.