vCSA 6.5.0 U2G 13638625 webclient working too slow and giving query timeout dataservice adapters errors

[RaMa2018]

In our environment  we have vCSA 6.5.0 U2G 13638625 with 100 ESXi host and around 600 VMs.  Webclient is running too slow and gives timeout errors (See attached) while trying to pull out any inventory details such as datastores, performance charges, VMs running on the datastores etc.  The refresh sign keep spinning and then it results in query timeout errors.  This issue is recurring after very 2 to 2.5 months.  As a workaround, we planned a reboot every 2 months and it was working as after reboot of appliance, webclient performance seems to be better for few days.  But, this time even after rebooting, it is giving same issue.  The query time out errors are very annoying and end users are unable to work properly due to this. 

Raised a support case with VMware and they tried to increase the memory size of vmware-vpxd-svcs and vsphere-client service to 2048 but it didnt resolve the problem.  Any suggestions on this ?  Has it got something to with this - We still have previous vCSA 6.0 appliance server in powered off mode in this inventory.  (Just considering) or anything else.  We take a daily clone of appliance but we stopped that also in order to check, but it didn't work out. 

[T180985]

Its not a fix but as a workaround you could try extending the timeout period from 120 to 300 seconds, might give VCSA enough time to complete your query?

1. Check the the webclient.properties file

/etc/vmware/vsphere-client/

2. Take a copy of the file by running the command:

cp webclient.properties webclient.properties.bak

3. Edit the webclient.properties file

/etc/vmware/vsphere-client/

4. Change the line:

dataservice.timeoutSeconds = 120 to dataservice.timeoutSeconds = 300

5. Restart the vCenter server for the change to take effect.

[Alex_Romeo]

HI,

just for a test, try working with the HTML5 client for 6.5 (even if it's not complete ... and just a test)

let's see if the same operations (those allowed) generate the same error.

ARomeo

[RaMa2018]

Would that also resolve the login time ? Because when a user logs in with AD credentials, it is taking long time to login. 

[RaMa2018]

Over HTML client, we aren't seeing errors but slowness is there.  I am just not getting this as slowness was getting resolved after appliance reboot. But this time, after reboot issue seems to be persisting.

[Alex_Romeo]

If the ticket is still open with support, try to see with them if a possible VCSA update could solve the slowness problem.

ARomeo

[VivekThacker]

You can try the below things.

Check the dataservice.log if you are still seeing the "the query execution time out" issue. Paste the logs over here for further review.

Also, are there any third party plugins registered to the vCenter? If yes, then disable them and check if the issue persists.

[scott28tt]

Might be an idea to stay engaged with VMware Support, given that you may end up applying a "fix" from the advice you're given here which could be unsupported and therefore gets undone by a future patch or update.

[Ajay1988]

Hello ,

That's a generic error which means a request for a property was sent and vCenter did not get any response in 120 seconds . Increasing the timeout will just delay the failures and not resolve it .

Do we have a VMware SR number ?

Also you have 100 Hosts . Is it exactly hundred or above 100 ? If above 100 ESXi ; then you should be a Medium Sized VCSA ( 8 vCPU's and 24GB RAM) .

Can you share the /etc/vmware/service-layout.mfx and also the vsphere_client_virgo.log from /var/log/vmware/vsphere-client/logs right after the issue is observed and mention the time .

Regards,

AJ

[g0rbi]

I've got the same issue here.