LukeLegend
Contributor
Contributor

Unable to perform any vCenter action in vSphere 6.5 which necessitate querying a storage cluster. Tasks simply hang for eternity :( with no error

We are experiencing a extremely odd problem with vSphere 6.5 which I'm hoping someone may have encountered before.  We have a support case open for it but we are not getting very far.  Basically any vCenter action which seems to necessitate querying a storage cluster appears to hang for eternity with no error this doesn't matter if its a NFS or iSCSI storage cluster all are impacted.  Tasks that we cant perform are for example are ...

- Adding a disk

- Deploying a clone

- changing disk size

We can however perform storage related tasks direct on the host targeted at single datastores,  additionally within vCenter we can deploy objects to standalone datastores. This is not ideal however when you need to deploy VM templates to storage clusters.  Initially we thought this was a permissions thing but this happens even for the administrator@vsphere.local account.  Has anyone seen this before, or something similar. 

0 Kudos
7 Replies
LukeLegend
Contributor
Contributor

Just an Interesting observation,  I just created a fresh storage cluster and enabled Storage DRS I wasn't able to deploy VM's.  Then I disabled it and suddenly I was unable to deploy VM's again, not sure how relevant/useful this information is?

0 Kudos
aadi369
Enthusiast
Enthusiast

Even I faced the same issue while performing any datastore relared activity/jobs.

Goind through logs, we saw driver aborts on the hosts where the datastore information is not reflecting correctly, we checked your driver versions and discovered that they are old.

Also check the Esxi build. If it is running on GA, kindly upgrade the same. This is know issue and resolved in 6.5 U1

===========================================================================================

If it is useful, plz mark answer as correct or helpful.

===========================================================================================

0 Kudos
LukeLegend
Contributor
Contributor

Not sure I understand bud, do you have a KB to what you are referring to, thanks in advance.

0 Kudos
aadi369
Enthusiast
Enthusiast

Kindly run below command and you will get output for the same. Kindly shre the same here

command

esxcfg-scsidevs -a

outuput example

vmhba64 lpfc link-up fc.20000090fa7b5f15:10000090fa7b5f15    (0000:11:00.1) Emulex Corporation Emulex LPe12000 8Gb PCIe Fibre Channel Adapter
vmhba65 iscsi_vmk        online iqn.1998-01.com.vmware:bhbesxi-r03-689e876biSCSI Software Adapter
[root@BHBESXi-R03:~]

0 Kudos
LukeLegend
Contributor
Contributor

Not sure what can be determined from the below ...

Device UID                            Device Type      Console Device                                            Size      Multipath PluginDisplay Name     

eui.16c750f6b1331bfc6c9ce9004d0199b2  Direct-Access    /vmfs/devices/disks/eui.16c750f6b1331bfc6c9ce9004d0199b2  2097152MB NMP     Nimble iSCSI Disk (eui.16c750f6b1331bfc6c9ce9004d0199b2)

eui.1b52c68fb8022bfc6c9ce9004d0199b2  Direct-Access    /vmfs/devices/disks/eui.1b52c68fb8022bfc6c9ce9004d0199b2  2097152MB NMP     Nimble iSCSI Disk (eui.1b52c68fb8022bfc6c9ce9004d0199b2)

eui.20bca21b928de6326c9ce9004d0199b2  Direct-Access    /vmfs/devices/disks/eui.20bca21b928de6326c9ce9004d0199b2  2097152MB NMP     Nimble iSCSI Disk (eui.20bca21b928de6326c9ce9004d0199b2)

eui.45fdde0bc2562ea56c9ce9004d0199b2  Direct-Access    /vmfs/devices/disks/eui.45fdde0bc2562ea56c9ce9004d0199b2  2097152MB NMP     Nimble iSCSI Disk (eui.45fdde0bc2562ea56c9ce9004d0199b2)

eui.4a6700c30df7018b6c9ce9004d0199b2  Direct-Access    /vmfs/devices/disks/eui.4a6700c30df7018b6c9ce9004d0199b2  2097152MB NMP     Nimble iSCSI Disk (eui.4a6700c30df7018b6c9ce9004d0199b2)

eui.74ce190f214bfd866c9ce9004d0199b2  Direct-Access    /vmfs/devices/disks/eui.74ce190f214bfd866c9ce9004d0199b2  2097152MB NMP     Nimble iSCSI Disk (eui.74ce190f214bfd866c9ce9004d0199b2)

eui.799d8ed81d1a15346c9ce9004d0199b2  Direct-Access    /vmfs/devices/disks/eui.799d8ed81d1a15346c9ce9004d0199b2  2097152MB NMP     Nimble iSCSI Disk (eui.799d8ed81d1a15346c9ce9004d0199b2)

eui.7e8fe7cfa3b52a606c9ce9004d0199b2  Direct-Access    /vmfs/devices/disks/eui.7e8fe7cfa3b52a606c9ce9004d0199b2  2097152MB NMP     Nimble iSCSI Disk (eui.7e8fe7cfa3b52a606c9ce9004d0199b2)

eui.89e01069371748936c9ce9004d0199b2  Direct-Access    /vmfs/devices/disks/eui.89e01069371748936c9ce9004d0199b2  2097152MB NMP     Nimble iSCSI Disk (eui.89e01069371748936c9ce9004d0199b2)

eui.93f7dbf9c3816bbf6c9ce9004d0199b2  Direct-Access    /vmfs/devices/disks/eui.93f7dbf9c3816bbf6c9ce9004d0199b2  2097152MB NMP     Nimble iSCSI Disk (eui.93f7dbf9c3816bbf6c9ce9004d0199b2)

eui.a86ec9db7c0ee4156c9ce9004d0199b2  Direct-Access    /vmfs/devices/disks/eui.a86ec9db7c0ee4156c9ce9004d0199b2  2097152MB NMP     Nimble iSCSI Disk (eui.a86ec9db7c0ee4156c9ce9004d0199b2)

eui.ad1b73b5bdd8335c6c9ce9004d0199b2  Direct-Access    /vmfs/devices/disks/eui.ad1b73b5bdd8335c6c9ce9004d0199b2  2097152MB NMP     Nimble iSCSI Disk (eui.ad1b73b5bdd8335c6c9ce9004d0199b2)

eui.c2b4d902237604a86c9ce9004d0199b2  Direct-Access    /vmfs/devices/disks/eui.c2b4d902237604a86c9ce9004d0199b2  2097152MB NMP     Nimble iSCSI Disk (eui.c2b4d902237604a86c9ce9004d0199b2)

eui.dc96b0e812cd0ceb6c9ce9004d0199b2  Direct-Access    /vmfs/devices/disks/eui.dc96b0e812cd0ceb6c9ce9004d0199b2  204800MB  NMP     Nimble iSCSI Disk (eui.dc96b0e812cd0ceb6c9ce9004d0199b2)

eui.e75b19f3523f4e1a6c9ce9004d0199b2  Direct-Access    /vmfs/devices/disks/eui.e75b19f3523f4e1a6c9ce9004d0199b2  2097152MB NMP     Nimble iSCSI Disk (eui.e75b19f3523f4e1a6c9ce9004d0199b2)

eui.f1506b18339919516c9ce9004d0199b2  Direct-Access    /vmfs/devices/disks/eui.f1506b18339919516c9ce9004d0199b2  2097152MB NMP     Nimble iSCSI Disk (eui.f1506b18339919516c9ce9004d0199b2)

naa.600a09803830304d643f4636386c346a  Direct-Access    /vmfs/devices/disks/naa.600a09803830304d643f4636386c346a  12288MB   NMP     NETAPP iSCSI Disk (naa.600a09803830304d643f4636386c346a)

[root@secretserver:~]

0 Kudos
aadi369
Enthusiast
Enthusiast

Seems no issue/erorr on the driver side....

What is the ESXi and vCetnr verion and build.?????

0 Kudos
LukeLegend
Contributor
Contributor

vCenter Version 6.5.0.13000 Build 7515524

and Host version is ...

VMware ESXi™

Copyright © 1998-2017 VMware, Inc. All rights reserved. This product is protected by U.S. and international copyright and intellectual property laws. VMware products are covered by one or more patents listed at http://www.vmware.com/go/patents.

1.23.0

6506686

6.5.0

7526125

0 Kudos