VMware Cloud Community
marnow
Enthusiast
Enthusiast

vSAN - Datastore capacity shows 0 after last patch

Hello,

Recently patched to VMware ESXi, 6.5.0, 7388607, the capacity view no longer works! I have to refresh the Summary View to pull the numbers but after another vCenter refresh info disappear...

Also I'm getting this message: VMware vSAN in cluster XXX in datacenter YYY does not have capacity.

Veeam is unable to backup the VMs.

Reply
0 Kudos
15 Replies
GreatWhiteTec
VMware Employee
VMware Employee

Hi Marnow,

What is the version of your vCenter? Did you upgrade/patch vCenter to a higher/same level build as your hosts prior to patching the hosts?

Your vCenter should be at build number 7312210

Reply
0 Kudos
marnow
Enthusiast
Enthusiast

Yes, I patched the vCenter as well. I'm running 6.5.0.13000 Build Number 7312210

Reply
0 Kudos
GreatWhiteTec
VMware Employee
VMware Employee

I've seen this is the past, and usually the resolution is a vCenter restart.

If you use RVC, do you see the correct info?

localhost/<datacenter>/datastores> show <vsanDatastore>

pastedImage_1.png

Reply
0 Kudos
TheBobkin
Champion
Champion

Hello marnow​,

Welcome to vSAN Community.

Determine where is seeing 0B datastore - whether it is an issue with the web-client, the vCenter or the vSAN cluster side:

- Check via CLI on a host with #df -h and that the hosts are clustered #esxcli vsan cluster get

- Check from rvc like GreatWhiteTec said.

- Check with UI client instead of Flash client (https://FQDN-or-IP-Address-of-VC/UI).

- Check which necessary vCenter services are running or not running (service-control --status --all in vCSA CLI)

Is this currently a production cluster and/or have any dependencies on vCenter? If not and the hosts see normal datastore size then try restarting the vCenter services or rebooting the vC.

VMware Knowledge Base

Bob

Reply
0 Kudos
marnow
Enthusiast
Enthusiast

Nope, I see 0 GB

/localhost/OU-UTS-VSAN/datastores> show 1

path: /localhost/OU-UTS-VSAN/datastores/vsanDatastore

type: vsan

url: ds:///vmfs/volumes/vsan:5208d5c6f81944d9-8520459c09710020/

multipleHostAccess: true

capacity: 0.00GB

free space: 0.00GB

/localhost/OU-UTS-VSAN/datastores>

Reply
0 Kudos
GreatWhiteTec
VMware Employee
VMware Employee

From vCenter UI. Click vCenter server (top) > Configure > Storage Provider. You should have 1 provider per vSAN host, what is the status for each of them?

Reply
0 Kudos
marnow
Enthusiast
Enthusiast

All Storage Providers are Online.

I see the issue in both versions of vCenter (Flash and HTML5), when I execute df-h on hosts, the numbers are correct:

vsan       203.8T  22.8T    181.0T  11% /vmfs/volumes/vsanDatastore

All services are up and running:

root@vc6 [ ~ ]# service-control --status --all

Running:

applmgmt lwsmd pschealth vmafdd vmcad vmdird vmdnsd vmonapi vmware-cis-license vmware-cm vmware-content-library vmware-eam vmware-perfcharts vmware-psc-client vmware-rhttpproxy vmware-sca vmware-sps vmware-statsmonitor vmware-sts-idmd vmware-stsd vmware-updatemgr vmware-vapi-endpoint vmware-vmon vmware-vpostgres vmware-vpxd vmware-vpxd-svcs vmware-vsan-health vmware-vsm vsphere-client vsphere-ui

Stopped:

vmcam vmware-imagebuilder vmware-mbcs vmware-netdumper vmware-rbd-watchdog vmware-vcha

I've attempted to reboot the vCenter, it failed with storage related issues, called support, we run

fsck -fy /dev/mapper/log_vg-log

VMware Knowledge Base

....some errors have been fixed and started normally.

Reply
0 Kudos
TheBobkin
Champion
Champion

Hello marnow​,

Possibly a long shot but check timezones and licensing as others have seen these resolve what appears to be similar issues:

https://communities.vmware.com/thread/552827

Check NTP sync between hosts and vCenter as this can cause too many issues to count.

Are other vSAN elements displaying properly such as Disk Management and vSAN Health?

Bob

Reply
0 Kudos
marnow
Enthusiast
Enthusiast

I think, it has something to do with one of the host being partitioned from the vSAN datastore...I rebooted the questionable host and the values were back while the host was offline...

When I execute df -h....i get

vsan         0.0B   0.0B      0.0B   0% /vmfs/volumes/vsanDatastore

There is no network issue, I verified that....should I leave and join the host to the vSAN cluster to fix this?

Reply
0 Kudos
TheBobkin
Champion
Champion

Hello marnow​,

Where are you checking df -h from when this shows as 0B?

A partitioned host will show 0B with df -h, this is expected.

Check that that host is participating in the cluster:

# esxcli vsan cluster get

If it is showing as cluster membership of 1 then it is partitioned from the rest of the cluster.

The above also tells if it is Unicast Mode enabled or not.

Check vSAN networking is configured correctly:

# esxcli vsan network list

Check ping between the vSAN configured vmk on one host to the others e.g.:

# vmkping -I vmkX 192.168.X.X

(get the IP and vmk# of the vSAN interface from GUI or #esxcfg-vmknic -l)

Check that hosts Unicast lists are fully populated with all other nodes in the clusters info:

# esxcli vsan unicastagent list

Make sure the host is not in Maintenance Mode or vSAN Decom state:

# esxcli vsan maintenancemode cancel

If all of the above is fine then try leave and rejoin cluster:

# esxcli vsan cluster leave

# esxcli vsan cluster join -u <Sub-cluster UUID from another cluster member>

What build version did you upgrade the cluster from, was it using Unicast and are all the disks upgraded to on-disk format v5?

Health check GUI will help a lot with determining many issues including much of the above (Cluster > Monitor > vSAN > Health).

Bob

Reply
0 Kudos
marnow
Enthusiast
Enthusiast

Currently working with support, we've removed the host from the cluster and the error disappeared....they are also investigating the "double unicast entries for witness" issue on each host in the cluster...

NodeUuid                              IsWitness  Supports Unicast  IP Address       Port  Iface Name

------------------------------------  ---------  ----------------  --------------  -----  ----------

5968d583-11ca-ff62-db19-246e963c74a8          0              true  192.168.250.10  12321

5968db7c-1580-f42a-1ae3-246e963e7fd8          0              true  192.168.250.12  12321

5968dec6-6963-8be6-36ca-246e963e7450          0              true  192.168.250.13  12321

5968f36f-ff76-8748-4eb3-246e963e71f0          0              true  192.168.250.23  12321

5968f251-5ed4-942b-04f2-246e963e1a08          0              true  192.168.250.21  12321

5968f315-bddc-30d6-2e03-246e963e1948          0              true  192.168.250.22  12321

5968f057-fd42-e63e-14b9-246e963e19d8          0              true  192.168.250.20  12321

5922e807-d875-58b0-064f-f4e9d494b820          1              true  10.8.120.21     12321

00000000-0000-0000-0000-000000000000          1              true  10.8.120.21     12321

They said it is not normal, vSAN Heath Check is reporting "vSAN cluster configuration consistency" error....

pastedImage_1.png

Reply
0 Kudos
GreatWhiteTec
VMware Employee
VMware Employee

Is this a Stretched Cluster? (doesn't look like)

Were any of those hosts used for a 2-node of SC configuration in the past?

Reply
0 Kudos
marnow
Enthusiast
Enthusiast

Yes, it is a stretched cluster 4+4+1 (witness). All fresh install, it stared after we removed host from the cluster.

Reply
0 Kudos
marnow
Enthusiast
Enthusiast

We had been able successfully join the host to the cluster, network sync error disappeared.

Reply
0 Kudos
TimondAU
Contributor
Contributor

The only thing that resolved this issue for me was restarting vCenter. (6.7u3)

The issue originally occurred after adding two additional hosts/disk groups to the cluster.

Everything showed healthy in skyline health check. RVC showed everything healthy. I could find no errors anywhere with the configuration or networking.

Simply rebooted the vCenter appliance and the vSAN datastore now shows the correct size.

Reply
0 Kudos