Hi!
I have a vSAN 6.6 cluster on 4-esxi hosts (All flash)
3 Hosts has 2 disk group (2 ssd 1 - cache, 1 capacity)
I want to increase up vsan space by adding disks to 4 host, disk group successfully creating, but space not increasing up.
vSAN health is ok.
vmkping is ok
I try to reinstall ESXI and restore configuration from backup, but no result
Also when i try to set 4 host to maintenace mode i have alert:
Disk group not used:
Cluster information from problem host:
Hello Taras,
I was going by the esxcli vsan cluster get output in the screenshot you posted:
"Maintenance Mode State: On"
If it still says this then check the vSAN decom state entry for that host (use the Local Node UUID to identify):
# cmmds-tool find -t NODE_DECOM_STATE -f json
0=not in vSAN MM, 6=in vSAN MM.
If it is in vSAN MM but not ESXi MM then put it in MM using:
#esxcli system maintenanceMode set -e true -m ensureObjectAccessibility
Then take it out of MM and see if the vSAN decom state changes to Off/0 .
Bob
Hello Taras,
That host is in Maintenance Mode (MM) at the vSAN-level already (this can vary from vCenter MM state after some actions).
Try taking it out using:
esxcli system maintenanceMode set -e false
Can you show us the full disk management page under cluster > configure > settings ?
Also, if you can connect via RVC and show us the output from :
vsan.disks_stats <path_to_cluster>
Another possibility is on-disk versions of existing disk-groups vs new one is incompatible, if this is the case then recreate disks with legacy format:
How to format vSAN Disk Groups with a legacy format version (2146221) | VMware KB
If all disk-groups are the same/compatible then provide the output noted above and we will look at other potential causes
Bob
Hi TheBobkin!
On all host disk version - 5
Host is not in maintenance mode
I try to recreate disk group with new disks (500GB SSD as cache and 2 TB SSD as capacity)
Result of vsan.disks_stats:
(Problem host - ts1-srv-esxi22)
/localhost/ts1/computers> vsan.disks_stats 0
2017-07-13 11:21:28 +0000: Fetching vSAN disk info from ts1-srv-esxi21.tscrm.com (may take a moment) ...
2017-07-13 11:21:28 +0000: Fetching vSAN disk info from ts1-srv-esxi20.tscrm.com (may take a moment) ...
2017-07-13 11:21:28 +0000: Fetching vSAN disk info from ts1-srv-esxi19.tscrm.com (may take a moment) ...
2017-07-13 11:21:28 +0000: Fetching vSAN disk info from ts1-srv-esxi22.tscrm.com (may take a moment) ...
2017-07-13 11:21:30 +0000: Done fetching vSAN disk infos
+----------------------+--------------------------+-------+------+------------+- --------+----------+------------+----------+----------+-------------+---------+- ---------+---------+
| | | | Num | Capacity | | | Physical | Physical | Physical | Logical | Logical | Logical | Status |
| DisplayName | Host | isSSD | Comp | Total | Used | Reserved | Capacity | Used | Reserved | Capacity | Used | Reserved | Health |
+----------------------+--------------------------+-------+------+------------+- --------+----------+------------+----------+----------+-------------+---------+- ---------+---------+
| naa.5002538c4035807b | ts1-srv-esxi19.tscrm.com | SSD | 0 | 894.25 GB | 0.00 % | 0.00 % | N/A | N/A | N/A | N/A | N/A | N/A | OK (v5) |
| naa.5002538c40615751 | ts1-srv-esxi19.tscrm.com | MD | 0 | 1693.87 GB | 4.07 % | 1.19 % | 1693.86 GB | 4.07 % | 1.19 % | 16383.99 GB | 1.32 % | 0.12 % | OK (v5) |
+----------------------+--------------------------+-------+------+------------+- --------+----------+------------+----------+----------+-------------+---------+- ---------+---------+
| naa.5002538c4035807f | ts1-srv-esxi19.tscrm.com | SSD | 0 | 894.25 GB | 0.00 % | 0.00 % | N/A | N/A | N/A | N/A | N/A | N/A | OK (v5) |
| naa.5002538c4047dc8e | ts1-srv-esxi19.tscrm.com | MD | 46 | 1693.87 GB | 62.09 % | 5.98 % | 1693.86 GB | 62.09 % | 5.98 % | 16383.99 GB | 12.95 % | 0.62 % | OK (v5) |
+----------------------+--------------------------+-------+------+------------+- --------+----------+------------+----------+----------+-------------+---------+- ---------+---------+
| naa.5002538c4032fe4a | ts1-srv-esxi19.tscrm.com | SSD | 0 | 894.25 GB | 0.00 % | 0.00 % | N/A | N/A | N/A | N/A | N/A | N/A | OK (v5) |
| naa.5002538c70015581 | ts1-srv-esxi19.tscrm.com | MD | 51 | 1693.87 GB | 57.18 % | 9.15 % | 1693.86 GB | 57.18 % | 9.15 % | 16383.99 GB | 12.62 % | 0.95 % | OK (v5) |
+----------------------+--------------------------+-------+------+------------+- --------+----------+------------+----------+----------+-------------+---------+- ---------+---------+
| naa.5002538c4032fe3a | ts1-srv-esxi20.tscrm.com | SSD | 0 | 894.25 GB | 0.00 % | 0.00 % | N/A | N/A | N/A | N/A | N/A | N/A | OK (v5) |
| naa.5002538c4061c2c0 | ts1-srv-esxi20.tscrm.com | MD | 12 | 1693.87 GB | 28.66 % | 3.94 % | 1693.86 GB | 28.66 % | 3.94 % | 16383.99 GB | 4.53 % | 0.41 % | OK (v5) |
+----------------------+--------------------------+-------+------+------------+- --------+----------+------------+----------+----------+-------------+---------+- ---------+---------+
| naa.5002538c40358083 | ts1-srv-esxi20.tscrm.com | SSD | 0 | 894.25 GB | 0.00 % | 0.00 % | N/A | N/A | N/A | N/A | N/A | N/A | OK (v5) |
| naa.5002538c4047dc78 | ts1-srv-esxi20.tscrm.com | MD | 81 | 1693.87 GB | 44.17 % | 4.63 % | 1693.86 GB | 44.17 % | 4.63 % | 16383.99 GB | 10.36 % | 0.48 % | OK (v5) |
+----------------------+--------------------------+-------+------+------------+- --------+----------+------------+----------+----------+-------------+---------+- ---------+---------+
| naa.5002538c4035807e | ts1-srv-esxi20.tscrm.com | SSD | 0 | 894.25 GB | 0.00 % | 0.00 % | N/A | N/A | N/A | N/A | N/A | N/A | OK (v5) |
| naa.5002538c4061c2c2 | ts1-srv-esxi20.tscrm.com | MD | 0 | 1693.87 GB | 4.07 % | 1.19 % | 1693.86 GB | 4.07 % | 1.19 % | 16383.99 GB | 1.32 % | 0.12 % | OK (v5) |
+----------------------+--------------------------+-------+------+------------+- --------+----------+------------+----------+----------+-------------+---------+- ---------+---------+
| naa.5002538c4032fde5 | ts1-srv-esxi21.tscrm.com | SSD | 0 | 894.25 GB | 0.00 % | 0.00 % | N/A | N/A | N/A | N/A | N/A | N/A | OK (v5) |
| naa.5002538c4047dc92 | ts1-srv-esxi21.tscrm.com | MD | 34 | 1693.87 GB | 26.50 % | 6.34 % | 1693.86 GB | 26.50 % | 6.34 % | 16383.99 GB | 5.63 % | 0.66 % | OK (v5) |
+----------------------+--------------------------+-------+------+------------+- --------+----------+------------+----------+----------+-------------+---------+- ---------+---------+
| naa.55cd2e404c138ea1 | ts1-srv-esxi21.tscrm.com | SSD | 0 | 447.13 GB | 0.00 % | 0.00 % | N/A | N/A | N/A | N/A | N/A | N/A | OK (v5) |
| naa.5002538c4047dc91 | ts1-srv-esxi21.tscrm.com | MD | 60 | 1693.87 GB | 64.37 % | 4.56 % | 1693.86 GB | 64.37 % | 4.56 % | 16383.99 GB | 12.10 % | 0.47 % | OK (v5) |
+----------------------+--------------------------+-------+------+------------+- --------+----------+------------+----------+----------+-------------+---------+- ---------+---------+
| naa.55cd2e404c158153 | ts1-srv-esxi22.tscrm.com | SSD | 0 | 447.13 GB | 0.00 % | 0.00 % | N/A | N/A | N/A | N/A | N/A | N/A | OK (v5) |
| naa.5002538c4061c2c1 | ts1-srv-esxi22.tscrm.com | MD | 0 | 1693.87 GB | 4.07 % | 1.19 % | 1693.86 GB | 4.07 % | 1.19 % | 16383.99 GB | 1.32 % | 0.12 % | OK (v5) |
+----------------------+--------------------------+-------+------+------------+- --------+----------+------------+----------+----------+-------------+---------+- ---------+---------+
/localhost/ts1/computers> vsan.disks_stats 02017-07-13 11:21:28 +0000: Fetching vSAN disk info from ts1-srv-esxi21.tscrm.com (may take a moment) ...2017-07-13 11:21:28 +0000: Fetching vSAN disk info from ts1-srv-esxi20.tscrm.com (may take a moment) ...2017-07-13 11:21:28 +0000: Fetching vSAN disk info from ts1-srv-esxi19.tscrm.com (may take a moment) ...2017-07-13 11:21:28 +0000: Fetching vSAN disk info from ts1-srv-esxi22.tscrm.com (may take a moment) ...2017-07-13 11:21:30 +0000: Done fetching vSAN disk infos+----------------------+--------------------------+-------+------+------------+- --------+----------+------------+----------+----------+-------------+---------+- ---------+---------+| | | | Num | Capacity | | | Physical | Physical | Physical | Logical | Logical | Logical | Status || DisplayName | Host | isSSD | Comp | Total | Used | Reserved | Capacity | Used | Reserved | Capacity | Used | Reserved | Health |+----------------------+--------------------------+-------+------+------------+- --------+----------+------------+----------+----------+-------------+---------+- ---------+---------+| naa.5002538c4035807b | ts1-srv-esxi19.tscrm.com | SSD | 0 | 894.25 GB | 0.00 % | 0.00 % | N/A | N/A | N/A | N/A | N/A | N/A | OK (v5) || naa.5002538c40615751 | ts1-srv-esxi19.tscrm.com | MD | 0 | 1693.87 GB | 4.07 % | 1.19 % | 1693.86 GB | 4.07 % | 1.19 % | 16383.99 GB | 1.32 % | 0.12 % | OK (v5) |+----------------------+--------------------------+-------+------+------------+- --------+----------+------------+----------+----------+-------------+---------+- ---------+---------+| naa.5002538c4035807f | ts1-srv-esxi19.tscrm.com | SSD | 0 | 894.25 GB | 0.00 % | 0.00 % | N/A | N/A | N/A | N/A | N/A | N/A | OK (v5) || naa.5002538c4047dc8e | ts1-srv-esxi19.tscrm.com | MD | 46 | 1693.87 GB | 62.09 % | 5.98 % | 1693.86 GB | 62.09 % | 5.98 % | 16383.99 GB | 12.95 % | 0.62 % | OK (v5) |+----------------------+--------------------------+-------+------+------------+- --------+----------+------------+----------+----------+-------------+---------+- ---------+---------+| naa.5002538c4032fe4a | ts1-srv-esxi19.tscrm.com | SSD | 0 | 894.25 GB | 0.00 % | 0.00 % | N/A | N/A | N/A | N/A | N/A | N/A | OK (v5) || naa.5002538c70015581 | ts1-srv-esxi19.tscrm.com | MD | 51 | 1693.87 GB | 57.18 % | 9.15 % | 1693.86 GB | 57.18 % | 9.15 % | 16383.99 GB | 12.62 % | 0.95 % | OK (v5) |+----------------------+--------------------------+-------+------+------------+- --------+----------+------------+----------+----------+-------------+---------+- ---------+---------+| naa.5002538c4032fe3a | ts1-srv-esxi20.tscrm.com | SSD | 0 | 894.25 GB | 0.00 % | 0.00 % | N/A | N/A | N/A | N/A | N/A | N/A | OK (v5) || naa.5002538c4061c2c0 | ts1-srv-esxi20.tscrm.com | MD | 12 | 1693.87 GB | 28.66 % | 3.94 % | 1693.86 GB | 28.66 % | 3.94 % | 16383.99 GB | 4.53 % | 0.41 % | OK (v5) |+----------------------+--------------------------+-------+------+------------+- --------+----------+------------+----------+----------+-------------+---------+- ---------+---------+| naa.5002538c40358083 | ts1-srv-esxi20.tscrm.com | SSD | 0 | 894.25 GB | 0.00 % | 0.00 % | N/A | N/A | N/A | N/A | N/A | N/A | OK (v5) || naa.5002538c4047dc78 | ts1-srv-esxi20.tscrm.com | MD | 81 | 1693.87 GB | 44.17 % | 4.63 % | 1693.86 GB | 44.17 % | 4.63 % | 16383.99 GB | 10.36 % | 0.48 % | OK (v5) |+----------------------+--------------------------+-------+------+------------+- --------+----------+------------+----------+----------+-------------+---------+- ---------+---------+| naa.5002538c4035807e | ts1-srv-esxi20.tscrm.com | SSD | 0 | 894.25 GB | 0.00 % | 0.00 % | N/A | N/A | N/A | N/A | N/A | N/A | OK (v5) || naa.5002538c4061c2c2 | ts1-srv-esxi20.tscrm.com | MD | 0 | 1693.87 GB | 4.07 % | 1.19 % | 1693.86 GB | 4.07 % | 1.19 % | 16383.99 GB | 1.32 % | 0.12 % | OK (v5) |+----------------------+--------------------------+-------+------+------------+- --------+----------+------------+----------+----------+-------------+---------+- ---------+---------+| naa.5002538c4032fde5 | ts1-srv-esxi21.tscrm.com | SSD | 0 | 894.25 GB | 0.00 % | 0.00 % | N/A | N/A | N/A | N/A | N/A | N/A | OK (v5) || naa.5002538c4047dc92 | ts1-srv-esxi21.tscrm.com | MD | 34 | 1693.87 GB | 26.50 % | 6.34 % | 1693.86 GB | 26.50 % | 6.34 % | 16383.99 GB | 5.63 % | 0.66 % | OK (v5) |+----------------------+--------------------------+-------+------+------------+- --------+----------+------------+----------+----------+-------------+---------+- ---------+---------+| naa.55cd2e404c138ea1 | ts1-srv-esxi21.tscrm.com | SSD | 0 | 447.13 GB | 0.00 % | 0.00 % | N/A | N/A | N/A | N/A | N/A | N/A | OK (v5) || naa.5002538c4047dc91 | ts1-srv-esxi21.tscrm.com | MD | 60 | 1693.87 GB | 64.37 % | 4.56 % | 1693.86 GB | 64.37 % | 4.56 % | 16383.99 GB | 12.10 % | 0.47 % | OK (v5) |+----------------------+--------------------------+-------+------+------------+- --------+----------+------------+----------+----------+-------------+---------+- ---------+---------+| naa.55cd2e404c158153 | ts1-srv-esxi22.tscrm.com | SSD | 0 | 447.13 GB | 0.00 % | 0.00 % | N/A | N/A | N/A | N/A | N/A | N/A | OK (v5) || naa.5002538c4061c2c1 | ts1-srv-esxi22.tscrm.com | MD | 0 | 1693.87 GB | 4.07 % | 1.19 % | 1693.86 GB | 4.07 % | 1.19 % | 16383.99 GB | 1.32 % | 0.12 % | OK (v5) |+----------------------+--------------------------+-------+------+------------+- --------+----------+------------+----------+----------+-------------+---------+- ---------+---------+
Hello Taras,
I was going by the esxcli vsan cluster get output in the screenshot you posted:
"Maintenance Mode State: On"
If it still says this then check the vSAN decom state entry for that host (use the Local Node UUID to identify):
# cmmds-tool find -t NODE_DECOM_STATE -f json
0=not in vSAN MM, 6=in vSAN MM.
If it is in vSAN MM but not ESXi MM then put it in MM using:
#esxcli system maintenanceMode set -e true -m ensureObjectAccessibility
Then take it out of MM and see if the vSAN decom state changes to Off/0 .
Bob
Bob,
Thank you, host really in vSAN maintenance mode but not ESXi MM .
Tomorrow i'll try to set host to maintenance mode.
Bob, thank you!
The problem is resolved.
@Taras_v-
just an opsec comment. You may not want to post host fqdn's.