- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
VSAN problem after crash one cache drive
Hi.
I have 4 node ( ftt=1 ) cluster. After damage vsan doesn't rebuild object. I had one of object ABSENT. After delete inaccessible object I can't power on VM. There is problem with vmdk. How to recover it? On power there is file not found
/localhost/dc1/computers/cls01/resourcePool/vms> vsan.vm_object_info Vmxxxxx
2023-10-29 23:59:52 +0000: Fetching vSAN disk info from 192.168.2.152 (may take a moment) ...
2023-10-29 23:59:52 +0000: Fetching vSAN disk info from 192.168.2.153 (may take a moment) ...
2023-10-29 23:59:52 +0000: Fetching vSAN disk info from 192.168.2.151 (may take a moment) ...
2023-10-29 23:59:52 +0000: Fetching vSAN disk info from 192.168.2.176 (may take a moment) ...
2023-10-29 23:59:52 +0000: Fetching vSAN disk info from 192.168.2.160 (may take a moment) ...
2023-10-29 23:59:53 +0000: Done fetching vSAN disk infos
VM VMxxxx:
VM has a non-vSAN datastore
Disk backing:
[cls01-vsan01] 23381365-0851-f719-0c77-e43d1a2ad266/VMxxxx-000001.vmdk
[cls01-vsan01] 23381365-0851-f719-0c77-e43d1a2ad266/VMxxxx.vmdk
[cls01-vsan01] 23381365-0851-f719-0c77-e43d1a2ad266/VMxxxx.vmx
DOM Object: 23381365-0851-f719-0c77-e43d1a2ad266 (v11, owner: 192.168.2.160, proxy owner: None, policy: stripeWidth = 1, cacheReservation = 0, proportionalCapacity = [0, 100], hostFailuresToTolerate = 1, forceProvisioning = 0, spbmProfileId = aa6d5a82-1c88-45da-85d3-3d74b91a5bad, spbmProfileGenerationNumber = 0, CSN = 269, SCSN = 263, spbmProfileName = vSAN Default Storage Policy)
RAID_1
Component: c6673e65-9ab4-a26c-bc5f-b8ca3a694c4c (state: ACTIVE (5), host: 192.168.2.151, capacity: naa.5002538f41a5b9f5, cache: t10.NVMe____Samsung_SSD_980_1TB_____________________2446B111D5382500,
votes: 1, usage: 0.6 GB, proxy component: false)
Component: a6793e65-7c2c-bca7-25aa-e43d1a2ad266 (state: ACTIVE (5), host: 192.168.2.153, capacity: naa.600508b1001cc6658131bdaa6b99e85f, cache: naa.600508b1001cb55d75f95b40596da2ec,
votes: 1, usage: 0.6 GB, proxy component: false)
Witness: e7e03e65-c625-919d-ec18-e43d1a2ad266 (state: ACTIVE (5), host: 192.168.2.160, capacity: naa.6d4ae520a53f52002b76d7671f806885, cache: t10.NVMe____Samsung_SSD_960_EVO_250GB_______________C80EB38153382500,
votes: 1, usage: 0.0 GB, proxy component: false)
[cls01-vsan01] 23381365-0851-f719-0c77-e43d1a2ad266/VMxxxx-000001.vmdk
DOM Object: 2b321b65-f40d-0c96-5313-e43d1a2ad266 (v11, owner: 192.168.2.153, proxy owner: None, policy: stripeWidth = 1, cacheReservation = 0, proportionalCapacity = [0, 100], hostFailuresToTolerate = 1, forceProvisioning = 0, spbmProfileId = aa6d5a82-1c88-45da-85d3-3d74b91a5bad, spbmProfileGenerationNumber = 0, CSN = 243, SCSN = 232, spbmProfileName = vSAN Default Storage Policy)
RAID_1
Component: 7ded3d65-febc-1ad0-b794-b8ca3a694c4c (state: ACTIVE (5), host: 192.168.2.153, capacity: naa.5002538a06cf4d60, cache: naa.600508b1001cb55d75f95b40596da2ec,
votes: 1, usage: 64.8 GB, proxy component: false)
Component: b8793e65-d23c-01bb-4b87-90e2ba16a33c (state: ACTIVE (5), host: 192.168.2.151, capacity: naa.5002538f3364b839, cache: naa.5002538f41a5ba07,
votes: 1, usage: 64.8 GB, proxy component: false)
Witness: fc983d65-f7b6-82b2-41a6-b8ca3a694c4c (state: ACTIVE (5), host: 192.168.2.152, capacity: naa.5002538e90b64be7, cache: t10.NVMe____Corsair_MP400___________________________E61B20C04BA77964,
votes: 1, usage: 0.0 GB, proxy component: false)
[cls01-vsan01] 23381365-0851-f719-0c77-e43d1a2ad266/VMxxxx.vmdk
Couldn't find info about DOM object '24381365-729a-538f-f3cf-e43d1a2ad266'
--> this is doesn't exists: '24381365-729a-538f-f3cf-e43d1a2ad266'
files in folder of vm:
-rw------- 1 root root 0 Oct 29 03:18 .2b321b65-f40d-0c96-5313-e43d1a2ad266.lck
drwxr-xr-x 1 root root 560 Oct 29 23:57 .dvsData
-r-------- 1 root root 1441792 Sep 26 19:59 .fbb.sf
-r-------- 1 root root 267026432 Sep 26 19:59 .fdc.sf
-rw------- 1 root root 0 Oct 2 21:12 .old.2b321b65-f40d-0c96-5313-e43d1a2ad266.lck.old
-r-------- 1 root root 1179648 Sep 26 19:59 .pb2.sf
-r-------- 1 root root 268435456 Sep 26 19:59 .pbc.sf
-r-------- 1 root root 262733824 Sep 26 19:59 .sbc.sf
drwx------ 1 root root 280 Sep 26 19:59 .sdd.sf
-r-------- 1 root root 4194304 Sep 26 19:59 .vh.sf
-rw------- 1 root root 325 Oct 20 11:58 VMxxxxx-000001.vmdk
-rw-r--r-- 1 root root 1625 Oct 29 23:23 VMxxxxx-062a49d7.hlog
-rw------- 1 root root 295336 Oct 2 21:12 VMxxxxx-Snapshot1.vmsn
-rw------- 1 root root 270840 Oct 23 08:05 VMxxxxx.nvram
-rw------- 1 root root 555 Sep 26 20:21 VMxxxxx.vmdk
-rw-r--r-- 1 root root 485 Oct 2 21:12 VMxxxxx.vmsd
-rwxr-xr-x 1 root root 3808 Oct 29 23:57 VMxxxxx.vmx
-rw------- 1 root root 3932 Sep 27 21:27 VMxxxxx.vmxf
-rw------- 1 root root 555 Oct 30 00:03 old.vmdk
-rw-r--r-- 1 root root 37261 Oct 19 08:11 vmware-58.log
-rw-r--r-- 1 root root 37262 Oct 19 08:14 vmware-59.log
-rw-r--r-- 1 root root 37262 Oct 19 08:17 vmware-60.log
-rw-r--r-- 1 root root 37349 Oct 19 08:20 vmware-61.log
-rw-r--r-- 1 root root 37262 Oct 19 08:23 vmware-62.log
-rw-r--r-- 1 root root 328691 Oct 20 11:58 vmware-63.log
-rw-r--r-- 1 root root 242190 Oct 23 08:05 vmware.log
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I would recommend not touching the the environment and contacting support ASAP. Either that, or restore from backup.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@jynx_km, Is this a homelab? If not then why are you using unsupported consumer-grade disks?
Can you cat the file VMxxxxx.vmdk to indicate what object is backing the base-disk?
What was the UUID of the object you deleted and why did you delete that?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
# Disk DescriptorFile
version=4
encoding="UTF-8"
CID=8714a425
parentCID=ffffffff
createType="vmfs"
# Extent description
RW 503316480 VMFS "vsan://24381365-729a-538f-f3cf-e43d1a2ad266"
# The Disk Data Base
#DDB
ddb.adapterType = "lsilogic"
ddb.geometry.cylinders = "31330"
ddb.geometry.heads = "255"
ddb.geometry.sectors = "63"
ddb.longContentID = "c7e852a3a673d0f1b3d6d1838714a425"
ddb.thinProvisioned = "1"
ddb.toolsInstallType = "1"
ddb.toolsVersion = "11333"
ddb.uuid = "60 00 C2 94 7e 0c bf ea-63 dc 16 9d 10 f1 42 60"
ddb.virtualHWVersion = "14"
i delete:
This disk was absent ( damaged )
Component State: ABSENT, Address Space(B): 29870784512 (27.82GB), Disk UUID: 52732347-62a1-4604-b2c9-e1d13d7258c2
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@jynx_km What you referenced there is a component, not an object - did you delete object with UUID 24381365-729a-538f-f3cf-e43d1a2ad266?
If yes, then this gone and you need to restore from backups.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Ok. I understand - but this object was inaccessible. I couldn't recover it. Is there any way to get file with snapshot? Copy or move to other place?
I have more that objects which are inaccessible, and I can't do anything for them:
Detected 9 objects to be inaccessible
Detected ce5ee760-8414-7819-071f-e43d1a2ad266 on 192.168..x.x to be inaccessible
Detected 0575e760-b0c2-95c1-5a08-e43d1a2b31be on 192.168..x.x to be inaccessible
Detected a9b21c64-02a3-7701-e0b1-e43d1a2b31be on 192.168.x.x to be inaccessible
Detected b0730162-08ff-5f4d-5bce-f4ce46aa0cb4 on 192.168.x.xto be inaccessible
Detected c59c1e65-5423-7653-8a91-90e2ba16a33c on 192.168.x.x to be inaccessible
Detected c6fb9963-baf7-b10d-dade-f4ce46aa0cb4 on 192.168..x.x to be inaccessible
Detected dbfb9963-a87f-501f-acb6-f4ce46aa0cb4 on 192.168.x.x to be inaccessible
Detected fbb24761-a087-51d8-2c5e-f4ce46aa0cb4 on 192.168.x.x to be inaccessible
Detected 75bcb964-3af5-65e7-7f3e-90e2ba16a33c on 192.168.x.x to be inaccessible
