- VMware Technology Network
- :
- Cloud & SDDC
- :
- ESXi
- :
- ESXi Discussions
- :
- ESXi 5.5 Datastore Corruption...
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
ESXi 5.5 Datastore Corruption...
Hello,
Looking for some guidance here. It appears my RAID controller experienced a fatal error and reset itself multiple times over the weekend. Today, all the VMs on my ESXi 5.5 host are now inaccessible. The latest log file and the -flat.vmdk files are missing for each VM. I dismounted the datastore and ran a VOMA analysis, which is reporting extensive corruption. I'd attach the file, but it is over 56MB in size and is too large to attach to the post. Below are a few segments of the analysis file showing the corruption. Just wondering if you think there is any point of purchasing and opening a case with VMware Support, or am I beating a dead horse at this point?
Checking if device is actively used by other hosts
Running VMFS Checker version 1.0 in default mode
Initializing LVM metadata, Basic Checks will be done
Phase 1: Checking VMFS header and resource files
Detected VMFS file system (labeled:'datastore1') with UUID:556f14b7-d3bb20db-fb8d-40f2e9af86b8, Version 5:60
Phase 2: Checking VMFS heartbeat region
ON-DISK ERROR: Invalid HB address <18446743491834347520>
Phase 3: Checking all file descriptors.
Found stale lock [type 10c00001 offset 173344768 v 129, hb offset 3231744
gen 43, mode 1, owner 58e8cca8-1a667fdb-38d7-40f2e9af86b8 mtime 134
num 0 gblnum 0 gblgen 0 gblbrk 0]
Found stale lock [type 10c00001 offset 173350912 v 23, hb offset 3231744
gen 43, mode 1, owner 58e8cca8-1a667fdb-38d7-40f2e9af86b8 mtime 190
num 0 gblnum 0 gblgen 0 gblbrk 0]
ON-DISK ERROR: <FD c376 r4> corrupted address <PB c698 r288>
ON-DISK ERROR: <FD c376 r4> : invalid FD type 0x700
Found stale lock [type 10c00001 offset 173365248 v 33, hb offset 3231744
gen 43, mode 1, owner 58e8cca8-1a667fdb-38d7-40f2e9af86b8 mtime 192
num 0 gblnum 0 gblgen 0 gblbrk 0]
Found stale lock [type 10c00001 offset 173369344 v 162, hb offset 3231744
gen 43, mode 1, owner 58e8cca8-1a667fdb-38d7-40f2e9af86b8 mtime 253
num 0 gblnum 0 gblgen 0 gblbrk 0]
Found stale lock [type 10c00001 offset 173375488 v 50, hb offset 3231744
gen 43, mode 1, owner 58e8cca8-1a667fdb-38d7-40f2e9af86b8 mtime 799
num 0 gblnum 0 gblgen 0 gblbrk 0]
ON-DISK ERROR: <FD c376 r16> corrupted address <INVALID c32293 r931>
ON-DISK ERROR: <FD c376 r16> : invalid FD type 0x9066c389
Found stale lock [type 10c00001 offset 173389824 v 57, hb offset 3231744
gen 43, mode 1, owner 58e8cca8-1a667fdb-38d7-40f2e9af86b8 mtime 804
num 0 gblnum 0 gblgen 0 gblbrk 0]
Found stale lock [type 10c00001 offset 173436928 v 82, hb offset 3231744
gen 43, mode 1, owner 58e8cca8-1a667fdb-38d7-40f2e9af86b8 mtime 120
num 0 gblnum 0 gblgen 0 gblbrk 0]
Found stale lock [type 10c00001 offset 173438976 v 83, hb offset 3231744
gen 43, mode 1, owner 58e8cca8-1a667fdb-38d7-40f2e9af86b8 mtime 132
num 0 gblnum 0 gblgen 0 gblbrk 0]
Found stale lock [type 10c00001 offset 173441024 v 84, hb offset 3231744
gen 43, mode 1, owner 58e8cca8-1a667fdb-38d7-40f2e9af86b8 mtime 143
num 0 gblnum 0 gblgen 0 gblbrk 0]
Found stale lock [type 10c00001 offset 173443072 v 85, hb offset 3231744
gen 43, mode 1, owner 58e8cca8-1a667fdb-38d7-40f2e9af86b8 mtime 155
num 0 gblnum 0 gblgen 0 gblbrk 0]
ON-DISK ERROR: <FD c376 r49> corrupted address <INVALID c1027 r0>
ON-DISK ERROR: <FD c376 r49> : invalid FD type 0x85028b00
Found stale lock [type 10c00001 offset 173445120 v 86, hb offset 3231744
gen 43, mode 1, owner 58e8cca8-1a667fdb-38d7-40f2e9af86b8 mtime 167
num 0 gblnum 0 gblgen 0 gblbrk 0]
Found stale lock [type 10c00001 offset 173447168 v 88, hb offset 3231744
gen 43, mode 1, owner 58e8cca8-1a667fdb-38d7-40f2e9af86b8 mtime 236
num 0 gblnum 0 gblgen 0 gblbrk 0]
Found stale lock [type 10c00001 offset 173449216 v 89, hb offset 3231744
gen 43, mode 1, owner 58e8cca8-1a667fdb-38d7-40f2e9af86b8 mtime 251
num 0 gblnum 0 gblgen 0 gblbrk 0]
Found stale lock [type 10c00001 offset 173451264 v 90, hb offset 3231744
gen 43, mode 1, owner 58e8cca8-1a667fdb-38d7-40f2e9af86b8 mtime 262
num 0 gblnum 0 gblgen 0 gblbrk 0]
Found stale lock [type 10c00001 offset 173453312 v 91, hb offset 3231744
gen 43, mode 1, owner 58e8cca8-1a667fdb-38d7-40f2e9af86b8 mtime 274
num 0 gblnum 0 gblgen 0 gblbrk 0]
ON-DISK ERROR: <FD c376 r54> corrupted address <INVALID c0 r0>
ON-DISK ERROR: <FD c376 r54> invalid generation 0
ON-DISK ERROR: <FD c376 r54> : invalid FD type 0x0
Found stale lock [type 10c00001 offset 173455360 v 92, hb offset 3231744
gen 43, mode 1, owner 58e8cca8-1a667fdb-38d7-40f2e9af86b8 mtime 286
num 0 gblnum 0 gblgen 0 gblbrk 0]
Phase 4: Checking pathname and connectivity.
ON-DISK ERROR: Invalid direntry <BBT File Server-flat.vmdk, 376, 16>
ON-DISK ERROR: Invalid direntry <vmware.log, 376, 54>
ON-DISK ERROR: Invalid direntry <vSphere Management Assistant-flat.vmdk, 376, 4>
ON-DISK ERROR: Invalid direntry <vmware.log, 376, 49>
Phase 5: Checking resource reference counts.
ON-DISK ERROR: PB inconsistency found: (608,0) allocated in bitmap, but never used
ON-DISK ERROR: PB inconsistency found: (608,1) allocated in bitmap, but never used
ON-DISK ERROR: PB inconsistency found: (608,2) allocated in bitmap, but never used
ON-DISK ERROR: PB inconsistency found: (608,4) allocated in bitmap, but never used
ON-DISK ERROR: PB inconsistency found: (608,5) allocated in bitmap, but never used
ON-DISK ERROR: PB inconsistency found: (608,6) allocated in bitmap, but never used
ON-DISK ERROR: PB inconsistency found: (608,7) allocated in bitmap, but never used
ON-DISK ERROR: PB inconsistency found: (608,8) allocated in bitmap, but never used
ON-DISK ERROR: PB inconsistency found: (608,9) allocated in bitmap, but never used
ON-DISK ERROR: PB inconsistency found: (608,10) allocated in bitmap, but never used
ON-DISK ERROR: PB inconsistency found: (608,11) allocated in bitmap, but never used
ON-DISK ERROR: PB inconsistency found: (608,12) allocated in bitmap, but never used
ON-DISK ERROR: PB inconsistency found: (608,13) allocated in bitmap, but never used
ON-DISK ERROR: PB inconsistency found: (608,14) allocated in bitmap, but never used
ON-DISK ERROR: PB inconsistency found: (608,15) allocated in bitmap, but never used
ON-DISK ERROR: PB inconsistency found: (609,0) allocated in bitmap, but never used
ON-DISK ERROR: PB inconsistency found: (609,1) allocated in bitmap, but never used
ON-DISK ERROR: PB inconsistency found: (609,2) allocated in bitmap, but never used
....thousands of these errors.
Total Errors Found: 669343
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello
I may be able to help.
Please read my instructions : Create a VMFS-Header-dump using an ESXi-Host in production | VM-Sickbay
Create a dump like I described in that link - upload it somewhere and provide a download link.
Then call me via skype - see my signature.
Do NOT try anything else before I have seen the dump.
Ulli
________________________________________________
Do you need support with a VMFS recovery problem ? - send a message via skype "sanbarrow"
I do not support Workstation 16 at this time ...