VMware Cloud Community
nazim109
Contributor
Contributor

vmware esxi virtual machine invalid

Dear ,

Thanks in advance.

I am facing issue in my vmware ESXI 6.5.0.

I have two datastore. 256GB ssd for OS and 1.6TB storage for virtual machines.  in the storage menu showing only 256GB storage.

But 1.6 TB showing at storage>Device also my all virtual machine status is invalid.

Can you please assist me how can i recover it without losing data?

 

name: vmfs/volumes/5aa2a95d-897c4df3-ae49-002590c62e7c

status : Invalid

 

Attached log.

 

Thanks

 

nazim109_0-1705011434982.png

 

nazim109_1-1705011463749.png

nazim109_2-1705011477985.png

 

Labels (1)
0 Kudos
20 Replies
hayoken
Contributor
Contributor

Did you figure out how to do this? This just happened to me to day when I lost power.

0 Kudos
a_p_
Leadership
Leadership

Dos the command esxcli storage vmfs snapshot list show any results?

André

0 Kudos
hayoken
Contributor
Contributor

I found out that it corrupted on the storage. I am just redoing the vm. Thanks!

0 Kudos
nazim109
Contributor
Contributor

André,

[root@localhost:~] esxcli storage vmfs snapshot list
[root@localhost:~]

 

 

No snapshot available.

0 Kudos
hayoken
Contributor
Contributor

The command he asked for was if you take snapshots of your vms. If you did, then you could have restored from one of those.

0 Kudos
nazim109
Contributor
Contributor

No snapshot is available.

Now what can i do now?

 

0 Kudos
a_p_
Leadership
Leadership

Please rescan the storage adapter, and then check whether the host's vmkernel.log contains related entries.

André

0 Kudos
nazim109
Contributor
Contributor

Dear André,

Already tried by rescan but not getting in data store.

Is there any other suggestion?

Thanks

0 Kudos
nazim109
Contributor
Contributor

this is the log of rescan

 

2015-01-19T17:42:37.977Z cpu1:66292)ScsiDeviceIO: 2948: Cmd(0x439509ac7500) 0x1a, CmdSN 0x5084 from world 0 to dev "naa.61866da07306b2001c4bb566bc470bf1" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.
2015-01-19T17:42:39.858Z cpu4:65773)FSS: 5749: No FS driver claimed device 'naa.61866da07306b2001c4bb566bc470bf1:1': No filesystem on the device
2015-01-19T17:42:39.867Z cpu21:67822 opID=98db16a)World: 12230: VC opID 85e2cd14 maps to vmkernel opID 98db16a
2015-01-19T17:42:39.867Z cpu21:67822 opID=98db16a)VC: 4511: Device rescan time 180 msec (total number of devices 6)
2015-01-19T17:42:39.867Z cpu21:67822 opID=98db16a)VC: 4514: Filesystem probe time 18 msec (devices probed 4 of 6)
2015-01-19T17:42:39.867Z cpu21:67822 opID=98db16a)VC: 4516: Refresh open volume time 1 msec

0 Kudos
a_p_
Leadership
Leadership

Does the following command (a single command line) show details about the missing datastore?

offset="128 2048"; for dev in `esxcfg-scsidevs -l | grep "Console Device:" | awk {'print $3'}`; do disk=$dev; echo $disk; partedUtil getptbl $disk; { for i in `echo $offset`; do echo "Checking offset found at $i:"; hexdump -n4 -s $((0x100000+(512*$i))) $disk; hexdump -n4 -s $((0x1300000+(512*$i))) $disk; hexdump -C -n 128 -s $((0x130001d + (512*$i))) $disk; done; } | grep -B 1 -A 5 d00d; echo "---------------------"; done

André

0 Kudos
nazim109
Contributor
Contributor

Dear 

Please have the output of this. 

[root@localhost:~] offset="128 2048"; for dev in `esxcfg-scsidevs -l | grep "Console Device:" | awk {'print $3'}`; do disk=$dev; echo $disk; partedUtil getptbl $disk; { fo
r i in `echo $offset`; do echo "Checking offset found at $i:"; hexdump -n4 -s $((0x100000+(512*$i))) $disk; hexdump -n4 -s $((0x1300000+(512*$i))) $disk; hexdump -C -n 128
-s $((0x130001d + (512*$i))) $disk; done; } | grep -B 1 -A 5 d00d; echo "---------------------";done
/vmfs/devices/disks/naa.61866da07306b2001c4bb566bc470bf1
Error: The backup GPT table is not at the end of the disk, as it should be. This might mean that another operating system believes the disk is smaller. Fix, by moving the backup to the end (and removing the old backup)?
Warning: Not all of the space available to /dev/disks/naa.61866da07306b2001c4bb566bc470bf1 appears to be used, you can fix the GPT to use all of the space (an extra 4300800 blocks) or continue with the current setting? This will also move the backup table at the end if is is not at the end already. diskSize (2924216320) AlternateLBA (2919915519) LastUsableLBA (2919915486) NewLastUsableLBA (2924216286)
gpt
182024 255 63 2924216320
1 128 2919915480 AA31E02A400F11DB9590000C2911D1B8 vmfs 0
---------------------
/vmfs/devices/disks/t10.ATA_____SanDisk_SDSSDA240G______________________173566803571________
gpt
29185 255 63 468862128
1 64 8191 C12A7328F81F11D2BA4B00A0C93EC93B systemPartition 128
5 8224 520191 EBD0A0A2B9E5443387C068B6B72699C7 linuxNative 0
6 520224 1032191 EBD0A0A2B9E5443387C068B6B72699C7 linuxNative 0
7 1032224 1257471 9D27538040AD11DBBF97000C2911D1B8 vmkDiagnostic 0
8 1257504 1843199 EBD0A0A2B9E5443387C068B6B72699C7 linuxNative 0
9 1843200 7086079 9D27538040AD11DBBF97000C2911D1B8 vmkDiagnostic 0
2 7086080 15472639 EBD0A0A2B9E5443387C068B6B72699C7 linuxNative 0
3 15472640 468862094 AA31E02A400F11DB9590000C2911D1B8 vmfs 0
---------------------

0 Kudos
a_p_
Leadership
Leadership

Can you please run the following 4 commands, and post the output?

hexdump -C "/dev/disks/naa.61866da07306b2001c4bb566bc470bf1" | grep -i 01400000 -A20 -B20
hexdump -C "/dev/disks/naa.61866da07306b2001c4bb566bc470bf1" | grep -i 01310000 -A20 -B20

partedUtil getptbl "/vmfs/devices/disks/naa.61866da07306b2001c4bb566bc470bf1"
partedUtil getUsableSectors "/vmfs/devices/disks/naa.61866da07306b2001c4bb566bc470bf1"

André

0 Kudos
nazim109
Contributor
Contributor

Dear,

 

This message is showing after few minutes of hex dump.

 

error:

 

Remote side sent SSH2_MSG_EXT_ INFO after

USERAUTH_SUCCESS

0 Kudos
nazim109
Contributor
Contributor

Here the another 2 command output:

[root@localhost:~] partedUtil getptbl "/vmfs/devices/disks/naa.61866da07306b2001c4bb566bc470bf1"
Error: The backup GPT table is not at the end of the disk, as it should be. This might mean that another operating system believes the disk is smaller. Fix, by moving the backup to the end (and removing the old backup)?
Warning: Not all of the space available to /dev/disks/naa.61866da07306b2001c4bb566bc470bf1 appears to be used, you can fix the GPT to use all of the space (an extra 4300800 blocks) or continue with the current setting? This will also move the backup table at the end if is is not at the end already. diskSize (2924216320) AlternateLBA (2919915519) LastUsableLBA (2919915486) NewLastUsableLBA (2924216286)
gpt
182024 255 63 2924216320
1 128 2919915480 AA31E02A400F11DB9590000C2911D1B8 vmfs 0

 


[root@localhost:~] partedUtil getUsableSectors "/vmfs/devices/disks/naa.61866da07306b2001c4bb566bc470bf1"
Error: The backup GPT table is not at the end of the disk, as it should be. This might mean that another operating system believes the disk is smaller. Fix, by moving the backup to the end (and removing the old backup)?
Warning: Not all of the space available to /dev/disks/naa.61866da07306b2001c4bb566bc470bf1 appears to be used, you can fix the GPT to use all of the space (an extra 4300800 blocks) or continue with the current setting? This will also move the backup table at the end if is is not at the end already. diskSize (2924216320) AlternateLBA (2919915519) LastUsableLBA (2919915486) NewLastUsableLBA (2924216286)
34 2924216286

0 Kudos
a_p_
Leadership
Leadership

I could be wrong, but to me this looks like a disk drive issue.

To rule out an ESXi (e.g. driver) issue, you could try an boot the system using a Live Linux, to see whether the HexDump works.
You may of course change the path to the disk device for the command.

André

0 Kudos
NateNateNAte
Hot Shot
Hot Shot

This certainly reads along like a disk failure.  Which is unfortunate, but can you access this disk any other way, and migrate vmdk files somewhere else?

0 Kudos
nazim109
Contributor
Contributor

Dear André

without hexdump is there any way to recover it?

 

0 Kudos
a_p_
Leadership
Leadership

It's actually not about hexdump, or any other command. It's basically about finding out whether the disk can be accessed at all using a Live Linux.

André

0 Kudos
nazim109
Contributor
Contributor

Dear,

No change last 1 hour.

 

nazim109_0-1705414014790.png

 

0 Kudos