VMware Cloud Community
mikevbn
Contributor
Contributor

Datastore vanished... how can I get it back?

Hi all,

Not sure if this is the right forum for this question. If not, could someone point me to the right place?

We have a VMWare host, which reports it's VMware ESX Server 3i, 3.5.0, 120505. This has been running fine for a long time (years, possibly). Last night, we noticed that one of the VMs on it was no longer online. Today I've been trying to figure out what's going on. In summary:

- The datastore is on a second HD. The primary HD contains VMWare and a few VMs. These VMs are still running OK, and I can see this datastore fine in the list of datastores. However, this is the only datastore visible.

- The HD with the missing datastore seems to be visible to VMWare. It appears in the BIOS output. I can see it at the console of the VMWare server, and in vSphere, under server -> Configuration -> Storage -> Add Storage... the disk is detected. It appears with the correct capacity, with no space available.

- If I try to go through the Add Storage wizard, I end up with a warning 'The current disk layout will be destroyed. All file systems and data will be lost permanently', which is obviously not what I want.

- If I go to /vmfs/volumes, I can see both VMFS1 (the working drive) and VMFS2 (the non-working one).

I can't figure out what went wrong, or how to recover from it. Any advice would be much appreciated! The person who set up this environment no longer works here, and I have very little experience with VMWare.

Thanks,

Mike

Reply
0 Kudos
5 Replies
a_p_
Leadership
Leadership

Welcome to the Community,

as a first step, run fdisk -lu on the command line, to find out whether the VMFS partition on the second disk has the correct partition type.

Please post the output if the partition type is not "FB" (VMFS)

André

Reply
0 Kudos
mikevbn
Contributor
Contributor

Hi André, thanks for the reply!

It looks OK. In total, the info reads:

Device: /dev/disks/vmhba32:0:0:1

Boot:

Start: 128

End: 293041664

Blocks: 146520768+

ID: fb

System: VMFS

Is that as you'd expect? Can you suggest a next step?

Thanks,

Mike

Reply
0 Kudos
a_p_
Leadership
Leadership

vmhba32 !? Are you using local storage or iSCSI?

The output of the command looks ok. The next step would be to perform a rescan on the adapter and check the entries for this rescan in /var/log/messages.

André

Reply
0 Kudos
mikevbn
Contributor
Contributor

This machine has a pair of SATA drives, both connected directly via the on-board SATA adapter. I'm really not familiar with iSCSI etc, unfortunately, though I can investigate if it's important?

I rescanned the adapter with the matching name ('vmhba32'), and then had to restart the machine (for an unrelated reason). Looking at the log, I found some suspicious-looking entries, but I'm not at all familiar with the way this should look. I've attached the entire log file, and below are some sample entries that might be relevant. Hopefully they indicate something useful...

Thanks very much for the assistance.

Mike

Message was edited by: a.p. - removed pasted log output for better readability, since the log file is attached anyway.

Reply
0 Kudos
a_p_
Leadership
Leadership

I'm not sure why the disk cannot be read. Instead of risking your data by trying different things (which could make things even worse) I'd strongly recommend you open a call with VMware. They should be able to interpret the error codes for the failed datastore.

André

Reply
0 Kudos