Virtual Machine failed suddenly

[tomjacobchirayi]

Hi,

I am running VMware vSphere 5.0 U2.

The virtual machine in question is running Windows Server 2003 Standard SP2. The vm was having 2 hard disks, which span accross 2 datastores.

Backup Tool: Veeam Backup and Replication v7

Issue.

One fine day, the virtual machine was down with the below waring.

Message from Server01: There is no

more space for virtual disk

/vmfs/volumes/502448f6-3d6b4a1f-ebe9-0026b-

92f4529/gmdb02.quattro.co.za_1/gmdb02.quatt-

ro.co.za_1-000002.vmdk. You might be able to

continue this session by freeing disk space on the

relevant volume, and clicking Retry. Click Cancel

to terminate this session.

1. Clicked cancel and the virtual machine was turned off.

2. Realised that the D drive of the server (this was in Datastore 2) and the datastore is running out of space

3. Received warning alert that the vm requires consolidation. So, consolidated the vm snapshot. It took a while and failed at 99%

4. Removed the affected hard disk from the virtual machine

5. Powered ON the virtual machine and it is working.

But, I need the data from the removed hard disk.

When checking, I could see that there are two delta disks in Datastore 2, and also two snapshot files + the base file.

So, in total there are 5 vmdk files.

./Server01_1_1-flat.vmdk

./Server01_1_1.vmdk

./Server01-ctk.vmdk

./Server01_1_1-000001.vmdk

./Server01_1_1-000001-delta.vmdk

./Server01_1_1-000001-ctk.vmdk

Now, I need to consolidate these files. It is not attached to the live server at the moment. Is there any way to consolidate these files manually.. from SSH... Please help me...

Thanks in advance..

[john23]

Datastore has the space during consolidation??  It requires more space for consolidation operation.

[continuum]

Next time you run into this scenario do NOT answer that question !!!!
Instead inspect the datastore and try to move VMs that are not running away from that datastore.
In worst case power off another VM and copy it via Datastorebrowser to your admin host.

You want to answer RETRY in such a case - but only after you made sure you were able to provide more free space.

Looks like you lost the content of snapshot 000002.vmdk.
In such a scenario I consider the message to consolidate snapshots as too dangerous. Do not attempt consolidations when the datastore is almost filled up.

It is safer to consolidate via vmkfstools and specify another datastore as the target. In your case the consolidation might have destroyed snapshot 000002.vmdk

To consolidate the snapshot 000001.vmdk use
vmkfstools -i Server01_1_1-000001.vmdk /vmfs/volumes/different-datastore/Server01_1_1.vmdk

Even if that fails no harm is done as this procedure creates a copy

Ulli

[tomjacobchirayi]

Hi guys,

It was an urgent requirement, and I could not withstand further downtime to the machine. So, I had to sacrifice the transaction data that happened during the snapshot period. In short, I have the data which was there before the snapshot was taken. All data which were there on the snapshot were lost. Any ways, thanks guys for the suggestions. I will keep this in mind if such situation comes later.

Thanks,

Tom