VMware Cloud Community
NNguyen08
Contributor
Contributor

There is no more space for virtual disk '/vmfs/volumes/5b0f6e3d-d0431e09dedb/Server_01/Server_01.vmdk'

Hi

Recently, two of our vms on the same datastore were stopped and threw this error:

There is no more space for virtual disk '/vmfs/volumes/5b0f6e3d-d0431e09dedb/Server_01/Server_01.vmdk'. You might be able to continue this session by freeing disk space on the relevant volume, and clicking Retry. Click Cancel to terminate this session.

VM configs: esxi 6.5, windows server 2012.

The datastore capacity is 2TB, used: 1500GB, allocated 4.5TB (over-allocated).

I did some search and found people got similar issue like on this article (link), which the solution was to delete snapshots to free space on the datastore. However, my case does not have any snapshots for those two servers.

The only possible cause I could think of is that because I've over allocated that datastore. But it doesn't make sense because all VMs on that datastore are thin provisioned, and that datastore still has 600GB free.

Is there anyone has got similar problem and could advise on this? Thanks a lot.

Reply
0 Kudos
11 Replies
ThompsG
Virtuoso
Virtuoso

Hi NNguyen08 and welcome to the community!

Have had similar issues because of the way vCenter now uses the Inventory data (no longer has an Inventory service but the data is being used). In our case we still had plenty of space however one our our ESXi servers wasn't see the correct datastore size. Easiest option is to go to each ESXi server and rescan storage.

You could check the space the ESXi server sees before doing this by running vdf -h to confirm if this is indeed your issue.

Kind regards.

Reply
0 Kudos
NNguyen08
Contributor
Contributor

Hi ThompsG

Thanks for the prompt response. Did you mean df -h instead of vdf -h?

The esxi host can see the datastore has 66% free with df -h.

On that host, I have 4 vms, and two of them have seen a same issue with the datastore.

I did a rescan before reading your post, I've re-scanned storage on the host again to see if it makes any difference. Will keep you posted.

Regards

Reply
0 Kudos
ThompsG
Virtuoso
Virtuoso

Yes sorry - meant df -h - must have been having a senior moment there!

Reply
0 Kudos
NNguyen08
Contributor
Contributor

Hi all, rescanning storage on the host level doesn't help. The message keeps coming back after a day or two.

Is there anyone could help? Thanks a lot.

There is no more space for virtual disk '/vmfs/volumes/5b0f6e3d-d0431e09dedb/Server_01/Server_01.vmdk'. You might be able to continue this session by freeing disk space on the relevant volume, and clicking Retry. Click Cancel to terminate this session.

Reply
0 Kudos
ThompsG
Virtuoso
Virtuoso

Hi NNguyen08,

Could you post the output of df -h for the host that is running that VM?

Thanks.

Reply
0 Kudos
NNguyen08
Contributor
Contributor

Hi ThompsG

Attached per requested.

The host have two HDDs on two datastores: 150GB on ESXI05P-Production and 200Gb on ESXI05P-Archive. Both HDDs are thin provisioned. The error was on the partition on ESXI05P-Archive. Anyway, both datastore have free space.

vmdk_error.jpg

Regards

Reply
0 Kudos
ThompsG
Virtuoso
Virtuoso

When you look in vCenter after getting the message, does the datastore report no space as well?

Reply
0 Kudos
NNguyen08
Contributor
Contributor

Hi ThompsG

No, in vCenter, datastore/monitor/performance or  datastore/summary, shows that the datastore has the same amount of free space as df -h output.

I've migrated the 200Gb HDD on ESXI05P-Archive to ESXI01P-Archive datastore to see if it makes any difference.

ESXI01P-Archive is not being over allocated like ESXI05P-Archive.

Reply
0 Kudos
ThompsG
Virtuoso
Virtuoso

Thanks - please report back how this goes Smiley Happy

Reply
0 Kudos
NNguyen08
Contributor
Contributor

Hi all

Updates on this case. After migrating hdds to other datastores that are not over allocated, the vms are working in a stable mode. No more error on vmdk files.

So I'm pretty sure it's a bug that vsphere host or vcenter does not think datastore has free space for vmdk file on over allocated datastore even when there is actual free space in the datastore. vm hosts are thin provisioned in this case.

Hopefully vmware can address this bug and release a fix.

Cheers

Reply
0 Kudos
ThompsG
Virtuoso
Virtuoso

Awesome and great work. Thanks for feeding back.

Reply
0 Kudos