VMware Cloud Community
brucecmc
Contributor
Contributor
Jump to solution

VM Power On error. No space left on device.

Hey folks.

I'm running ESX 3.5, have several (roughly 15) VM's built in a 2 node ESX Cluster. All VM's are shut down with the exception of the 2 I'm targeting to work with. This is a T&E enviornment.

I replicated from our remote site the VMFS volumes and mounted them locally.

I'm able to access the SAN storage, the datastores appear fine.

I can power up one of the VM's, but the other pops the error No space left on device.

The VM had 947mb of space left on its virtual disk, so I thought possibly this to be the issue. I added an extent to the VM and it now displays there is 14GB left on the virtual disk.

however, still generates the No Space left on device.

Where is this being generated from and how do i resolve it??

thanks

Bruce

Reply
0 Kudos
1 Solution

Accepted Solutions
Texiwill
Leadership
Leadership
Jump to solution

Hello,

Which did you 'extend'. The size of the VMDK or the size of the VMFS? If you extended the vmdk, that will have no impact. If you extended the LUN on which the VMFS resides, that is also not sufficient as the filesystem (VMFS) has been untouched. To extend a VMFS you need to add an 'extent' using the VIC. Or you can take everything off the VMFS and then recreate the VMFS, then put everything back.

What is the output of vdf -h?


Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

--
Edward L. Haletky
vExpert XIV: 2009-2023,
VMTN Community Moderator
vSphere Upgrade Saga: https://www.astroarch.com/blogs
GitHub Repo: https://github.com/Texiwill

View solution in original post

Reply
0 Kudos
9 Replies
Texiwill
Leadership
Leadership
Jump to solution

Hello,

Run 'vdf -h' on your ESX hosts. How full is the VMFS? You may not be able to create the .vswp file which is the size of your assigned memory. It certainly sounds like your VMFS is too full.


Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

--
Edward L. Haletky
vExpert XIV: 2009-2023,
VMTN Community Moderator
vSphere Upgrade Saga: https://www.astroarch.com/blogs
GitHub Repo: https://github.com/Texiwill
Reply
0 Kudos
brucecmc
Contributor
Contributor
Jump to solution

Yes,

looked at it and it shows 435mb space remaining (which I dont understand since I added the extent of 14GB). Is it necessary to rescan the scsi bus for the storage? the vdf -h reveals 99% usage with 435mb space remaining (which of course isnt going to accomodate the swap for the memory of 2gb)...

I'm still not clear on how to recover from this...if adding an extent doesnt provide the additional space???

thanks for the quick response.

Bruce

Reply
0 Kudos
Texiwill
Leadership
Leadership
Jump to solution

Hello,

If you add an extent then yes it is required to rescan. You need to first to present the extent then rescan to see the additional space, Then assign that as an Extent to your existing VMFS.


Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

--
Edward L. Haletky
vExpert XIV: 2009-2023,
VMTN Community Moderator
vSphere Upgrade Saga: https://www.astroarch.com/blogs
GitHub Repo: https://github.com/Texiwill
Reply
0 Kudos
brucecmc
Contributor
Contributor
Jump to solution

well, actually, yes I did all that...i meant after the extent is presented to the VM...I see that it shows the extension of the original disk (went from 118 gb to 137gb)...But, when i try to add any more than 1gb of mem to it, it bombs with the "no space left" error...

I thought be extending the disk out, it would over come this behavior... No???

bruce

Reply
0 Kudos
Texiwill
Leadership
Leadership
Jump to solution

Hello,

Which did you 'extend'. The size of the VMDK or the size of the VMFS? If you extended the vmdk, that will have no impact. If you extended the LUN on which the VMFS resides, that is also not sufficient as the filesystem (VMFS) has been untouched. To extend a VMFS you need to add an 'extent' using the VIC. Or you can take everything off the VMFS and then recreate the VMFS, then put everything back.

What is the output of vdf -h?


Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

--
Edward L. Haletky
vExpert XIV: 2009-2023,
VMTN Community Moderator
vSphere Upgrade Saga: https://www.astroarch.com/blogs
GitHub Repo: https://github.com/Texiwill
Reply
0 Kudos
RParker
Immortal
Immortal
Jump to solution

> (went from 118 gb to 137gb)...But, when i try to add any more than 1gb of mem to it, it bombs with the "no space left" error...

you overcommited your LUN space. Check the SAN volume. I bet the space you have allocated for the LUN exceeds the space the VOL has reserved....

Look at it closely and verify that the Volume has more than adequate space to hold the TOTAL space of the LUN (not the current space). Then you also have to leave a few gigs extra for wiggle room.

brucecmc
Contributor
Contributor
Jump to solution

Hi Edward,

I used VC to add an extent to the existing datastore. After doing so, the datastore shows it grew from 120gb to 134Gb (which would be correct based on the size of the LUN I used). however, when I attempt to add memory, beyond 1GB, it fails to start with "no space left" though the datastore clearly shows 14gb of space left...

I reviewed VC to see who the owner of the VM was. Turns out, it is ESX01.

When i ran vdf -h the following was the result:

ESX01 = Size: 137GB Used: 136GB Avail: 947mb % used 99% (THIS IS/WAS INCORRECT)

ESX02 = Size: 137GB Used: 123GB Avail: 14GB % used 89%

A disparity I note between the 2...These are clustered ESX Hosts...I performed a rescan of the hba on ESX01 and now the output of vdf -h reads the same for both ESX Hosts...

After the rescan of the hba bus on ESX01, it straightened it out...

So, the crux of the problem was that the virtual disk for this VM did not have sufficient space to create the vswp to accomodate the virtual memory...After adding the extent to increase the VMFS size, this provided sufficient space to create the vswp. this was the initial problem. That was coupled with ESX01 not recognizing the extent had been added.

I have read in the past that using Extents is not a recommended practice (forget the reason). But since this is T&E, I'll live with it...

Is there a more efficient/better method to over come this particular sizing issue that you are aware of?

Thanks for the pointers/guidance.

Bruce

Reply
0 Kudos
Texiwill
Leadership
Leadership
Jump to solution

Hello,

I reviewed VC to see who the owner of the VM was. Turns out, it is ESX01.

Ah a cluster. When you make a change to the Data Stores on one host you must rescan the adapters on all hosts.

I have read in the past that using Extents is not a recommended practice (forget the reason). But since this is T&E, I'll live with it...

THey are a management nightmare generally. To remove an extent you have to delete the entire VMFS.

Is there a more efficient/better method to over come this particular sizing issue that you are aware of?

Other than recreation of the VMFS, there is nothing that I know.


Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

--
Edward L. Haletky
vExpert XIV: 2009-2023,
VMTN Community Moderator
vSphere Upgrade Saga: https://www.astroarch.com/blogs
GitHub Repo: https://github.com/Texiwill
Reply
0 Kudos
brucecmc
Contributor
Contributor
Jump to solution

thanks edward for the responses...

did you get the posted points as the answerer of my question?

Reply
0 Kudos