VMware Cloud Community
uyozTic
Enthusiast
Enthusiast
Jump to solution

enlarged disk too much, now VM won't boot

i have a VM using half of avail space on a 500GB drive. i wanted to expand it to use the remaing ~250gb. i entered the max value listed in hardware settings for VM disk. appears that was too much and now a swap partition can't load.

can i save this VM another way or is shrinking the only way? 

from another post here i saw this.

  virtualman.wordpress.com/2016/02/24/shrink-a-vmware-virtual-machine-disk-vmdk/

1. but my VM won't even boot up so i can go and shrink disk.
2. from the instructions it mentions migrate or copy...do i have to migrate or copy it to another datastore? would prefer to use the whole drive it is currently on.

tia

following are the error messages.

----------------------------------------------------------------
msg when power on attempt:   Failed to power on virtual machine [name]. Module 'MonitorLoop' power on failed. Click here for more details."

-----------------------------------------------------------------
Failed - Module 'MonitorLoop' power on failed.
Errors

    Module 'MonitorLoop' power on failed.
    Failed to extend swap file /vmfs/volumes/[name of swap vswp].vswp from 0 KB to 12582912 KB : No space left on device
    Current swap file size is 0 KB.
    Failed to extend the virtual machine swap file
    Could not power on virtual machine: No space left on device.
    Failed to power on VM.
    Failed to start the virtual machine.

msg when editing VM hardware configuration:   
    Enter a disk size which is larger than its original capacity.

-----------------------------------------------------------------
storage stat for this disk:

STORAGE  FREE: 444 MB
                       100%
USED:             465.07 GB
CAPACITY:      465.5 GB
-----------------------------------------------------------------

Reply
0 Kudos
1 Solution

Accepted Solutions
continuum
Immortal
Immortal
Jump to solution

The predictable order is:

1. power off VM

2. start the vmkfstools -i copy to new location

3. if vmkfstools reports no errors go on

4. unregister VM

5. change path to the copied vmdk - preferably by editing the vmx (this has the most predictable results)

6. register VM again

7. if it boots correctly - you can then delete the old vmdk from disk.

Step 2 and 5 are also available via the GUI but none of them is as reliable as the manual approach (at least in my experience)

If you use the vmx-edit you just change the entry in the line

scsi0:0.fileName = "name.vmdk"

to

scsi0:0.fileName = "/vmfs/volumes/NAS/dir/name.vmdk"

and so make sure that no other parameters are changed without you being aware of it.

---------------------------------------------------------------------------------------------------------

Was it helpful? Let us know by completing this short survey here.


________________________________________________
Do you need support with a VMFS recovery problem ? - send a message via skype "sanbarrow"
I do not support Workstation 16 at this time ...

View solution in original post

Reply
0 Kudos
20 Replies
IRIX201110141
Champion
Champion
Jump to solution

To get the VM back and running

1. You can set the memory reservation to 100% (Default is zero) which ends up in a 0bytes swap file

2. you can just migrate the VM $Home to another Datastore where enough space exists for creating the swap file during VM startup time

I would prefer method #2. If you have a vCenter its just a few clicks.

About the disk. If you have enough space on a different datastore

1. You can move the disk into another datastore

2. You can move the disk and convert it to ThinProvisioning rather than using lazy/zero Thick. Later you can move it back into the original datastore. Keep in mind not to expand the Windows NTFS filesystem to 100% and choose only your 250GB. So you will be save that the vDisk cant grow up to its datastore limits.

3. Restore from Backup

Regards,

Joerg

Reply
0 Kudos
continuum
Immortal
Immortal
Jump to solution

> would prefer to use the whole drive it is currently on.

That is indeed possible - unless you already expanded the partitions inside of the guest.

The instructions are too scary for most admins so I dont post it here - feel free call me via skype and I do it for you.

But your main problem is running out of space anyway - so rather use the approach that has been suggested by Irix64.

I use dd to cutoff the flat.vmdk in place - so this will work without moving the VM at all...

Ulli


________________________________________________
Do you need support with a VMFS recovery problem ? - send a message via skype "sanbarrow"
I do not support Workstation 16 at this time ...

Reply
0 Kudos
uyozTic
Enthusiast
Enthusiast
Jump to solution

Joerg,

>>2. You can move the disk and convert it to ThinProvisioning rather than using lazy/zero Thick. Later you can move it back into the original datastore. Keep in mind not to expand the Windows NTFS filesystem to 100% and choose only your 250GB. So you will be save that the vDisk cant grow up to its datastore limits.

will this work with a Ubuntu based guest? it is not windows. i have gparted to boot from virtual CD for this VM if that helps.

when converting to Thin Provisioning, is that done via Web Client of the stand alone converter?

thanks for reply

Reply
0 Kudos
uyozTic
Enthusiast
Enthusiast
Jump to solution

Ulli,

> would prefer to use the whole drive it is currently on.

>>That is indeed possible - unless you already expanded the partitions inside of the guest.

i have not expanded inside the guest after the recent "edit hardware expansion".  i was about too with gparted but as you know the VM would not boot.

>>But your main problem is running out of space anyway - so rather use the approach that has been suggested by Irix64.

OK, i can try Irix64's method.  i have a free 500GB drive in the host available that i could move it too but i guess that isn't enough space as the current disk this VM is on is 500GB.  the current size of the VMDK 498,216,206,336.  i have a few TB's free on my NAS i can move it too at least temporarily.

is moving the VM it like migrating "registering an existing VM"?

>>I use dd to cutoff the flat.vmdk in place - so this will work without moving the VM at all...

i copied the VM to my NAS as a backup so i could experiment cutting on the flat.vmdk though i'll try Irix64 suggestion as soon as you two reply about my questions on moving.

i can certainly empathize with other admins and "scariness" of your method.   not so sure about skype, but i could try that if i mess up Irix64 method. 

thanks!

Reply
0 Kudos
continuum
Immortal
Immortal
Jump to solution

Rule number one: do not run out of space on a VMFS volume.

>  i have a few TB's free on my NAS i can move it too at least temporarily.

Go for that - if your VMFS is filled so far that it cant even allocate a vmem file then really first make sure to get some free space on that volume.

I recommend to copy the 500gb that you tried to expand to your free space on NAS preferably from commandline with vmkfstools.

vmkfstools -i /vmfs/volumes/fulldatastore/directory/name.vmdk /vmfs/volumes/emptynas/newdir/name.vmdk

That makes sure nothing goes wrong while doing the copy.

If the vmdk arrives without errors it should be fine.

After that reconfigure your VM and try it.

If it works you can delete the old copy.


________________________________________________
Do you need support with a VMFS recovery problem ? - send a message via skype "sanbarrow"
I do not support Workstation 16 at this time ...

Reply
0 Kudos
uyozTic
Enthusiast
Enthusiast
Jump to solution

thanks, It is cloning now only at 28% ...

>>After that reconfigure your VM and try it.

when you say reconfigure, i would go to edit the VM and add a new hard disk pointing to location for cloned vmdk on the NAS ?

Reply
0 Kudos
continuum
Immortal
Immortal
Jump to solution

The predictable order is:

1. power off VM

2. start the vmkfstools -i copy to new location

3. if vmkfstools reports no errors go on

4. unregister VM

5. change path to the copied vmdk - preferably by editing the vmx (this has the most predictable results)

6. register VM again

7. if it boots correctly - you can then delete the old vmdk from disk.

Step 2 and 5 are also available via the GUI but none of them is as reliable as the manual approach (at least in my experience)

If you use the vmx-edit you just change the entry in the line

scsi0:0.fileName = "name.vmdk"

to

scsi0:0.fileName = "/vmfs/volumes/NAS/dir/name.vmdk"

and so make sure that no other parameters are changed without you being aware of it.

---------------------------------------------------------------------------------------------------------

Was it helpful? Let us know by completing this short survey here.


________________________________________________
Do you need support with a VMFS recovery problem ? - send a message via skype "sanbarrow"
I do not support Workstation 16 at this time ...

Reply
0 Kudos
IRIX201110141
Champion
Champion
Jump to solution

Ulli is the one and only who will be able to "cut" off the unused part of the vmdk by using "dd" Smiley Happy This is only possible  because you havent expand the GuestOS filesystem yet.

If you follow my way be sure that you use the "-d thin" when using the vmkfstools command. This will create a copy of your VMKD which is way smaller than your 500GB vmdk.

You say its a linux VM... if you create a new linux VM you can mount the existing linux vmdk into the new one and just copy the files around. You can do this with every Linux live CD. Yes you need some time and effort to get grub back working. My linux admin use a propper "tar" command to migrate phys. linux servers into a VM in the old days.

Regards,

Joerg

Reply
0 Kudos
uyozTic
Enthusiast
Enthusiast
Jump to solution

>>Ulli is the one and only who will be able to "cut" off the unused part of the vmdk by using "dd"

kinda thought that myself. 😉  if i can't get it to boot after trying the suggested methods, i'll ask him about doing that.

>>If you follow my way be sure that you use the "-d thin" when using the vmkfstools command. This will create a copy of your VMKD which is way smaller than your 500GB vmdk.

i didn't add the "-d thin" option. the command returned this during process:    "Destination disk format: Thin"

i assumed it defaulted to "thin"

if i need to add that, let me know and i can rerun the command.

>>You say its a linux VM... if you create a new linux VM you can mount the existing linux vmdk into the new one and just copy the files around. You can do this with every Linux live CD.

i do this and have saved a few VM's that way. and when using gparted i load it live from virtual CD. works great. someone on these forums suggested it awhile back.

>>Yes you need some time and effort to get grub back working.

if Ulli's method works i may not need to tweak grub2, hoping. hope try his latest reply instructions in a couple hours and see if it boots up.

appreciate your help!

Reply
0 Kudos
uyozTic
Enthusiast
Enthusiast
Jump to solution

great!

thanks much for the instructions, will try this in a few hours... i'll let you know results

Reply
0 Kudos
uyozTic
Enthusiast
Enthusiast
Jump to solution

unregistering, edit vmx, registering went ok.

when booting though not enough space for swap file. swap file still pointing to original disk, not NAS

     Failed to extend swap file /vmfs/volumes/[folder id value]/[name]-3410eb2d.vswp from 0 KB to 12582912 KB : No space left on device

do i use vmdktools to copy swap file and then change this key's value in vmx file to , "sched.swap.derivedName=/to NAS vm folder/swapfilename.swp" ?

seems like that work or not ??

tia

Reply
0 Kudos
IRIX201110141
Champion
Champion
Jump to solution

vmkfstools creates a copy ... it does not move files around which means you the original vmdk in place and not space left.

1. Set the Memory Reservation of the VM to 100% and the VM will start

2. You can specify a new location for the the $Home of your VM which means the swap is created on another datastore. Your vDisk can stay in place

3.. If your copy does work

- Ulli can try to cut of the end of the file

or

- You can delete the original VMDK... and than you can copy the thin vmdk back to the original datastore.

Regards,

Joerg

Reply
0 Kudos
continuum
Immortal
Immortal
Jump to solution

It would be smart if you unregister the vmx once again and then move the vmx file to the new home.

This way those suckers dont feel homesick ....


________________________________________________
Do you need support with a VMFS recovery problem ? - send a message via skype "sanbarrow"
I do not support Workstation 16 at this time ...

Reply
0 Kudos
uyozTic
Enthusiast
Enthusiast
Jump to solution

bravo! bravo! bravo!!

when vmkfstools copied the name-flat.vmdk file it created two files and named them as name-flat.vmdk and name-flat-flat.vmdk. thus...

   1) when booting esxi complained about name-flat.vmdk so i renamed it to name.vmdk

   2) renamed name-flat-flat.vmdk to name-flat.vmdk and edited name.vmx to reflect that.

   3). i copied the name.vmx file to NAS folder as Ulli suggested & did not delete it from its original location (just in case)

vm then booted! seems to be running ok and accessible.

any post boot vmware things i should do besides expand the disk from inside guest now?

muchas gracias

Reply
0 Kudos
uyozTic
Enthusiast
Enthusiast
Jump to solution

hi Joerg,

i set the memory reservation of this VM to 100% it didn't start. but that isn't due to your suggestion not helping. vmware complained about  the same of the vmdk file. the copy worked via Ulli's suggestions. vmkfstools named a file like "vm-name-flat.vmdk"  vs "vm-name.vmdk" and vmware didn't like that. took me a few times to see that error msg to realize what was going on.

it is now booting after Ulli suggested moving the vmdk conf file to NAS dir and me renaming that file.

i'm going to get a new 1TB SSD for on host and likely move this VM to it then or reinstall it. it is a SecurityOnion VM i installed last year and i have a lot of configs on it i had forgotten to write down. my error, i get it.  now that it has booted i got all the setup and config data copied so a reinstall would be rather painless.

thank you very much for your time and help!!

Reply
0 Kudos
IRIX201110141
Champion
Champion
Jump to solution

Ok, but for the docs.

vm.vmdk is the descriptor, a small ascii text file which is human readable and a vm-flat.vmdk which holds the data. They come as a pair and you always work whith the descriptor when working with vmkfstools. If you take a look into the descriptor you can see it referres to the -flat file.

Regards,

Joerg

Reply
0 Kudos
uyozTic
Enthusiast
Enthusiast
Jump to solution

you both gave great help and couldn't have gotten this to work without both your help.. Ulli gave more detailed instructions to get the files copied and config.

does "correct help" allow to splt ?

Reply
0 Kudos
continuum
Immortal
Immortal
Jump to solution

> vmkfstools named a file like "vm-name-flat.vmdk"  vs "vm-name.vmdk" and vmware didn't like that.

Have you never seen flat.vmdks before ?

That will be a problem in future that you should fix as soon as possible.

Download WinSCP and connect it to your ESXi.

Go to any VM in one of your datastores and see how the files are listed in the WinSCP-GUI.

Now go to the same directory in the Vsphere-webinterface.

The list will look different and you really must understand that this display is oversimplified and dumbed down.

Confusing ? - yes - but a decade ago the developers really thought that oversimplifying the file display would make things easier.

So the lesson to learn for you is: when ever you want to inspect a VM closely and want predictable results step 1 is: do not use vSphere datastorebrowser and switch to WinSCP.

Does that make sense ? - no - of course not.


________________________________________________
Do you need support with a VMFS recovery problem ? - send a message via skype "sanbarrow"
I do not support Workstation 16 at this time ...

Reply
0 Kudos
uyozTic
Enthusiast
Enthusiast
Jump to solution

got it. thx

Reply
0 Kudos