Remove snapshot hang after a backup

[proussel]

Hello all!

I'm using vRanger to backup my VMs and recently every time I try to backup a big VM, the task "remove snapshot" hang during 10-15 minutes and I cannot access my VM or do anything... I asked vizioncore for support they just told me to put the CPU reservation at 1500 and increase the memory of the console... I did that but still hanging... I read on vizioncore forum that it's a issue from ESX and a patch was suppose to change that (the posts were from 2007), but I also installed the last few weeks ago and no changes... It is becomming critical for us because it's a production VM, so we had to stop the backup to continue...

I can see the task running in the VIC and I can see the performances and everything of my VM (even if it's 0 :s) but I can't stop the task or restart the VM

Would really appreciate any help or idea on this topic!

Thanks and regards,

Pierre

[Texiwill]

Hello,

From the CLI, restart the mgmt-vmware service using: 'service mgmt-vmware restart' once you do that you will most likely have to commit the snapshot by hand using either snapalert (vmprofessional.com) or snaphunter (xtravirt.com), unless it appears within the VIC, then you can commit it.

If that does not work, then it is time to look at the logfiles on the ESX host. /var/log/vmkernel specifically as well as /var/log/vmware/hostd.log

Best regards,
Edward L. Haletky
VMware Communities User Moderator, VMware vExpert 2009, DABCC Analyst
====
Now Available on Rough-Cuts: 'VMware vSphere(TM) and Virtual Infrastructure Security: Securing ESX and the Virtual Environment'
Also available 'VMWare ESX Server in the Enterprise'
SearchVMware Pro|Blue Gears|Top Virtualization Security Links|Virtualization Security Round Table Podcast

[proussel]

Hello, thanks for your reply! As it's a production VM I still didn't have the opportunity of trying this, I'll try today, but I have a big issue, I can't afford to be in front of my VM, wait for the end of the backup and do something manually because:

1- I can't afford to have any downtime (even if it's few minutes)

2- The backup will run at night, I can wake up to do that

Is there any way to remove the hang?

EDIT: I tryed to restart vmware services with your command, but it causes an error and my VM is now in the state "invalid"

Thanks and regards

Pierre

[athlon_crazy]

On which ESX version your problematic VM was running currently?

VMware newbie..

Zen Systems Sdn Bhd

www.no-x.org

[proussel]

3.5

[depping]

Which update? Any additional patches installed?

Duncan

VCDX | VMware Communities User Moderator

-

---

Blogging:

Twitter:

If you find this information useful, please award points for "correct" or "helpful".

[athlon_crazy]

Yes which update, coz my ESX build 64607 sometime went wrong to the VMs when I restarted mgmt-vmware services.

VMware newbie..

Zen Systems Sdn Bhd

www.no-x.org

[proussel]

I installed the update 2 found in this page:

(so obviously build 103908)... I had to install this update otherwise 2 of my network cards were not recognized but restarting the vm-services don't seems viable for me as I would need a physical intervention, which is not possible each time

I also ask for my issue to a vranger guy, he ask me to create a snapshot manually and when I tryed to remove it I had exactly the same issue, the removal hanged at 95% and I could'nt access my vm during this time, so I guess it's more related to ESX than to vRanger

[Nick_F]

Not had any snapshotting issues with U2 here, how much free disk space do you have in your datastore?

[proussel]

Like 500Gb, and my VM is 80Gb... No issue on this side

[proussel]

No one have any ideas? Vizioncore support is giving me some really hard time and they don't help at all... Anyone with a solution would save my life

Thanks and regards

[TB_Rich]

I have just had the same kinda issue, 15mins and hanging on snapshot removal's. VM un responsive during that time.

For us, as were running NFS (are you perhaps???) we needed to have a patch (included from 3.5 U3) and enable a /etc/vmware/config change (add line prefvmx.consolidateDeleteNFSLocks = "TRUE").

Cheers

[proussel]

Thanks for the reply, we will try to install the last patch and change the configuration as soon as we can afford to have downtime on our server and I'll post the result here.

Thanks and regards