I've opened a case with HP, they have called me back. Hopefully I'll have an answer by tomorrow.
Can you give me the SR number?
Note also that there is a known bug that snapshots break after you VMotion a VM. So if you VMotion a VM, then you can't take any snapshots of it until you power off or suspend the VM. Perhaps you're seeing this?
Wow okay so that's something that's not documented anywhere and it would've been probably really nice if that was communicated to your techs, because he and I have been scratching our heads at this for two weeks.
The SR is 274985, my tech is Damian.
Yeah, we just found out about it last week. So it's hot off the press.
OK I called the tech I was working with on this and let him know about this.
If this is the case - that VMotion breaks snapshotting - then this is an bug big enough to drive a TRUCK through because as far as I know Consolidated Backup requires snapshots to operate, which means that VMotion breaks VCB. That's rediculously stupid.
Luckily this bug does NOT affect VCB. VCB uses non-memory snapshots, and only memory snapshots are affected by VMotion. But obviously this is still a bad bug.
The bizarre thing is there must be some other condition that has to be in place before this bug occurs, because I could swear we VMotioned a machine and snapshotted it and it worked before. Is there more to this than what's been mentioned here?
My tech told me that it's Bug 118647 but I don't know how to look that up or if that's public information. Could you tell us what conditions are required to cause this bug? Or is it really just my imagination that this worked for a long time?
Bug 118647 is this defect's id in our internal bug tracking system. Only VMware employees have access to it (so I'm not sure what good telling you it's bug 118647 does you, except possibly to indirectly tell you that engineers here are working on it).
My understanding is that if you VMotion a VM and then attempt a snapshot with the "Snapshot the Virtual Machine's memory" box checked, then the snapshot will fail every time. Only after power cycling the VM will snapshots work again (until you VMotion it, of course). You could try doing a quick VMotion and then a snapshot with the "snapshot memory" box checked to verify this.
Note that suspend functionality is not affected. You can suspend a VMotion'ed VM, you just can't take a snapshot. Also, I believe that snapshot functionality is new to ESX 3.0. I don't think we had it in 2.x. Thus this bug couldn't exist in 2.x because we didn't have snapshots. So perhaps that's where your confusion is coming from?
I can only agree, we are experiencing this issue also.
The power off option does generally resolve the issue but again not always. What does seem to always resolve the issue is to power off. Clone the server.. But what a rather nasty painfull routine..
It appears that if a Guest VM has an existing snapshot (yes we have a couple who continue to run dev servers in snapshot) they don't suffer from this issue. Not always but sometimes when you try to VMotion these servers (while they are in snapshot) they fail the VMotion.
I have just Vmotioned one it worked this time..Pot luck it seems..
The power off option does generally resolve the issue
but again not always. What does seem to always
resolve the issue is to power off. Clone the server..
But what a rather nasty painfull routine..
What specific 'issue' are you talking about here? Is it taking a snapshot after VMotion? If so, my understanding is that a poweroff will always fix it. If it's not, then I believe you're hitting a different problem. Is that the case?
It appears that if a Guest VM has an existing
snapshot (yes we have a couple who continue to run
dev servers in snapshot) they don't suffer from this
As long as you don't take any new snapshots, VMotion should be fine.
Not always but sometimes when you try to
VMotion these servers (while they are in snapshot)
they fail the VMotion.
What is the error? Do you have any logs that you can post?
I am seeing this issue as well. I'm suprised I never saw it until now.
The only thing that is new to my environment is that I just enable HA/DRS and clustered my ESX hosts. If I VMotion a VM, I get the error. If I power off/on it works again. If I VMotion it again, it breaks.
I even tried removing my cluster, but I still experience the same problem.
Is a fix planned in the next release? Perhaps I'll file my own SR.
There's no need to file any more SR's. The problem is understood and should be fixed in an upcoming release.
This is a long-standing bug in VMWare, and has been around since at least version 3 of VMWare Workstation, which is when I first encountered it. In the past, it was possible to change a path setting in a particular configuration file for the virtual machine in question. Today, with VMWare 5.x, it is an easier matter. Here is the solution:
1) For the VM in question, go to VM -> Settings -> Options tab -> General in VMWare Workstation (for example)
2) Note the folder specified under Working Directory (where suspend files and snapshots would be stored)
3) Create a subfolder (e.g., call it Snapshots) under the folder in step 2
4) Under General, Browse to the subfolder created in step 3, selecting it as the new location for suspend files and snapshots.
NOTE: Your virtual machine must be powered off to perform the above steps. Once they are completed, suspend should work.
This is a perfect example of shoddy QA processes at VMWare, and a bug that VMWare should have corrected long ago. The bug occurs because when a VMs original hard drive is replaced (e.g., with a cloned drive that is larger), the VMs configuration settings end up pointing to an EMPTY (impossible) path for snapshots and suspends (a kind of snapshot) even though the path shown for the working directory for snapshots and suspends is a real path. This is the sort of bug that could be fixed very easily by any decent software engineer. (Sorry to burst your bubble VMWare, but I believe in stating things as they are in reality.)
P.S. I have been a VMWare BETA tester since the release of Workstation 2, and heavily promote VMWare's products to my clients because on the whole, I think VMWare is a great company with a great deal to offer the entire IT industry through its products.
I have to admit, this is incredibly frustrating too. I love VMWare, but when I hit these kinds of things it really annoys me. Thank God you have at least a hack solution for this bug.
Thank you, thank you, thank you Timothy Huckabay!
Your post just saved me 1/2 day of re-doing an image.