VMware Cloud Community
stony007_de
Enthusiast
Enthusiast

After KB-1008130 Patch losing Heatbeat after snapshot

Hi Guys,

On friday i was installing the I/O Error Fix Patch " KB-1008130. The install was fine and without problems. Now i got the following problem.

If i start a Backup via vcb, the VMWareTools runs and the VMWare-"Tools Status=ToolsOk".

The snapshot is done and the VMwareTools get the state "Not running"

so the backup cannot work successfull! If i start the vmware tools manually the same effect appears.

any ideas?

stony007_de

Reply
0 Kudos
77 Replies
heky
Contributor
Contributor

It's another damn patch. We are dealing with it for a week, and currently there are only 3 solutions:

1)just log into guest machine and tools change state to OK

2)restart mgmt-vmware service

3)restart vmware tools service via post processing script afterwards vcb snapshot

have a good patching :smileysilly:

Reply
0 Kudos
Leon3020
Contributor
Contributor

I ran into this issue as well, I would say VMware is becoming NO.2 of Microsoft more and more Smiley Sad

Reply
0 Kudos
stony007_de
Enthusiast
Enthusiast

Oh very hard words!!!

Any body get a result of an vmware service request?

I'm still waiting for.. :smileyangry:

Reply
0 Kudos
VMegalodon
Contributor
Contributor

there is a patch for ESXi available with build 14319 that should fix it, but not for ESX

i did not hear anything from VMWare!

Reply
0 Kudos
cf13
Enthusiast
Enthusiast

I have exactly the same problem, but in my configuration (I use Networker for backups) the only problem is that status of Tools displays as "Not running". Lucky for me, all backups on machines with "Not running" Tools works fine.

Reply
0 Kudos
stony007_de
Enthusiast
Enthusiast

Luck for you, but i think you are in the minority.

if you use vcb the vmware tools must run! if it isn't vcb cant find the vmname, it''s only available via ip address!

Reply
0 Kudos
cf13
Enthusiast
Enthusiast

In my environment Tools are running on all VM's that shows "not running" status in VC. It's hard to check if Tools are running or not, because logging in to VM automatically changes Tools status from "not running" to "Tools OK". But you can use Computer Management Snap-in and connect to remote computer (VM with "not running" Tools) and check services. In my environment VMware Tools Service is running even if VC shows "not running"

Reply
0 Kudos
VMegalodon
Contributor
Contributor

Here is the reaction of VMWare:

Apologies for the delay.

I have determined that this issue may be due to a recently discovered bug applying patch ESX350-200901401-SG. The only way to get the VMware tools back to running status is to restart mgmt-vmware service , looks like some tools functionality is getting broken when it is called by the VCB.

Restarting mgmt-vmware service

When this operation is performed, the ESX will appear as 'unresponsive' in VC until the service restarts and sends heartbeats to VC

1) Ensure VMs StartUp/Shutdown are set to 'Disabled' in VC

Configuration Tab>Virtual Machines StartUp/Shutdown state 'Disabled'

2) On the ESX server as user "root" type 'service mgmt-vmware restart'

I will place this SR against the bug report and advise you of any updates.

Reply
0 Kudos
monderick
Enthusiast
Enthusiast

also need a fix for this issue.

vm's randomly display vmtools as 'not running' after running VCB, since applying the mentioned patch.

will try the service mgmt-vmware restart workaround, as Virtual Machines StartUp/Shutdown state was already set to 'Disabled'.

Reply
0 Kudos
turkina
Contributor
Contributor

I suppose until this is fixed all we can do is create a cron job to restart mgmt-vmware as often as needed.

VCP3/4/5, VCAP5-DCA
Reply
0 Kudos
Jan_MS
Contributor
Contributor

We have the same problemes here and i will open a support call.

Reply
0 Kudos
mk75
Contributor
Contributor

We have this problem too.

When we see a VM that have lost its vmtools heartbeat there are different info retrieved by the VI client depending on how you connect to your host.

If we connect directly to the ESX host were the VM is located it report back as running with heartbeat, IP addr etc.

When we connect to VC it say tools not running.

We have a case opened regarding this and apparently there have been some people reporting about this problem before us.

(as early as 6th of february)

Vmware have also confirmed that this problem is moved into their bug database for further analysis.

However we have still not received any info when this problem may be solved.

/mk75

Reply
0 Kudos
kooltechies
Expert
Expert

Hi All,

After seeing all the updates on this thread it's clear that this issue is a cosmetic UI display issue on VI Client connected with VC server it doesn't happen on the VI client connected to the ESX server and there is no problem in the working of the VCB, all the VCB operations are successful.

So in summary there is no VCB functionality loss we see due to this cosmetic UI display issue.

Thanks,

Samir

Blog : http://thinkingloudoncloud.com || Twitter : @kooltechies || P.S : If you think that the answer is correct/helpful please consider rewarding points.
Reply
0 Kudos
Morten_Dalgaard
Enthusiast
Enthusiast

Actually, there is loss of functionality.

Using vcbvmname to try and resolve a VM by IP for an example will fail when VC shows tools as not running.

At least one backup application (Netbackup) uses this in the backup process, and will fail the backup if it is unable to resolve the VM by IP.

Reply
0 Kudos
mk75
Contributor
Contributor

Hi.

Its no cosmetic error/problem.

VCB do not work after this patch is applied.

What happen is that it all fail right after the snapshot is created.

/mk75

Reply
0 Kudos
monderick
Enthusiast
Enthusiast

there is most definately a loss in functionality.

VCB backups randomly fail for vm's with vmtool status of 'not running'.

as soon as i recycle the vmware-mgmt service of the host ESX server, they are able to succeed.

Reply
0 Kudos
kooltechies
Expert
Expert

Hi,

All the VCB tests that I have carried out after this patch have been successful with every possible method , using vm ip , vm name , vm uuid. I have run VCB in a loop multiple times and have not faced any issue.

Thanks,

Samir

P.S : If you think that the answer is helpful please consider rewarding points.

Blog : http://thinkingloudoncloud.com || Twitter : @kooltechies || P.S : If you think that the answer is correct/helpful please consider rewarding points.
Reply
0 Kudos
turkina
Contributor
Contributor

That's nice.

My environment, on the other hand, will randomly fail to backup using Netbackup 6.5.3 and VCB 1.5.

VCP3/4/5, VCAP5-DCA
Reply
0 Kudos
cf13
Enthusiast
Enthusiast

Then we are both lucky. As I wrote before my VCB works fine for every VM that shows "not running" status in VC. I use Networker.

Reply
0 Kudos
CMXMartin
Contributor
Contributor

Same problem here:

Using VI Client and VC Server Build 119598 connected to ESX 3.5 143128

VMTools show as "Not Running", however my backups still function using VCB 1.5, Backup Exec 12.5 (all available Hotfixes) and BE Agent for VMWare.

Connecting to Remote Windows MMC shows the service as running, in VC it shows as "Not Running"

Connecting VI Client directly to ESX host shows VMTools as "OK".

Reply
0 Kudos