VMware Communities
lpatters35254
Contributor
Contributor

VMware Workstation-10.0.1 fails after RedHat-6 software update.

After updating software on a RedHat-6 system (yum update), workstation-10.0.1 will 'crash' the system.

The update included:

kernel-2.6.32-431

glib2-2.12-1.132

The 'Bug Tracking' tool reports problems with glib2 and kernel 'hang'.

Suggestion?

Thanks!

Tags (1)
83 Replies
xishengzhang
VMware Employee
VMware Employee

I think you could re-install WS10.0.1 after your new update of host OS..

Regards

Reply
0 Kudos
steveadler
Contributor
Contributor

I'm having the same problem and its driving me crazy. Some more info on this.

I have two virutal system, a linux one running fedora 19 and a windows 7 one. The linux virtual system runs fine, but when I boot up the windows 7 it freezes my workstation. When I boot up my windows 7 virtual system, the freeze does not occur until I log into my windows 7 virtual system. But once I log in, it freezes up in less than a minute.

When I my boot red hat enterprise linux system using the previous kernel 2.6.32-358.23.2, and I boot up the windows 7 virtual system, my workstation does not freeze up, but the networking on the windows 7 virtual system does not work.

One thing I'm going to try as a temporary work around is to de-install the vmware tools software package, which hopefully will de-install virtualized aware drivers, which I think may be causing the problem.

I'll keep posting to this thread as I work through this problem.

Reply
0 Kudos
lpatters35254
Contributor
Contributor

Update: I have created a support case with RedHat.

cscarff
Contributor
Contributor

Yes, it does seem to only crash RHEL 6.5 when bootting Window 7 in VMware Workstation 9.0.3 or 10.0.1 on Red Hat Enterprise Linux 6.5. Tried a legacy install with a yum update then a clean install of RHEL 6.5 (x64).

abrt message:  Process /usr/bin/gettings was killed by signal 6 (SIGARBT)

Using Package:  glib2-2.26.1-3.el6

Chris
Reply
0 Kudos
eldonr
Contributor
Contributor

I tested booting Windows XP under 9.0.3 also with the same result -- it always crashed my RHEL6 host right where VMware Tools was apparently starting up during login.  However, unlike Windows 7 and XP, a Windows 98 guest was fine, as was a Fedora 18 guest.  Then I repeated my tests with VMware Workstation 10.0.1 (including creating a new VM of XP SP3 -- no difference) and then with Workstation 9.0.2, all with the same result.  That's when I finally realised it might be the kernel update that changed things, and rolled back to the previous kernel, which solved the problem -- I had unfortunately made the mistake of updating to RHEL 6.5 and VMware Workstation 9.0.3 (from 9.0.2) at the same time, so had plenty of opportunity to make incorrect guesses as to what broke things 😉

The vmcore-dmesg.txt files in my crash dump directories point to "Process vmware" and all end with the same "Call Trace" as follows:

<4>Call Trace:
<4> [<ffffffff8144b2a1>] ? release_sock+0xb1/0xf0
<4> [<ffffffffa0b7a370>] VSockVmciAllowDgram+0xcf4/0x2df0 [vsock]
<4> [<ffffffff8109b2a0>] ? autoremove_wake_function+0x0/0x40
<4> [<ffffffff81448867>] sys_connect+0xd7/0xf0
<4> [<ffffffff810e2067>] ? audit_syscall_entry+0x1d7/0x200
<4> [<ffffffff810e1e5e>] ? __audit_syscall_exit+0x25e/0x290
<4> [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
<4>Code:  Bad RIP value.
<1>RIP  [<00000001ffffffff>] 0x1ffffffff
<4> RSP <ffff88038f923dc0>
<4>CR2: 00000001ffffffff

Thanks to the OP () for creating a support case with Red Hat.  Do you or Red Hat want other RHEL6 + VMware Workstation users to weigh in on the issue, or are comments in this discussion thread sufficient?

Reply
0 Kudos
steveadler
Contributor
Contributor

I'm glad to see someone else having this problem. For some reason, I need to have the latest version of the RHEL kernel. This bug is driving me crazy! My world has collapsed on me since I can't run my windows 7 environment. I hope someone finds a fix to this soon... 😕

(Well, I don't want to say I'm glad your having crashes... Just that I'm not alone with this problem...)

Reply
0 Kudos
lpatters35254
Contributor
Contributor

The RedHat support case's email conversation has begun.

Reply
0 Kudos
steveadler
Contributor
Contributor

Do you have a bug number or way of indicating which is the case that was opened with Red Hat so that I can follow it? thanks.

Reply
0 Kudos
lpatters35254
Contributor
Contributor

The RedHat case number is shown below.

   00986965

Reply
0 Kudos
NobuoSha
Contributor
Contributor

Thanks for the information. My guess is the interaction of vmware tool's 3D feature and the new kernel from RHEL 6.5. Not sure if Red Hat will try to solve this problem or not since they might say that the problem is with using third party software (the kernel module from VMware Workstation). Probably it's a good idea to open up a case with VMware as well). When I have time, I will try to disable the 3D acceleration feature to see if that makes any difference or not. Thanks.

Reply
0 Kudos
steveadler
Contributor
Contributor

I just tried the 2D test and my system crashed.... The bit which did happen was that it didn't crash until I logged in and my desktop was being setup. That's when the crashed happened, so I see why one would think its a video driver bug.

Reply
0 Kudos
lpatters35254
Contributor
Contributor

Below is a copy of the latest email message from RedHat.

Case Title       : System failure after starting VMware-Workstation-10.0.1.

Case Number      : 00986965

Case Open Date   : 2013-11-24 08:38:04

Problem Type     : Other

Most recent comment: On 2013-11-29 08:27:25, Kochukaleekal, Deepu commented:

"Hello Lewis,

gsettings has been included in glib2-2.26.1-3 version which was released with RHEL 6.5 . This is causing the 'GLib-GIO-ERROR' error as it refers to schemas such as 'org.gnome.system.proxy', which are actually not present in system.

The schemas are provided with 'gsettings-desktop-schemas' package which is not available with RHEL 6.5 at the moment.

We have raised a bug report in Red Hat BugZilla addressing this issue. We have proposed to include 'gsettings-desktop-schemas' package in RHEL 6.5 .Our Engineering Team would be working on it.

I am setting a Negotiated Entitlement for 15 days for this case, as Engineering Process needs more time. If you have any concern, please get back to us and we shall do the needful.

I shall keep you updated with the progress on BugZilla.


Reply
0 Kudos
keepertoad
Contributor
Contributor

FWIW, I have three machines that exhibit the same behaviour using the 6.5 kernel, 2.6.32-431.el6.x86_64 on CentOS 6.5, VMware 8.0.6 running XP.  Down grading to the last 6.4 kernel solves the issue but obviously not the best solution.

Thanks for the RH bug number.

Reply
0 Kudos
log4ay
Contributor
Contributor

I am running rhel6.5 and am also having issues with win7 guest. I worked through some diagnostics with rhel engineers and below is their assesment based on my crash report. Looks like vmware engineers need to look into VSockVmciAllowDgram module.

BUG: unable to handle kernel paging request at 00000001ffffffff

IP: [<00000001ffffffff>] 0x1ffffffff

PGD 0

Oops: 0010 [#1] SMP

last sysfs file: /sys/devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0A:00/power_supply/BAT0/charge_full

CPU 0

Modules linked in: bluetooth vmnet(U) parport_pc vsock(U) vmci(U) vmmon(U) ebtable_nat ebtables bridge tpm_infineon autofs4 8021q garp stp llc fuse cpufreq_ondemand acpi_cpufreq freq_table mperf ipt_REJECT iptable_filter xt_CHECKSUM iptable_mangle ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ip_tables ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables vhost_net macvtap macvlan tun kvm_intel kvm uinput ppdev iTCO_wdt iTCO_vendor_support microcode dell_laptop dcdbas arc4 iwldvm parport mac80211 ipv6 iwlwifi cfg80211 rfkill sg uvcvideo videodev v4l2_compat_ioctl32 i2c_i801 lpc_ich mfd_core shpchp e1000e ptp pps_core snd_hda_codec_hdmi snd_hda_codec_idt snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm snd_timer snd soundcore snd_page_alloc ext4 jbd2 mbcache sr_mod cdrom sd_mod crc_t10dif sdhci_pci sdhci mmc_core firewire_ohci firewire_core crc_itu_t xhci_hcd ahci usb_storage nouveau ttm drm_kms_helper drm i2c_algo_bit i2c_core mxm_wmi video output wmi dm_mirror dm_region_hash dm_log dm_mod [last unloaded: vmnet]

Pid: 24146, comm: vmware Not tainted 2.6.32-431.el6.x86_64 #1 Dell Inc. Precision M4600/03HYJK

RIP: 0010:[<00000001ffffffff>] [<00000001ffffffff>] 0x1ffffffff

RSP: 0018:ffff88042497bdc0 EFLAGS: 00010206

RAX: 00000001ffffffff RBX: ffff880400c4ba48 RCX: ffff88031a8f94a0

RDX: 0000000000000001 RSI: 0000000000000206 RDI: ffff880400c4ba00

RBP: ffff88042497bde8 R08: ffffffff81eb28e0 R09: 0000000000000000

R10: 0000000000000000 R11: 0000000000000000 R12: ffff880400c4ba00

R13: 00000000ffffff8d R14: ffff88031a8f9480 R15: ffff88042497bec8

FS: 00007f7f32c82700(0000) GS:ffff880028200000(0000) knlGS:0000000000000000

CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033

CR2: 00000001ffffffff CR3: 0000000401fec000 CR4: 00000000000407f0

DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000

DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400

Process vmware (pid: 24146, threadinfo ffff88042497a000, task ffff880407d6f540)

Stack:

ffffffff8144b2a1 ffff88042497bdd8 00000000000007d0 ffff880400c4ba00

<d> 00000000ffffff8d ffff88042497bea8 ffffffffa0748370 ffffffffffffffff

<d> 00000000000007c6 ffff8803fbf93c80 0000080200000003 0000000000000000

Call Trace:

[<ffffffff8144b2a1>] ? release_sock+0xb1/0xf0

[<ffffffffa0748370>] VSockVmciAllowDgram+0xcf0/0x2dec [vsock] <- vsock

[<ffffffff8109b2a0>] ? autoremove_wake_function+0x0/0x40

[<ffffffff81448867>] sys_connect+0xd7/0xf0

[<ffffffff810e2067>] ? audit_syscall_entry+0x1d7/0x200

[<ffffffff810e1e5e>] ? __audit_syscall_exit+0x25e/0x290

[<ffffffff8100b072>] system_call_fastpath+0x16/0x1b

Code: Bad RIP value.

RIP [<00000001ffffffff>] 0x1ffffffff

RSP <ffff88042497bdc0>

CR2: 00000001ffffffff

~~~

We have very limited visibility to this:

~~~

crash> mod -t

NAME TAINTS

vmnet (U)

vmmon (U)

vmci (U)

vsock (U) <- 3rd party provided module

~~~

From what I can tell in the backtrace, the vmware process called the following:

connect()

-> VSockVmciAllowDgram() <- Provided by vmware-tools package

  -> release_sock() <- This is where we failed.

This vmcore will need to be provided to VMWare support in order to begin the troubleshooting process from within the vsock module. We would be more than happy to assist further, but without the source to this module or familiarity with it, we simply are unable to assist without VMWare's assistance.

Unfortunately we cannot see into VMWare's VSockVmciAllowDgram module, so we cannot see what is happening.

Reply
0 Kudos
jperrygodfrey
Contributor
Contributor

I have found a workaround for anyone who wants to run their VM's until this bug is fixed.  
Boot the VM in Safemode and then uninstall VMware Tools and check to manually install tools on the VM settings.  Google this term to get the instructions for Safemode uninstall "Uninstall Programs Packaged with Windows Installer (MSI) in Safe Mode"

Safe mode seems to work for all machines I have tested so far, but when the VMWare tools are installed it crashes the Red Hat Workstation right at log on.

1stefan
Contributor
Contributor

I have chosen a different workaround: Use the old kernel (2.6.32-358.23.2 instead of 2.6.32-431). The Windows VM runs fine on the old kernel.

Just edit /boot/grub/menu.lst and change the default so you don't have to remember to select the old kernel after every reboot.

jperrygodfrey
Contributor
Contributor

That was the first thing i tried to use the old 2.6.32-358.23.2 but it would not boot in that one for some reason.  So there are two workarounds so far...

Reply
0 Kudos
cscarff
Contributor
Contributor

jperrygodfrey, it's good to see someone else reported not being able to boot off the previous kernel.  I too could not fully boot into 2.6.32.358.23.2 or even the one before that.  I figured it was how I had secured (STIG'd) my box.  But I guess not.  It was time to do a clean (and better) install anyways.

Chris
Reply
0 Kudos
mseppola
Contributor
Contributor

I upgraded to Centos 6.5 yesterday, and had the same issue.  Your fix worked like a charm.  Thanks for the post! Smiley Happy

Reply
0 Kudos