Hi,
we have two RHEL6.3 VMs which need 15 minutes to boot!
The problem happens when starting the network interfaces in vmsvc.
Host: ESXi 5.1, latest 5.1 tools from repo installed, two vmxnet3 interfaces
Both interfaces are connected to two different portgroups on the same swtich with same VLAN ID (!) and one uplink.
Please help how to find out the cause of this.
Thx Rainer
2nd Edit: After renaming files in previous post it works after subsequent reboot.
Edit: Nevermind. After another reboot the problem came back.
I was able to solve this by doing the following steps. Now I don't get the message at bootup anymore, nor does eth0 nor eth1 take more than 1/2 second to become online.
Another datapoint. I installed vmtools (5.1u1) on four RHEL 6.4 VMs today. All were the same install, fully patched. The only difference was that two were single cpu (1x1core), two were SMP (2x2core (4 core total)). I only experienced the problem on the SMP VMs.
The library rename resolved the issue for the two SMP VMs.
I’m having messages in syslog like the ones described in this thread every 30 seconds:
May 9 09:28:46 vdebian vmsvc[2640]: [ warning] [guestinfo] RecordRoutingInfo: Unable to collect IPv4 routing table.
May 9 09:28:46 vdebian vmsvc[2640]: [ warning] [guestinfo] RecordRoutingInfo: Unable to collect IPv6 routing table.
May 9 09:28:46 vdebian vmsvc[2640]: [ warning] [guestinfo] Failed to get nic info.
May 9 09:29:16 vdebian vmsvc[2640]: [ warning] [guestinfo] RecordRoutingInfo: Unable to collect IPv4 routing table.
May 9 09:29:16 vdebian vmsvc[2640]: [ warning] [guestinfo] RecordRoutingInfo: Unable to collect IPv6 routing table.
May 9 09:29:16 vdebian vmsvc[2640]: [ warning] [guestinfo] Failed to get nic info.
I’m seeing this on some VM’s with up to date debian wheezy, VMwareTools-9.0.5-1065307.tar.gz, and several different ESXi 5.1U1 (build 1065491) hosts.
VM’s perform ok, they have no connectivity issues at all. Just the syslog spam every 30 seconds (which is a little bit annoying).
I also have other running debian wheezy with the same version of vmware-tools that do not show theese messages on their syslogs.
The thing all the VM’s showing theese messages have in common is network interfaces with 802.1q vlan’s (whose names are like eth0.XXX).
On two of the VM’s which didn’t show those messages (one with a 32bit system and the other with a 64bit one) I did the following test:
So this is, in my case, definitely related to vlan’s.
Anyone else seeing this? Is this a bug in vmware tools? Is there any known workaround to stop that syslog spam?
PS.: I tried removing those two libtimeSync.so libraries, no difference at all.
Bug is not fixed. :smileycry:
Problem persists with fully patched ESXi 5.1 release build 1117900
and latest OSP vmware tools 9.0.5
My RHEL6 64 VMs are fully patched, too
But here the workaround with renaming libtimeSync.so worked, vm reboots fine.
Thx
Rainer
I sorted it out setting Selinux to Permissive/Targeted.
Cenots 6.4 + kernel 2.6.32-358.6.2, esx 5.1 build 1117900, latest tools
no libtimesync rename
I this this is what is creating issues with Selinux
netlink: 12 bytes leftover after parsing attributes.(dmesg)
False positive
After some reboots issue came out again... so no Selinux
Hi Wolf,
Did you follow the workaround with renaming libtimeSync.so ? Doing the workaround works for everyone else AFAICS.
Bryan
blharris wrote:
Did you follow the workaround with renaming libtimeSync.so ? Doing the workaround works for everyone else AFAICS.
No it doesn’t (see my previous post).
Although my VM’s work fine, I have the messages metioned by OP every 30 seconds in the syslog; wich is annoying.
I opened a SR on May 13th. I’ve been exchanging some mails with support but, ATM they haven’t figured out it yet.
[NOTE, I had posted this same message from the wrong account. I deleted that message and re-posted it from the right one. Sorry about that]
Hello Experts,
We ran in to a similar issue for our Avaya OVA deployments. What we figured out was that there is a race condition between VMware tools service and the Network service during boot up. The simple fix will be to have the VMware tools service start up ahead of the Network Service.
The following bash script goes about implementing this:
-----
VM_TOOLS_CONF=/etc/init/vmware-tools.conf
TOOL_LINK_ARRAY=(\
/etc/rc2.d/S03vmware-tools \
/etc/rc3.d/S03vmware-tools \
/etc/rc5.d/S03vmware-tools \
/etc/rc0.d/K99vmware-tools \
/etc/rc1.d/K99vmware-tools \
/etc/rc6.d/K99vmware-tools \
);
function printHelp() {
echo "Usage: bootSyncFix.sh -enable|-disable|-status"
echo
echo "-enable will enable a workaround for a known VMWare issue during bootup"
echo " where the bootup stalls on network startup and a message in the following"
echo " format is displayed in a loop:"
echo "vmsvc[1234]: [warning] [guestinfo] RecordRoutingInfo: Unable to collect IPv4 routing table."
echo
echo " This change will modify the VMWare tools service to start before"
echo " the network service."
echo "-disable will disable the above workaround and will revert the VMWare"
echo " tools service to the standard VMWare init startup."
echo "-status will display the status of the system (whether the boot sync"
echo " fix is enabled or disabled). This mode will not modify the system."
exit 1
}
# Implement/enable boot sync fix
function enable() {
echo "Enabling boot sync fix..."
# Try to move file so long as it hasn't already been moved
if [ -f /etc/vmware-tools/vmware-tools.conf -a ! -f /etc/init/vmware-tools.conf ]; then
echo "vmware-tools.conf already relocated"
else
# If destination already exists; backup file before overwriting
if [ -f /etc/vmware-tools/vmware-tools.conf ]; then
mv -f /etc/vmware-tools/vmware-tools.conf /etc/vmware-tools/vmware-tools.conf.bak
fi
mv -f /etc/init/vmware-tools.conf /etc/vmware-tools/
fi
for LINK in "${TOOL_LINK_ARRAY[@]}"; do
ln -s /etc/vmware-tools/services.sh $LINK 2>&1 | grep -v "File exists"
done
echo "...done. Any configuration changes will take place on the next bootup"
}
# Revert back to original standard configuration
function disable() {
echo "Disabling boot sync fix..."
# Try to move file so long as it hasn't already been moved
if [ -f /etc/init/vmware-tools.conf -a ! -f /etc/vmware-tools/vmware-tools.conf ]; then
echo "vmware-tools.conf already relocated"
else
# If destination already exists; backup file before overwriting
if [ -f /etc/init/vmware-tools.conf ]; then
mv -f /etc/init/vmware-tools.conf /etc/init/vmware-tools.conf.bak
fi
mv -f /etc/vmware-tools/vmware-tools.conf /etc/init/
fi
for LINK in "${TOOL_LINK_ARRAY[@]}"; do
if [ -L $LINK ]; then
rm -f $LINK
elif [ -f $LINK ]; then
# If file exists and is not a symlink, then print warning but don't delete file
echo "Warning: Not deleting non-symlink $LINK"
fi
done
echo "...done. Any configuration changes will take place on the next bootup"
}
# Return code:
# 0 -- Enabled
# 1 -- Disabled
# 2 -- Partially enabled
function status() {
echo "Checking status of boot sync fix..."
# By default, assume we are partially disabled unless we detect otherwise
STATUS=2
if [ -f /etc/rc2.d/S03vmware-tools -a -f /etc/rc3.d/S03vmware-tools -a -f /etc/rc5.d/S03vmware-tools -a -f /etc/rc0.d/K99vmware-tools -a -f /etc/rc1.d/K99vmware-tools -a -f /etc/rc6.d/K99vmware-tools -a -f /etc/vmware-tools/vmware-tools.conf ]; then
# All files are there; fix is enabled
STATUS=0
elif [ ! -f /etc/rc2.d/S03vmware-tools -a ! -f /etc/rc3.d/S03vmware-tools -a ! -f /etc/rc5.d/S03vmware-tools -a ! -f /etc/rc0.d/K99vmware-tools -a ! -f /etc/rc1.d/K99vmware-tools -a ! -f /etc/rc6.d/K99vmware-tools -a ! -f /etc/vmware-tools/vmware-tools.conf ]; then
# Fix completely not implemented
STATUS=1
fi
case $STATUS in
0) echo "Boot sync fix is enabled";;
1) echo "Boot sync fix is NOT enabled";;
*) echo "Boot sync fix is partially enabled";;
esac
return $STATUS
}
### Script start ###
# Check the current user -- needs to be root level
USER_ID=`id -u`
if [ $USER_ID != "0" ] ; then
echo "This script must be run by a root-level superuser (UID 0)"
echo
exit 1
fi
ARG1=$(echo "$1" | tr A-Z a-z)
case $ARG1 in
-enable) enable;;
-disable) disable;;
-status) status;;
*) printHelp;;
esac
I am also having this problem with RHEL 6.2 It is very annoying getting calls in the middle of the night because a VM was rebooted and never came back......just to find out they only needed to wait 15 mins in between reboots.
There has to be a Fix. Renaming files in over 100 vms doesnt seem like an acceptible solution.
Please respond if anyone finds a better solution.
Thanks
I can confirm that adding NTP servers to the ESXi host with the affected VM resolved it for me. Have done several reboots of the problematic VM with no problem since configuring NTP on the host. This may not be an option for everyone so I suppose it should still be addressed inside of the VMware tools package.
I revently got this error after trying to clone a cloned copy of a CentOS 5.6 VM.
I imported CentOS OVF from scratch, updated tools and hardware and cloned from template in vCenter and no problem.
If I then tried to clone from the cloned copy I got this error. Not sure if this helps. To resolve my issue I re imported the OVF again and cloned from that. Had to do some rework in the base VM but it ended up working.
The following VMware KB has been published to document this issue. The primary cause is in the Linux iputilis package and this is causing the delay in the boot process. A workaround is available in the KB advising to set rtc.diffFromUTC=0 within the vmx file.
I managed to fix the issue by setting the rtc.diffFromUTC=0 option in the vmx file. Then I got the error http://cdn.chriscolotti.us/wp-content/uploads/2013/02/vmsvct.png and managed to fix it as described here
Workaround: vSphere 5.1 Guest Unable to collect IPv4 routing table
by renaming libtimeSync.so to libtimeSync.so.bak (or something else you choose) in
/usr/lib/vmware-tools/plugins32/vmsvc/libtimeSync.so
/usr/lib/vmware-tools/plugins64/vmsvc/libtimeSync.so
What?
You managed to fix the issue by setting rtc.diffFromUTC=0 and then you got the error - again?
I tried the .vmx mod, renaming the .so 's and even the script by smuthali (with a few changes for file locations). None of those did the trick.
I was seeing
Unable to collect IPv4 routing table
vmsvc[712]: [ warning] [guestinfo] Failed to get nic info.
spat out together every 30 seconds.
The following worked for me, and may shed some light on the root cause:
Every 30 seconds I am getting two log entries written to syslog:
vmsvc[712]: [ warning] [guestinfo] Failed to get nic info.
vmsvc[712]: [ warning] [guestinfo] Failed to get vmstats.
This makes the /var/log/syslog file grow unnecessarily and tools like
logcheck end up spamming me about the messages.
Apparently the 'nic info' message is produced if open-vm-tools is compiled
using --without-dnet and the 'vmstats' message is produced if open-vm-tools
is compiled using --without-procps.
I can turn these messages off by adding the following to the
/etc/vmware-tools/tools.conf file:
[logging]
vmsvc.level=error
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=639767
the /etc/vmware-tools/tools.conf fix worked for me.
I have a:
* CentOS 5.9+ 64-bit patched VM
* kernel: 2.6.18-348.18.1.el5
* vSphere 5.1U1
* VMware Tools: 9.0.5.21789 (build-1065307)
* VM version: 7
The VM was originally created under ESXi 3.5, cloned to 4.0, 4.1 and eventually to 4.1U3
The VM has two virtual NICs, eth0 and eth1.
eth0 was configured as VMXNET2 and dhcp
eth1 was configured as VMXNET3 and static IP
I tried renaming the libtimeSync.so file in /usr/lib/vmware-tools/plugins64/vmsvc/ to libtimeSync.so.bak as described in earlier posts.
I tried adding entries to the .vmx file as described in VMware KB: Linux virtual machine's boot is slow and displays the warning: RecordRoutingInfo: Unable ...
and VMware KB: Disabling Time Synchronization.
None of this stopped the "Unable to collect IPv4 routing table" messages I would get on startup on eth0
I would not get any error on eth1; however, it appeared not to actually work (with a ping test to another host)
I then deleted both virtual NICs and recreated them as follows:
eth0 was configured as VMXNET3 and dhcp
eth1 was configured as VMXNET3 and static IP
(i.e., as before but with both as VMXNET3)
That fixed the problem with the routing table message. Both NICs now work as expected. It may have been possible simply to reconfigure eth0 as VMXNET3 without deleting and recreating the NICs. Perhaps VMXNET2 and VMXNET3 do not work properly together under ESXi 5.1. (This "mixed NIC" configuration was working fine under ESXi 4.1U3 - and that's where I had been last using this VM.)
System is running "CentOS release 6.4" on "ESXi-5.5.0-1331820-standard"
Network traffic works fine on eth0 (dhcp) and eth0.100 (static)
Problems appears to be "vmtools fails to parse VLAN interfaces"
(disable eth0.100 and no more warnings)
#1 They're not just warnings; ESXi server (VM summary) doesn't show the eth0.100 IP address
#2 My system has no "tools.conf" file to edit; how do I "fix" this by hiding warning messages?
# ifconfig|grep eth0
eth0 Link encap:Ethernet HWaddr 00:0C:29:F6:F1:3D
eth0.100 Link encap:Ethernet HWaddr 00:0C:29:F6:F1:3D
# vmware-toolbox-cmd -v
9.4.0.25793 (build-1280544)
# find / -print | grep \\/tools\\.conf$
# find / -print | grep tools\\.conf$
/etc/modprobe.d/vmware-tools.conf
/etc/init/vmware-tools.conf
/etc/dracut.conf.d/vmware-tools.conf
# tail /var/log/messages
Oct 21 17:45:41 devrock1 vmsvc[1168]: [ warning] [guestinfo] Failed to get nic info.
Oct 21 17:46:11 devrock1 vmsvc[1168]: [ warning] [guestinfo] RecordRoutingInfo: Unable to collect IPv4 routing table.
Oct 21 17:46:11 devrock1 vmsvc[1168]: [ warning] [guestinfo] RecordRoutingInfo: Unable to collect IPv6 routing table.
Oct 21 17:46:11 devrock1 vmsvc[1168]: [ warning] [guestinfo] Failed to get nic info.
Oct 21 17:46:41 devrock1 vmsvc[1168]: [ warning] [guestinfo] RecordRoutingInfo: Unable to collect IPv4 routing table.
Oct 21 17:46:41 devrock1 vmsvc[1168]: [ warning] [guestinfo] RecordRoutingInfo: Unable to collect IPv6 routing table.
...
In my case, Network Manager was also completely disabled, however, what fixed it for me was to remove the line NM_CONTROLLED="yes" from /etc/sysconfig/network-scripts/ifcfg-eth0
Try this: vmsvc on a Linux console displays a warning similar to:
|