Has anyone ever seen a total disconnect on all interfaces (all vswitches, vmnics, and eths) at the same time?
I have redundant connections for each interface (dual port NC380T NIC's) to redundant swtiches. Per our networking team, no glitches, outages, or events happened at this time. However, all our VM's, SC, and iSCSI (VM software connected) saw a blip and disconnected. HP Insight Manager didn't detect any failures for the NIC's. Is this a vSwitch issue or a greater internal ESX problem I am unaware of?
System is a HP Proliant DL580 G5.
Here's the log from /var/log/messages.2
Jun 20 06:51:52 ESX-01 insmod: /lib/modules/2.4.21-47.0.1.ELvmnix/kernel/drivers/net/bnx2/bnx2.o: init_module: No such device
Jun 20 06:51:52 ESX-01 insmod: Hint: insmod errors can be caused by incorrect module parameters, including invalid IO or IRQ parameters. You ma
y find more information in syslog or the output from dmesg
Jun 20 06:51:52 ESX-01 insmod: /lib/modules/2.4.21-47.0.1.ELvmnix/kernel/drivers/net/bnx2/bnx2.o: insmod eth0 failed
Jun 20 06:51:52 ESX-01 insmod: /lib/modules/2.4.21-47.0.1.ELvmnix/kernel/drivers/net/bnx2/bnx2.o: init_module: No such device
Jun 20 06:51:52 ESX-01 insmod: Hint: insmod errors can be caused by incorrect module parameters, including invalid IO or IRQ parameters. You ma
y find more information in syslog or the output from dmesg
Jun 20 06:51:52 ESX-01 insmod: /lib/modules/2.4.21-47.0.1.ELvmnix/kernel/drivers/net/bnx2/bnx2.o: insmod eth1 failed
Jun 20 06:51:52 ESX-01 insmod: /lib/modules/2.4.21-47.0.1.ELvmnix/kernel/drivers/net/bnx2/bnx2.o: init_module: No such device
Jun 20 06:51:52 ESX-01 insmod: Hint: insmod errors can be caused by incorrect module parameters, including invalid IO or IRQ parameters. You ma
y find more information in syslog or the output from dmesg
Jun 20 06:51:52 ESX-01 insmod: /lib/modules/2.4.21-47.0.1.ELvmnix/kernel/drivers/net/bnx2/bnx2.o: insmod eth2 failed
Jun 20 06:51:52 ESX-01 insmod: /lib/modules/2.4.21-47.0.1.ELvmnix/kernel/drivers/net/bnx2/bnx2.o: init_module: No such device
Jun 20 06:51:52 ESX-01 insmod: Hint: insmod errors can be caused by incorrect module parameters, including invalid IO or IRQ parameters. You ma
y find more information in syslog or the output from dmesg
Jun 20 06:51:52 ESX-01 insmod: /lib/modules/2.4.21-47.0.1.ELvmnix/kernel/drivers/net/bnx2/bnx2.o: insmod eth3 failed
Jun 20 06:51:52 ESX-01 insmod: /lib/modules/2.4.21-47.0.1.ELvmnix/kernel/drivers/net/bnx2/bnx2.o: init_module: No such device
Jun 20 06:51:52 ESX-01 insmod: Hint: insmod errors can be caused by incorrect module parameters, including invalid IO or IRQ parameters. You ma
y find more information in syslog or the output from dmesg
Jun 20 06:51:52 ESX-01 insmod: /lib/modules/2.4.21-47.0.1.ELvmnix/kernel/drivers/net/bnx2/bnx2.o: insmod eth4 failed
Jun 20 06:51:52 ESX-01 insmod: /lib/modules/2.4.21-47.0.1.ELvmnix/kernel/drivers/net/bnx2/bnx2.o: init_module: No such device
Jun 20 06:51:52 ESX-01 insmod: Hint: insmod errors can be caused by incorrect module parameters, including invalid IO or IRQ parameters. You ma
y find more information in syslog or the output from dmesg
Jun 20 06:51:52 ESX-01 insmod: /lib/modules/2.4.21-47.0.1.ELvmnix/kernel/drivers/net/bnx2/bnx2.o: insmod eth5 failed
Jun 20 06:51:52 ESX-01 insmod: /lib/modules/2.4.21-47.0.1.ELvmnix/kernel/drivers/net/bnx2/bnx2.o: init_module: No such device
Jun 20 06:51:52 ESX-01 insmod: Hint: insmod errors can be caused by incorrect module parameters, including invalid IO or IRQ parameters. You ma
y find more information in syslog or the output from dmesg
Jun 20 06:51:52 ESX-01 insmod: /lib/modules/2.4.21-47.0.1.ELvmnix/kernel/drivers/net/bnx2/bnx2.o: insmod eth6 failed
Jun 20 06:51:52 ESX-01 insmod: /lib/modules/2.4.21-47.0.1.ELvmnix/kernel/drivers/net/bnx2/bnx2.o: init_module: No such device
Jun 20 06:51:52 ESX-01 insmod: Hint: insmod errors can be caused by incorrect module parameters, including invalid IO or IRQ parameters. You ma
y find more information in syslog or the output from dmesg
Jun 20 06:51:52 ESX-01 insmod: /lib/modules/2.4.21-47.0.1.ELvmnix/kernel/drivers/net/bnx2/bnx2.o: insmod eth7 failed
Jun 20 07:01:02 ESX-01 syslogd 1.4.1: restart.
Jun 20 07:55:47 ESX-01 sshd[27840]: Connection closed by 10.10.10.54
hi
I have the same problem and as a result backups on my hosts have failed to run. in the SC the message file is full of these error messages but in VC all the connections are present. Did you get an answer for your problem?
Sting
hi,
all teh network ports cannot go down , but as they are using a single module "bnx2" (as per your log), could you please check if this module has been unloaded .
because if this is done, then all the network cards using this module will be down.
check for this module is loaded or not from "vmkload_mod -l", if the module is not loaded then try to reload using "vmkload_mod bnx2"
-Karunakar
probably the module gone bad, or that's the way it seems. Maybe the complete nic went off and on for a sec. This is why I try to always use 2 different vendors or models in a vSwitch!
Duncan
Blogging: http://www.yellow-bricks.com
If you find this information useful, please award points for "correct" or "helpful".
/var/log/messages contain messages and warnings for the Service Console, not the VMkernel. Your network drivers aren't supposed to load in the Service Console, so the errors / warnings are probably normal. The VMkernel loads these drivers when the hypervisor is initiated.
The only time the Service Console loads the network drivers is when you select "Service Console Only (troubleshooting mode".
Hope this helps.
Paul
We found that the G5's had a BIOS issue that would cause an NMI which in turn would cause an ASR. After applying this BIOS fix/upgrade, we haven't come across this issue since June. I believe it is now fixed for good using the 09/23/08 BIOS fix.
Thanks!