ESX Crash when laptop connects

[mmcgee686]

I have what appears to be a very unique issue...

We have a SUN 4200M2 and 2 SUN 4200 servers running ESX 3.5. The M2 server is crashing quite regular. after some investigation, we discovered it only crashed during business ours. Long story short, I discovered it only crashed when MY HP laptop was on the network. Even better, only when I was wired. The Wireless connection did/does not cause a crash... We verified there was not a conflict between IP addresses by hard setting an IP on a network not valid on our network. Still crashed. Verified the ports on the switches... Still crashed on multiple ports. Changed the IP of the VMWare server. Still crashed... Tried a completely new server, with the same result... Crash... Verified the MAC addresses were not the same... They weren't...

We verified all the installed applications, the HOSTS file, anything else that might reference the VMWARE server or IP. Nothing was found... Uninstalled the VI client... Still crashed... As a matter of fact, it will crash even after a reboot of my laptop before I log on. But I keep my laptop off the network, it will run for days. As soon as I plug in, it crashes within 60 seconds...

The only way we can prevent the crash, don't plug in my computer... This in theory would be a temp fix, but our concern is could/would another laptop, server, or desktop potentially cause the same issue down the road... We need to find the root cause...

I am running Vista Ultimate x64 on my laptop with Symantec Endpoint Security... Office 2007, other company applications. Nothing different from anyone else...

Running out of ideas. Any help would be greatly appreciated.

[ewannema]

This is a very interesting problem. I say interesting because I have no initial ideas as to what the problem could be. What type of crash are you seeing? Power off, hang, purple screen of death? Does your laptop have any utilities that could affect the VMFS storage?

Have you opened a support case with VMware? They should be able to take a look at the output of a vm-support command to help determine the cause of the problem.

If you want to dig into it yourself I would see what the last entries in some of the /var/log files are. The general log file location is /var/log/messages. If you want to post a copy of your log files then we can take a look at them.

[mmcgee686]

Yes. It is definately interesting... The Purple Screen of Death (PSOD) is what we get when it crashes... As for my laptop, I use the same utilities through my wireless that I use when I am wired. The issues only happen when wired... But to be safe, I uninstalled everything off my laptop that I thought even remotely would communicate with the VMWARE servers...

We have a case open, and have for nearly a month... They had no idea what the issue was. It wasn't until this week that I was able to identify my wired lan connection as the source, but not the root, cause... We have sent numerous dump files with no result.

I will try to get the log files posted today to see what you (and others) may think...

Thanks for the help!

[AWo]

Everything grounded appropriate?

Changed the NIC's of the ESX host?

AWo

[mmcgee686]

We have actually completely replaced the entire server with a duplicate model and suffered the same results... We have two other servers in the farm but different model, and do NOT have this issue with these. It is specific to the 1 model type, and specific to VMWARE. Other OS installs on the failing model don't crash. Just VMWare...

Thanks for any help...

[AWo]

I don't know if this is too far away, but a long time ago we had an IBM AS400 crashing when it receives file transfer requests (and only this ones, terminal sessions caused nothing) from a SNA gateway. It was the MTU packet size. Does it happen when you plug your laptop in with the NIC disabled in Windows beforehand?

Or.....maybe it is related to SUNeruptions

AWo

[ewannema]

The grounding suggestion is an interesting one I had not thought about that. Here are a few more questions:

1. Does it only happen when your OS is loaded. Does it happen when it is in safe mode with networking?

2. Does it happen if your laptop is plugged into a different port (or switch?)

3. What about if you give your ESX server a different name/IP?

4. Is there any monitoring software that might send something like a Wake On LAN packet? Perhaps something gets filtered by your access point.

5. The logs are probably a better indicator, but if that is not providing good enough information for you. If it only happens when your computer is up and running and network connected you might want to run a packet trace and see what the last bit of traffic your machine is sending when the host blows up.

[mmcgee686]

The grounding suggestion is an interesting one I had not thought about that. Here are a few more questions:

1. Does it only happen when your OS is loaded. Does it happen when it is in safe mode with networking?

Answer: It will happen even before I log in. Have not tried Safe Mode yet... Will try next...

1. Does it happen if your laptop is plugged into a different port (or switch?)

Answer: Yes, different ports and different switch. Even with different IP addresses that are NOT valid IPs (169.x.x.x)

1. What about if you give your ESX server a different name/IP?

Answer: Tried both different IPs and differnt host name. Same result every time..

1. Is there any monitoring software that might send something like a Wake On LAN packet? Perhaps something gets filtered by your access point.

Answer: Don't think so. We have put a sniffer box in line to the server and have been unable to locate any eroneous traffice.

1. The logs are probably a better indicator, but if that is not providing good enough information for you. If it only happens when your computer is up and running and network connected you might want to run a packet trace and see what the last bit of traffic your machine is sending when the host blows up.

Answer: See last answer. Tried it and nothing unusual is seen...

Good questions though...

[wila]

Hi,

I know you are on VI 3.5, but your issue sounds like the first item that is fixed by this patch in 3.0.2 :

http://kb.vmware.com/kb/1002424

--

Wil

[ewannema]

It would be worth checking the log entries against those at the end of the article. There is at least one other instance of a fixed bug being reintroduced in 3.5. Also, there are some patches out for 3.5 now. Nothing stuck out in a quick glance, but there were some driver updates. Might be worth checking those out at well.

[ezed]

cool problem, i would think VMware would be asking for a sniffer trace, as this sounds like a great DOS attack.