VMware Communities: Message List - Bridged networking just quit!

Re: Line

amyweaver29 — Mon, 02 Nov 2009 03:07:53 GMT

I have the same concern as Bruce. Will someone guide us through this?

Thanks much,

AmyWeaver

Bewegungsmelder\

Re: Line

vanRandy — Fri, 30 Oct 2009 18:31:06 GMT

welcome to my T1 Line.Are you know what is T1 Line?
T1 Line is a...................

T1 line

Re: Bridged networking just quit! - After Windows 7 upgrade on host

cosnet — Thu, 29 Oct 2009 15:42:20 GMT

Hi,

I just installed the ungrade to Windows 7 x64 ultimate from Vista x64 Ultimate. Windows 7 is the host running Vmware Server. When I tried to lauch my client I got an error that it could not connect to the server. I reinstalled server and the fixed that problem. However, now I have the same issue as reported here. I cannot get my guest running Windows Vista x32 to access the network. It apears that somehow the bridging has failed and I cannot get Windows 7 to bridge my VMware Network adapters (two show up) to the built card of the PC.

Any ideas would greatly be appreciated.

Thanks,

Bruce

Re: Bridged networking just quit!

balboa410 — Thu, 08 Jan 2009 17:15:36 GMT

Yes, as I posted before, it is definitely a big problem on the version 1.X. The new version (2.X) corrects it implementing a new way to deal with the communication between the host and the guest. I hope anyone having this problem could upgrade as soon as possible. I did it and now everything is a lot better.

Re: Bridged networking just quit!

laleger2 — Thu, 08 Jan 2009 15:05:48 GMT

Interestingly upgrading to server 2.0 seems to have solved my file transfer problems between guest and host. Keep in mind I'm still using the host-only network connection for file transfer between my guest and host, however, the symptoms before for file transfers via a bridged connection and host-only connection were identical. Not only have I not had a single issue with any file transfers for 2 weeks, the time in which the file transfers complete are consistent now. Before, a 20GB file might take 45 minutes or 2 hours. Now, that same file consistently transfers in 45 minutes.

Re: Bridged networking just quit!

laleger2 — Wed, 24 Dec 2008 15:56:12 GMT

Still fighting this problem with 1.0.5 (WS2003 x64 host, and WS2003 x64 guest). I started with Broadcom NICS, then switched to Intel NICS and locked down speed/duplex settings. This seemed to solve the problem for a few weeks, then it began occuring again consistently! Convinved it was still a network problem, I switched to doing my network transfers over a host-only network connection. I understand that host-only network connections are software based and bypass the physical NIC. To my suprise, the problem is still occuring! I always get a delayed write failed error upon each occurence. My next course of action will be to upgrade to 2.0. This is so frustrating!!!

Re: Bridged networking just quit!

devzero — Sat, 13 Dec 2008 01:41:26 GMT

I think the fault could lie in the VMware Bridge Driver. I believe this is where the MAC table is maintained.

what about vmnetUserif.sys (VMware Network Application Interface) ?

anybody seen this networking issue with vmware server 2.0 ?

for those just jumping on this thread, also see http://communities.vmware.com/thread/147560?tstart=0

Re: Bridged networking just quit!

pbmgmbh — Wed, 03 Dec 2008 16:00:59 GMT

Me again,
sorry for the german link in an english moderated forum; this one is the equivalent english one (beside it includes more information)
http://www.intel.com/support/network/sb/CS-005897.htm
looking forward for someone to proove

Re: Bridged networking just quit!

pbmgmbh — Wed, 03 Dec 2008 14:16:14 GMT

Hello everyone,
we encounter the same bridge problem (as i presume) and couldn't manage to get rid of it. But yesterday i (wildering in foreign forum) happened to get an interesting link, where Intel describes a problem with promiscous mode (shure this is the mode vmware puts its lan adapters).
http://www.intel.com/support/de/network/sb/cs-005897.htm
Maybe someone can test this on a vmserver system, as i can't at the moment (server is prox.30km away from me).
Will come back to look for response.

Re: Bridged networking just quit!

balboa410 — Mon, 24 Nov 2008 16:18:32 GMT

Ok. I know this discussion is about VMware 1.X, but as many of the readers come here looking for a final solution, and I was one really affected by this big BUG, I can tell you that I fixed it this way:

-Uninstalled VMware Server 1.X
-Downloaded and Installed VMware Sever 2.0

I am amazed how well it is working now. My Virtual Machines are running as if they were the real things, no network delays or performance degradance!!! The Web console is great, asi I can see the Process and Memory comsumed by each guest and the host. I think that if you are having issues with your actual VMware 1.X, you should really try version 2.0 instead.

Now I can copy big files from guest to guest and from host to guest and vice-versa. No more "network time outs"!!! And I could copy files as big as 40GB just testing!! Instead of the 3MBps now I usually get copies at about 7MBps. They just fixed the big bug and now it is working great.

My advice to some of you that just want to fix the whole problem, install the last version. In my case, it rocks!

Running Ubuntu 8.04 Server LTS, VMware Server 2.0 over an HP Proliant ML110 with software raid.

Re: Bridged networking just quit!

mikem2002 — Sun, 23 Nov 2008 22:34:21 GMT

We don't comment on it due to the fact that this is a version 1.0 forum. Version 1.0x works fine for the most part. The jury is still out on 2.0 in my opinion due to many factors.

For your problem, I suggest installing Ubuntu 6.06 LTS as 8.04 is not a supported version when running VMWare Server 1.0. Someone else here mentioned performance issues when using a version of Ubuntu newer than 6.06 if I remember correctly.

Re: Bridged networking just quit!

balboa410 — Sun, 23 Nov 2008 09:12:41 GMT

There is a new release...a major release (VMware Server 2.0). Why I don't see any comment about it? Do it has the same problems about the networking and bridge disconnection??? Have any of you tested it already? Maybe they fixed this issue there (I hope) as I really need it. I can't transfer any big file (even 200MB) without having a problem!!!

Well, if any of you have tried 2.0, please tell me, as I can't change the one I am using now. I am having problems and I have:

-HP Proliant ML110 G5
-Ubuntu Server 8.04 LTS
-VMware Server 1.0.8
-Windows Guests (SBS 2003 Server, WinXP)

I am experiencing the same problems that all of you. I need to try the solution about using a single processor, because I see that the problem is worst when there is much traffic or load on CPU. Remember that in a virtual enviroment all of the network resources are managed by the CPU unit, in a non-virtual environment, this load is easily managed by the network adapters' CPUs.

I have a VM that is not so loaded, and it works well when I pass the command:

<code>/usr/sbin/ethtool -K eth0 tx off</code>After doing it, I restart the VMware service on Ubuntu. Everything gets better, as I cancel the added process of checksum checking. But, I have another VM that has a lot of services and loads the CPU completely, and then the problem comes back. Please, someone post experiences with VMware Server 2.0, as the VMware team is not trying to help us on this matter. :(

Re: Bridged networking just quit!

mikem2002 — Wed, 12 Nov 2008 01:51:57 GMT

Yes, I'm replying to myself here

Even taking my own advice has only resulted in failure...

Setup two servers with Win2003 x64 host, 7 VM's each. One of the virtual interfaces disconnects at least once a week on both . Time to remove Windows and install Linux as a host. Thi is the only true "solution" that I have run into.

Also, my admin computer running Vista x64 has recently run into the same problem this past week with Workstation 6.0x after about 10 months online.

Re: Bridged networking just quit!

mikem2002 — Tue, 08 Jul 2008 14:07:14 GMT

It's been a while, just wondering if others have had good success with the trick that seemed to work for us...

Using multiple NIC's, unbind the VMWare Bridge Protocol from the 'main' NIC
for the computer/server. Leave the VMWare Bridge Protocol bound to
other interfaces where you want guests to connect to the LAN. In the
"Manage Virtual Networks" program, uncheck the Automatically Configure
option, then bind the VMnet's manually.

I've found this to work quite well. Dunno if this is solution per se, but I haven't had a single issue since using this method.

Anyone else try this...did it work? did it fail?

The hosts we've tried this with are Server 1.05 & 1.06 Win2003 x64 R2 SP1 & SP2, Vista x64 with & w/o SP1 with Workstation 6.0x. So far, not a single issue on any of these systems.

Re: Bridged networking just quit!

Creator620 — Tue, 01 Jul 2008 18:11:13 GMT

I will be out of the office returning July 2, If this is Urgent please contact Colin or Dawn for assistances

Re: Bridged networking just quit!

D_o_S — Tue, 01 Jul 2008 18:02:03 GMT

Hi, further to my previous post below of the 1/6/8.

This did NOT solve my problem - it re-occured a few days later. Following this I tried:

A new NIC with signed drivers (no joy!)
Various disabling and faffing with IPv6 (which VM Ware Server Beta 2 really doesn't seem to like)
Reinstalling the beta etc.

However, I've now had over a weeks uptime and a completely different league of performance with the NIC... my resolution - downloading and installing VM Ware workstation. Bizzarely the Beta Server images don't work natively with workstation and the NIC appears disabled and you need to manually edit the VMX file removing 2 lines referencing bridge mode to get them working (found this in another community). Bit shabby that you have to manually edit the VMX file, worse that the NIC just stops workign and you don't get a version mismatch error.

Anyhow, workstation works like a dream, has a usable interface (unlike the server beta) and works with snapshots. The only pain in the ass is the lack of support for running workstation as a service. (Never understood this limitation on the superior paid for product!?)

Anyhoo, hope this helps someone!

Old post below----------------------------------------------------------------------------------------------------------------

Hi. I'm running

Windows Vista host x64 Host
Windows 2008 x64 Guest

Intel Quad Core 6600 2.4 Ghz
6gb of RAM

NIC is NVidia nForce on board Gb NIC

Since using x64 I've had nothing but problems with the NIC's. Connectivity to the guest (even on the host) is removed after maybe 1-2 days - nothing logged in event log on either machine.

What I have noticed is that actively disabling the WAN miniport IPV6 in device manager (view -> show hidden) seems to increase the gap between outages...

Next step for me is try changing the NIC's ... feel a bit disappointed that VMware haven't cracked this - it's obviously affecting a lot of people running x64 platforms and I won't go near a production environment with x64 and VM until things are improved...

Re: Bridged networking just quit!

Creator620 — Tue, 01 Jul 2008 17:01:10 GMT

I will be out of the office returning July 2, If this is Urgent please contact Colin or Dawn for assistances

Re: Bridged networking just quit!

COS — Tue, 01 Jul 2008 16:51:11 GMT

Disconnect ALL of the CD/DVD's from EVERY Guest OS. Then reboot the whole thing.

Re: Bridged networking just quit!

Creator620 — Tue, 01 Jul 2008 15:31:13 GMT

I will be out of the office returning July 2, If this is Urgent please contact Colin or Dawn for assistances

Re: Bridged networking just quit!

Creator620 — Tue, 01 Jul 2008 15:21:14 GMT

I will be out of the office returning July 2, If this is Urgent please contact Colin or Dawn for assistances

Re: Bridged networking just quit!

Peter_vm — Tue, 01 Jul 2008 15:21:14 GMT

ronnieredd wrote:
Since there is not a reasonably priced alternative we stayed with virtual server as the recommended upgrade path for this data center. This issue has gone too long without resolve. If it's not taken care of soon, we will go somewhere else. If we move on, our workstation licences will not be renewed and I will scrap our plans for the upcoming esx dc project.

What say you vmware?

I do not work for VMware but extrapolating problems of one application (VMware Server 1.0.x) and your hardware, to other (ESX), does not seem logical.

Re: Bridged networking just quit!

ronnieredd — Tue, 01 Jul 2008 15:12:05 GMT

Jumping in on page 28.

Running vmware server on hp 585 four procs 32GB ram.

Host Ubuntu 7.10 amd64 bit.

Same problem as everyone else. When the network stops responding the host windows or ubuntu 32 bit guests freeze up and do not respond to mouse/keyboard inputs. Wait 5 minutes (approx.) and the input gets realized and networking comes back. I also have the same problem only worse on a Ubuntu guest webserver that looses it's brains on the date/time and completely freezes up requiring a hard reset.

It does not appear to be a hardware (host) issue or host OS problem. The issue is vmware server and it is a bug! The guest ubuntu webserver was created and was running fine on a windows xp 32 bit host.

Due to uptime issues, I'm currently running one ubuntu based svn guest server in virtualbox (vmware disk) on an identical host -- without this issue. I too am looking for alternatives. VirtualBox was a pain in the @ss to set up, however it's currently been running for 5 days without issue. I'm looking at KVM too.

We were vmware early adopters. GSX was a better product. We loved (of course!) when it went free. There hasn't been a completely stable version since. The windows host service sometimes won't start by itself. The linux version has issues on linux guests.

Since there is not a reasonably priced alternative we stayed with virtual server as the recommended upgrade path for this data center. This issue has gone too long without resolve. If it's not taken care of soon, we will go somewhere else. If we move on, our workstation licences will not be renewed and I will scrap our plans for the upcoming esx dc project.

What say you vmware?

Re: Bridged networking just quit!

D_o_S — Thu, 19 Jun 2008 07:29:21 GMT

I think I may have solved my problem.

Kit - Intel quad core / 4 Gb RAM / Asus P5N SLI

OS - Host = Vista x64, VM = Windows 2008 x64

Problem - weird slow connectivity, network connections (RDP / IMAP etc) are all slow, jittery and stall. RDP sessions seem to get "stuck" and freeze. Ultimately some Windows services crash / terminate unexpectedly. Worst affected is Exchange 2008 - the Hub Transport service freezes - I believe due to the network transport problems.

Solution - removing IPv6 helped reduce jitter, but problem persisted. Finally, I managed to obtain new drivers for the onboard ASUS NIC which seems to have provided 24 hours of jitter free operation. The new drivers are Microsoft produced and signed and offer more options (VLAN tagging etc). If the crashing starts again I will post - otherwise I'm hoping for the best

Re: Bridged networking just quit!

aaron_sf — Wed, 18 Jun 2008 04:43:05 GMT

I have - doesn't matter if you are running vmware server or not - from the descriptions here it looks like a general networking issue with vmware, server or fusion (mac)

Re: Bridged networking just quit!

booyah5 — Wed, 18 Jun 2008 04:18:53 GMT

I've been experiencing this problem with the latest VMware server 1.0.6 build-91891 on WinXP Pro SP2 host with Debian 40r3 and WinXP Pro SP2 as guests. I didn't have this problem before I upgraded to that version. I also have this same install on a Vista Home Premium SP1 host without any problems.

In my case the network just cuts out - "no route to host" when I ping the host system from the guest though ifconfig and ipconfig show the IP I should have. There's no error messages in the XP guest event viewer that are relevant, and rebooting the guests, or shutting down and switching to host only or Nat networking doesn't offer any solution. Only rebooting the host brings networking back up.

When I get around to it, I think I'll downgrade to the last version and see if this helps.

Re: Bridged networking just quit!

Peter_vm — Tue, 17 Jun 2008 16:55:54 GMT

aaron_sf wrote:
I don't see how to do this on Fusion...?

What "Fusion"? This a "Vmware Server" forum.
Post your comments regarding VMware Fusion in their forum.

Re: Bridged networking just quit!

John Brook — Tue, 17 Jun 2008 16:24:34 GMT

Hiya

I didn't say you had to be running VMWare server - I was asking which virtualisation product you were running.

It seems you are running Fusion, which i've never heard of, maybe this is the equivalent of VMWare Player for the Mac. Anyway I'm not familiar with it so sorry I can't help.

Cheers, John

Re: Bridged networking just quit!

aaron_sf — Tue, 17 Jun 2008 16:18:53 GMT

I don't see how to do this on Fusion...?

Re: Bridged networking just quit!

aaron_sf — Tue, 17 Jun 2008 16:17:42 GMT

why do I have to be running a server?

I have a Mac Pro 8-core, 24 GB ram, Fusion, and a collection of about 30 VMs ... I just want to be able to run as many of them locally as my resources allow

Re: Bridged networking just quit!

aaron_sf — Tue, 17 Jun 2008 16:16:10 GMT

where is this setting? I don't see it anywhere... i'm running Fusion

Re: Bridged networking just quit!

aaron_sf — Tue, 17 Jun 2008 16:15:13 GMT

everything else works fine - but when i go to the browser, it cant load 'www.google.com' for example...

Re: Bridged networking just quit!

Peter_vm — Tue, 17 Jun 2008 10:28:05 GMT

Maybe you should disable Automatic Bridging in VMware Virtual Network Settings...

Re: Bridged networking just quit!

John Brook — Tue, 17 Jun 2008 08:21:12 GMT

Hello there...

OK it looks like the problem was that for some reason the last time the host (and guests) were re-booted, VMnet0 attached itself to the NIC for our private backup network, rather than the NIC for the main network.

Creating VMnet4 fixed the problem, since I had to specify which NIC to use, whereas VMNet0 defaults to allowing VMWare to choose which NIC to use.

Temporarily disabling the backup NIC on the host and re-booting fixed it - VMnet0 is connected to the right NIC again (it had no choice). Then re-enabled backup NIC on host.

No idea why the backup NIC was chosen on that re-boot - it had always used the correct NIC before.

Cheers, John

Re: Bridged networking just quit!

John Brook — Tue, 17 Jun 2008 08:16:31 GMT

... oh, and I should have said, if you aren't running VMWare server, what are you running?

Re: Bridged networking just quit!

John Brook — Tue, 17 Jun 2008 08:14:50 GMT

Hiya

When you say they 'don't work', are they behaving normally other than that they can't access the network? EG are programs responsive etc? Or are there other symptoms?

Cheers, John

Re: Bridged networking just quit!

aaron_sf — Mon, 16 Jun 2008 21:50:24 GMT

I'm not using VMWare server at all and i've been seeing this problem for many months. I'm using an 8core Mac Pro, 24Gb ram, Intel PRO NIC .. and I have about 30 different VMs, of which I like to have about 3-6 running at any one time. I have them all set up in bridged mode. I can usually get 3 or 4 running simultaneously... but often after that, vm's that I fire up will just not work (browsers cant display web page etc) ..

Occasionally I am able to get a new vm running, maybe by closing old ones, rebooting, repairing net connections.. I really don't know if what I do helps or its just luck that I get it working... once I even uninstalled the NIC adapter in the vm.. rebooted.. and it worked! Usually though I give up after messing with it for half an hour...

Sometimes, i'll come back to it later without touching anything and lo! it's working..!

I have no idea wha is going on.

it's a real problem...

Re: Bridged networking just quit!

John Brook — Thu, 12 Jun 2008 08:43:37 GMT

Hello there...

certainly looks like a vmware problem to me.

No, I don't have any Linux servers - I'm afraid i know nothing at all about Linux so i won't be able to create any , either.

I will have a look to see if there is a way to re-create vmnet0 though.

I'll get back here later.

Cheers, John

Re: Bridged networking just quit!

devzero — Wed, 11 Jun 2008 19:22:07 GMT

interesting. i will mind that when i come across that problem next time.

that also supports my theory that this is more likely a vmware problem than a problem with the host/nic/drivers.

do you have linux VMs on that system ?

if not, could you setup a linux vm and attach that to vmnet0 ?
what happens if you set the linux VM`s virtual nic into promiscuous mode? (run tcpdump -i eth0).
does that cure the VM´s networking issue ?
(please take a look at my previous posts - i`d be interested if your problem shows the same symphtoms)

Re: Bridged networking just quit!

John Brook — Wed, 11 Jun 2008 08:06:52 GMT

Hiya again

Yes, I switched 1 VM back to VMNet0 and instantly I couldn't ping in or out.

Switched back to VMnet4 and instantly ping worked again in both directions.

Is there a way to remove VMNet0 and re-create it? I guess if the problem re-occurs I'll move everything onto VMNet5, but that will only leave me 6 and 7.....

Cheers, John

Re: Bridged networking just quit!

John Brook — Wed, 11 Jun 2008 07:58:55 GMT

Hiya

Good idea, I'll give it a go in a quiet moment

Cheers, John

Re: Bridged networking just quit!

devzero — Tue, 10 Jun 2008 18:04:12 GMT

Finally, I just created a new bridged switch on VMNet4, then I changed each VM guest from bridged to custom and chose
VMNet4. Bingo - without a restart of host or guest, they all worked.

if you switch the VMs back to vmnet0 - does the problem re-appear ?

Re: Bridged networking just quit!

John Brook — Tue, 10 Jun 2008 14:22:47 GMT

Hello there...

HP DL380G5, 16 gig, dual HP gig NICs (1 NIC for private backup network) running VMWare server 1.0.4.

4 VM guests all running fine for months. This morning, bang, none of the VMs were contactable - couldn't ping them.

Tried various things in this thread, no joy.

Finally, I just created a new bridged switch on VMNet4, then I changed each VM guest from bridged to custom and chose VMNet4. Bingo - without a restart of host or guest, they all worked.

This doesn't fix the underlying problem whatever it is, but it does have my live servers running again.

Worth trying if you get the dead bridge problem, as it is so quick & easy to do.

Cheers, John

Re: Bridged networking just quit!

D_o_S — Sun, 01 Jun 2008 09:55:15 GMT

Hi. I'm running

Windows Vista host x64 Host
Windows 2008 x64 Guest

Intel Quad Core 6600 2.4 Ghz
6gb of RAM

NIC is NVidia nForce on board Gb NIC

Since using x64 I've had nothing but problems with the NIC's. Connectivity to the guest (even on the host) is removed after maybe 1-2 days - nothing logged in event log on either machine.

What I have noticed is that actively disabling the WAN miniport IPV6 in device manager (view -> show hidden) seems to increase the gap between outages...

Re: Bridged networking just quit!

Creator620 — Wed, 21 May 2008 18:17:09 GMT

I will be out of the office returning May 22, If this is Urgent please contact Colin or Dawn for assistances

Re: Bridged networking just quit!

Creator620 — Wed, 21 May 2008 18:17:07 GMT

I will be out of the office returning May 22, If this is Urgent please contact Colin or Dawn for assistances

Re: Bridged networking just quit!

andybei — Wed, 21 May 2008 18:14:03 GMT

klbrewster.... Mine has the issue with Intel nics... Sorry to say.

Andy

Re: Bridged networking just quit!

klbrewster — Wed, 21 May 2008 18:11:37 GMT

+Greetings everyone, it's been more than two weeks since switching out
my Broadcom NetXtreme II NIC's with Intel PRO 1000/PT NIC's and my
server backups have been working flawlessly!+

Just a quick note. I saw this a few weeks ago and figured for cheap money, I'd give it a try. Swapped the the Broadcom for Intels. Since then, knock on wood, everything has been fine.

Re: Bridged networking just quit!

devzero — Tue, 20 May 2008 19:52:02 GMT

If someone can enlighten the rest of us on how to "Set Virtual Ethernet Adapters in Promiscuous Mode" for
VMWare Server 1.04 it would be greatly appreciated by all.

read the "NEWS: UPDATE" section inside my posting before my last posting.
i don`t know if this works for windows guests, i tried one sniffer in windows, but it didn`t work - but i wasn`t sure if it set the interface into promiscuous mode. on linux tcpdump or ifconfig are your friend.....

Re: Bridged networking just quit!

COS — Tue, 20 May 2008 19:03:10 GMT

That's awesome news and looks like it works great in ESX however, this topic is for the Free Windows version of VMWare Server. If someone can enlighten the rest of us on how to "Set Virtual Ethernet Adapters in Promiscuous Mode" for VMWare Server 1.04 it would be greatly appreciated by all. I had no idea that this affected ESX as well.

Thanks

fyi, I found a good write up on how to do this at the guest level but then goes into ESX stuff here...
http://communities.vmware.com/message/371562

Re: Bridged networking just quit!

devzero — Tue, 20 May 2008 15:57:57 GMT

so - if i hear that there are people on this thread with such problem and with a service contract - and vmware couldn`t help - what about closing this thread because of vendor inactivity ?

let`s close this and open a new one.....

wouldn`t that raise chances to get some help ?

if i was a support engineer, i wouldn`t dare touching this thread - there are too much postings from too many people with too many different problems and too many "solutions".

Re: Bridged networking just quit!

Peter_vm — Fri, 16 May 2008 04:08:39 GMT

Maybe. But VMware is not going to admit it or fix it.

Re: Bridged networking just quit!

devzero — Thu, 15 May 2008 09:37:48 GMT

NEWS: PROMISCUOUS MODE ON VIRTUAL NIC CURES THE NETWORKING PROBLEM !

hi !

i repost my last report since i have new information.

the customer was hit by that problem again, and i found some more interesting details by chance.

I didn`t see the network issue for a long time, but a customer had it today and i had a chance to analyze.

one VM he was using for admin purpose (cisco tools) lost it`s BRIDGED network connection.

i could not make it work again, reboot of VM didn`t help.

i switched vm from bridged network to host-only-network and set a different IP inside VM, but that didn`t help either.
i also changed the VMs nic from AMD PCNet to e1000 , but that also didn`t help.

here are some more details:

ping
from host (192.168.109.1)
to VM (192.168.109.10)
connected via vmnet1

ping request times out

C:\Programme\VMware\VMware Server>vnetsniffer /e vmnet1
len 74 src 00:50:56:c0:00:01 dst 00:0c:29:07:db:b4 IP src 192.168.109.1 dst 192.168.109.10 ICMP ping request
len 74 src 00:50:56:c0:00:01 dst 00:0c:29:07:db:b4 IP src 192.168.109.1 dst 192.168.109.10 ICMP ping request
len 74 src 00:50:56:c0:00:01 dst 00:0c:29:07:db:b4 IP src 192.168.109.1 dst 192.168.109.10 ICMP ping request

as we can see, the packets from the host appear on vmnet1 - but no response from VM.

now - vice versa

ping
from VM (192.168.109.10)
to host (192.168.109.1)
but ping in VM tells that "destination host unreachable"

let`s take a look:

C:\Programme\VMware\VMware Server>vnetsniffer /e vmnet1
len 42 src 00:0c:29:07:db:b4 dst ff:ff:ff:ff:ff:ff ARP sender 00:0c:29:07:db:b4 192.168.109.10 target 00:00:00:00:00:00 192.168.109.1 ARP request
len 42 src 00:50:56:c0:00:01 dst 00:0c:29:07:db:b4 ARP sender 00:50:56:c0:00:01 192.168.109.1 target 00:0c:29:07:db:b4 192.168.109.10 ARP reply
len 42 src 00:0c:29:07:db:b4 dst ff:ff:ff:ff:ff:ff ARP sender 00:0c:29:07:db:b4 192.168.109.10 target 00:00:00:00:00:00 192.168.109.1 ARP request
len 42 src 00:50:56:c0:00:01 dst 00:0c:29:07:db:b4 ARP sender 00:50:56:c0:00:01 192.168.109.1 target 00:0c:29:07:db:b4 192.168.109.10 ARP reply
len 42 src 00:0c:29:07:db:b4 dst ff:ff:ff:ff:ff:ff ARP sender 00:0c:29:07:db:b4 192.168.109.10 target 00:00:00:00:00:00 192.168.109.1 ARP request
len 42 src 00:50:56:c0:00:01 dst 00:0c:29:07:db:b4 ARP sender 00:50:56:c0:00:01 192.168.109.1 target 00:0c:29:07:db:b4 192.168.109.10 ARP reply
len 42 src 00:0c:29:07:db:b4 dst ff:ff:ff:ff:ff:ff ARP sender 00:0c:29:07:db:b4 192.168.109.10 target 00:00:00:00:00:00 192.168.109.1 ARP request
len 42 src 00:50:56:c0:00:01 dst 00:0c:29:07:db:b4 ARP sender 00:50:56:c0:00:01 192.168.109.1 target 00:0c:29:07:db:b4 192.168.109.10 ARP reply

as we can see, VM sends ARP to vmnet1, packets pass vmnet1, reaching vmnet1 virtual host interface and host is giving arp reply.

i can see, that host has learned correct MAC/IP of VM (arp -a) , but it seems that VM never receives those arp replies.

VM doesn`t receive ANY packet, as the interface statistics tell.

that may explain, why VM is sending arp request again and again.

now comes the best:

>If i assign the VM`s network identity (ethernet0.generatedAddress) to a different VM on same host (e.g. linux vm instead of windows), the problem remains and the VM inherits the network issue - the >linux vm has now the same symptom as the windows VM.

>BUT - if i change the VMs mac adress to a different one, the problem goes away.

>If i revert the mac, the problem re-appears.

>it seems, that the vmware virtual networking/switch has got "stuck" with that specific mac adress and doesn`t forward any packets into a VM anymore.

>so we have:

>packet from guest-os -> vmnic -> vmnet1 -> vmnet1-host-nic -> host-os --->OK!
>packet from host-os -> vmnet1-host-nic -> vmnet1 --|here must be a problem| -> vmnic ->guest-os --->NotOK!!!

>this really looks like an issue with vmware virtual networking, i.e. the virtual hub/switch implementation.

NEWS:
UPDATE

since i could relocate the problem from a windows vm to a linux vm (by assiging the same mac adress to that vm) and reproduce it there immediately, now the crazy observation:

if i put the virtual ethernet interface inside the guest into promiscuous mode ("tcpdump -i eth0" or "ifconfig eth0 promisc up"), the problem is gone immediately !

vmware, will you start help finding the root cause of this this serious problem, please ?

here some more:

ping from host to vm (which doesn`t work:)

C:\Programme\VMware\VMware Server>vnetstats /lines:1 /interval:1000 vmnet1
Ports Rcv Xmt BrRcv BrXmt Err Dr NoP NoB Err Dr NoP NoB
4 3 22278 26104 0 0 0 0 0 0 0 0 0 0
Ports Rcv Xmt BrRcv BrXmt Err Dr NoP NoB Err Dr NoP NoB
4 3 22278 26104 0 0 0 0 0 0 0 0 0 0
Ports Rcv Xmt BrRcv BrXmt Err Dr NoP NoB Err Dr NoP NoB
4 3 22278 26104 0 0 0 0 0 0 0 0 0 0
Ports Rcv Xmt BrRcv BrXmt Err Dr NoP NoB Err Dr NoP NoB
4 3 22278 26104 0 0 0 0 0 0 0 0 0 0
Ports Rcv Xmt BrRcv BrXmt Err Dr NoP NoB Err Dr NoP NoB
4 3 22279 26104 0 0 0 0 0 0 0 0 0 0
Ports Rcv Xmt BrRcv BrXmt Err Dr NoP NoB Err Dr NoP NoB
4 3 22279 26104 0 0 0 0 0 0 0 0 0 0
Ports Rcv Xmt BrRcv BrXmt Err Dr NoP NoB Err Dr NoP NoB
4 3 22279 26104 0 0 0 0 0 0 0 0 0 0
Ports Rcv Xmt BrRcv BrXmt Err Dr NoP NoB Err Dr NoP NoB
4 3 22279 26104 0 0 0 0 0 0 0 0 0 0
Ports Rcv Xmt BrRcv BrXmt Err Dr NoP NoB Err Dr NoP NoB
4 3 22279 26104 0 0 0 0 0 0 0 0 0 0

as we can see vmnet1 is receiving packet, but doesn`t transmit any.

now i put eth0 in VM into promiscuous mode:

C:\Programme\VMware\VMware Server>vnetstats /lines:1 /interval:1000 vmnet1
Ports Rcv Xmt BrRcv BrXmt Err Dr NoP NoB Err Dr NoP NoB
4 3 22302 26104 0 0 0 0 0 0 0 0 0 0
Ports Rcv Xmt BrRcv BrXmt Err Dr NoP NoB Err Dr NoP NoB
4 3 22303 26104 0 0 0 0 0 0 0 0 0 0
Ports Rcv Xmt BrRcv BrXmt Err Dr NoP NoB Err Dr NoP NoB
4 3 22303 26104 0 0 0 0 0 0 0 0 0 0
Ports Rcv Xmt BrRcv BrXmt Err Dr NoP NoB Err Dr NoP NoB
4 3 22303 26104 0 0 0 0 0 0 0 0 0 0
Ports Rcv Xmt BrRcv BrXmt Err Dr NoP NoB Err Dr NoP NoB
4 3 22303 26104 0 0 0 0 0 0 0 0 0 0
Ports Rcv Xmt BrRcv BrXmt Err Dr NoP NoB Err Dr NoP NoB
4 3 22303 26104 0 0 0 0 0 0 0 0 0 0
Ports Rcv Xmt BrRcv BrXmt Err Dr NoP NoB Err Dr NoP NoB
4 3 22303 26104 0 0 0 0 0 0 0 0 0 0
Ports Rcv Xmt BrRcv BrXmt Err Dr NoP NoB Err Dr NoP NoB
4 3 22305 26106 0 0 0 0 0 0 0 0 0 0
Ports Rcv Xmt BrRcv BrXmt Err Dr NoP NoB Err Dr NoP NoB
4 3 22307 26108 0 0 0 0 0 0 0 0 0 0
Ports Rcv Xmt BrRcv BrXmt Err Dr NoP NoB Err Dr NoP NoB
4 3 22309 26110 0 0 0 0 0 0 0 0 0 0
Ports Rcv Xmt BrRcv BrXmt Err Dr NoP NoB Err Dr NoP NoB
4 3 22311 26112 0 0 0 0 0 0 0 0 0 0
Ports Rcv Xmt BrRcv BrXmt Err Dr NoP NoB Err Dr NoP NoB
4 3 22313 26114 0 0 0 0 0 0 0 0 0 0
Ports Rcv Xmt BrRcv BrXmt Err Dr NoP NoB Err Dr NoP NoB
4 3 22317 26118 0 0 0 0 0 0 0 0 0 0
Ports Rcv Xmt BrRcv BrXmt Err Dr NoP NoB Err Dr NoP NoB
4 3 22319 26120 0 0 0 0 0 0 0 0 0 0

as we can see, packets being received and transmitted - and all is well.

So it seems the virtual switch implementation has a problem sending packets
with specific mac adress to specific virtual switch ports.

how can this NOT be a vmware bug ?

Re: Bridged networking just quit!

devzero — Mon, 12 May 2008 22:47:28 GMT

We purchased a support contract with vmware and they have not been able to help me with this yet either.
After being bounced from technician to technician I finally got a call from the support manager saying that they
would like me to run the vm-support.vbs script while the VM is experiencing the drop in network connectivity

can you perhaps go back some postings and read into my findings? (posting from 21.04.2008 17:51)

maybe it`s worth checking if your VMs show the same symphtoms when they have that network issue....

Re: Bridged networking just quit!

jsnavely — Mon, 12 May 2008 21:46:25 GMT

I know what you mean. I actually have about 15K in the budget this quarter that is earmarked for vmware. I'm not happy about it, but they'll probably get it. There's just noone else out there with similar HA capabilities.

I liked the company much better before I ran 30+ vmware server upgrades (~5 upgrades across multiple servers), having to recreate my rather complex network configuration each and every time. It's actually the simple things, like poor installers, that have bugged me more than the big ones, like the bridged networking.

Re: Bridged networking just quit!

COS — Mon, 12 May 2008 21:37:15 GMT

Kind of like the "Drug Dealer" effect. Give out free free stuff and if you want better stuff pay for it. The MS solution just was not robust enoug. Believe me, we looked at it heavily. The new 2008 MS Virt Srv sems to be pretty good now. However, 5 months ago it was not available. ESX was not too expensice, $995 for an ES lic and more $$$ for premium support for 24x7.

Re: Bridged networking just quit!

jsnavely — Mon, 12 May 2008 21:28:40 GMT

LOL... you really showed them -- I guess that's their strategy. Leave bugs in free product so people will be forced to pay.

Yea, they're quick to say they can't repeat the issue, but that means they just don't give a ****.

1) Buy server with broadcom NICs
2) Load Win 2003 R2 x64 on host
3) Load Win 2003 R2 x64 on guest with two processors
4) generate heavy network load on guest - repeat until failure

It's not difficult to recreate. Oh, and if you want to "fix it", either switch the guest to a single processor, or buy some Intel NICs for your server. Done

Re: Bridged networking just quit!

COS — Mon, 12 May 2008 21:22:03 GMT

5 Months.

Re: Bridged networking just quit!

MichaelHu — Mon, 12 May 2008 21:20:37 GMT

Good to know, thank you. We have been working with VMsupport since the beginning of February this year trying to correct this problem. I am not optimistic that vmware support is going to be able to fix this anytime soon and so we are considering changing virtualization platforms.

How long have you been on the ESX platform now without any drops?

Re: Bridged networking just quit!

COS — Mon, 12 May 2008 21:13:17 GMT

This issue pissed us off to no end so we opted to use ESX 3.5 and have not seen this issue as of yet. KNOCK ON WOOD!
For kicks I did a VMWare COnvert of a guest exhibiting this bug to an ESX host and we did not see any problems. I loaded it down with 8 simultaneous file copies each at 2GB and SQL 2005 backups through Veritas and it did not fault.

Re: Bridged networking just quit!

MichaelHu — Mon, 12 May 2008 21:00:28 GMT

I attempted changing from the e1000 to the vmxnet driver for my VMs. The problem I had with this was that the Windows 2003 x64 guests were not able to recognize the device and install drivers for it. I tried reinstalling the vmware tools in hopes that it would install the right driver for the vmxnet interfaces, but this did not work. I also mounted the vmware tools iso and copied the vmxnet drivers over to the windows server and tried to manually install them for the vmxnet devices. This did not work for me.

I am at a loss at where to go next with this issue and have tried all of the relevant suggestions in this thread. We purchased a support contract with vmware and they have not been able to help me with this yet either. After being bounced from technician to technician I finally got a call from the support manager saying that they would like me to run the vm-support.vbs script while the VM is experiencing the drop in network connectivity. Has anyone else with a support contract been asked to provide the same information? and if so what was vmware's response/finding? They say that they cannot recreate the issue in their test environment.

Another question: Has anyone that has been experiencing this problem tried moving their VMs over to an ESX server? Or to another virtualization platform? If so what platform have you moved to?

Re: Bridged networking just quit!

olivelawn — Tue, 06 May 2008 06:58:31 GMT

disabling the cisco vpn service fixed the bridge networking issue for me. I was banging my head against my desk. thanks!

Re: Bridged networking just quit!

jblachowski — Mon, 05 May 2008 19:50:18 GMT

What driver is everyone using for their VM's. I've been using the e1000 driver. I've been working with support on this issue and now have become their test subject since they can't reproduce in their lab. Is anyone running the vmxnet driver or has anyone tried?

Basically these are the changes VMware wants me to make:

ethernet0.allow64bitvmxnet = "TRUE"
ethernet0.virtualDev = "vmxnet"

Thoughts?

Re: Bridged networking just quit!

jkordish — Mon, 05 May 2008 19:40:00 GMT

Hello all,

I was having the exact same issues as everyone describing here. I did everything I could possibly imagine to try and fix it. From upgrading my broadcom netXtreme driver to making registry changes.

Here is what I came up with that solved the issue for me. I apologize if this has already been addressed but I couldn't reader 25 pages of this thread.

In my situation it was installed on my work laptop which has the cisco vpn client installed on it. Apparently that is a no no. I went to my services on my host machine and stopped the Cisco Systems, Inc. VPN Service service. Bridge mode just started to work.

Hope this helps someone.

-cheers

Re: Bridged networking just quit!

rkrewinkel — Mon, 05 May 2008 12:10:39 GMT

hi,

I've just stumbled into this discussion, since we are having the exact same problem (still), but only on a linux host and guest system.
see my details: http://communities.vmware.com/thread/141700

Re: Bridged networking just quit!

laleger2 — Sat, 03 May 2008 17:32:30 GMT

Greetings everyone, it's been more than two weeks since switching out my Broadcom NetXtreme II NIC's with Intel PRO 1000/PT NIC's and my server backups have been working flawlessly! Before switching NIC's, server backups between my Guest OS (W2K3 x64) and Host OS (W2K3 x64) would fail almost every 2-3 days with Delayed Write Failed errors on the Guest OS. The backup file size might not be as large as others (4-5GB) but the failures were consistent even at that file size. I would be more than glad to provide more information/compare configuration with others who have attempted NIC swaps with no success.

Re: Bridged networking just quit!

the_sod — Fri, 25 Apr 2008 14:50:10 GMT

I installed several waves of Microsoft updates to our server last night (SBS 2003). All four of the VMs stopped responding on the network, but could talk to each other. They all used bridged networking and static IPs, running Ubuntu 7.10. A quick Google landed me here. After reading this thread and trying various solutions I found that nothing seemed to work.

Then I read devzero's post about the ARP issue (http://communities.vmware.com/message/920916#920916). Running "vnetsniffer /e vmnet0" showed me the VMs sending out ARP requests but never getting anything back. So I was able to confirm his findings with my system. But what to do about it? Cycling the bridge device didn't help, nor did changing the MAC addresses of the VMs.

Just for a lark, I shut down all the VMs and (at random,) added a second bridged NIC to one of them. After firing it up - everything worked. I powered it off, removed the NIC, and tried again - still working. I then fired up all the VMs, and they are all working. Since I've been under pressure to get these back up I haven't tried a full reboot of the host server.

I hope this helps some other people out. I guess something in the VMNet arp table/cache got flushed or reset when the new NIC was installed - whatever it was I'm glad for a sudden flash of "whatthehelli'lltryit-ness"

Re: Bridged networking just quit!

devzero — Wed, 23 Apr 2008 19:02:20 GMT

quoting my own thread:

it seems, that the vmware virtual networking/switch has got "stuck" with that specific mac adress and doesn`t forward any packets into a VM anymore.

so we have:

packet from guest-os -> vmnic -> vmnet1 -> vmnet1-host-nic -> host-os --->OK!
packet from host-os -> vmnet1-host-nic -> vmnet1 --|here must be a problem| -> vmnic ->guest-os --->NotOK!!!

this really looks like an issue with vmware virtual networking, i.e. the virtual hub/switch implementation.

can someone with networking issues check for this behaviour and confirm the same behaviour ?

Re: Bridged networking just quit!

devzero — Tue, 22 Apr 2008 19:09:22 GMT

Pardon my ignorance but what is KVM?

http://kvm.qumranet.com/kvmwiki
http://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine

I know it as Keyboard, Video, Mouse.

yes, same abbreviation

I think this thread has enough info for VMWare to fix this problem and that everyone involved should get a T-Shirt or something for all their input.

it`s more a matter of "when will they start listening and make some qualified person to join this thread to get into the issue"

there is silence for months now from vmware , and that`s the main problem.
they don`t have interest in fixing this one.

fixing it is a differnt story, because i think this thread may not be about a single problem.
maybe different issues/bugs meet here.... (related to host os, related to user error.....or perhaps related to a vmware bug...)

Re: Bridged networking just quit!

COS — Tue, 22 Apr 2008 18:59:20 GMT

Pardon my ignorance but what is KVM? I know it as Keyboard, Video, Mouse.
I think this thread has enough info for VMWare to fix this problem and that everyone involved should get a T-Shirt or something for all their input.

Re: Bridged networking just quit!

jmandawg — Tue, 22 Apr 2008 18:52:49 GMT

jsnavely wrote:
1) this thread isn't even about your problem
2) You're trying to run an OS vmware only has experimental support for
3) It wasn't even a vmware problem in the end

Why are you in here complaining again?

1) Isn't this thread about vmware networking bridges dying? Did i post in this in the wrong thread? My network bridge was dying, then i switched to KVM.

2) VMWare doesn't support Linux? Are you joking? Why can i download the linux vmware tarballs off their website then?

3) No S-H-I-T dumbass, neither are most of the solutions in this thread. I posted my solution in hopes that it might help someone else.

Was i really complaining? I'm so glad i don't have to deal with VMWare support and dumbasses like yourself anymore.

Why do you post here if you don't have anything of value to add?

Re: Bridged networking just quit!

jsnavely — Tue, 22 Apr 2008 13:49:10 GMT

1) this thread isn't even about your problem
2) You're trying to run an OS vmware only has experimental support for
3) It wasn't even a vmware problem in the end

Why are you in here complaining again?

Re: Bridged networking just quit!

jmandawg — Tue, 22 Apr 2008 12:46:03 GMT

I was having this problem on an Ubuntu x64 host. I got fed up with no support, so i switched everything to KVM. I started experiencing the same issue with KVM. After alot of research and pulling out hair, i figured out it was the sky2 driver installed by default in ubuntu. I updated the network driver with the one on Marvell's official website and no problems since. Network is slow, but it doesn't quit. Not positive if this will solve the vmware problem since i'm on kvm now, but it's worth a shot.

Also after seeing the amount of support we received in this thread, i think i'm going to stick with KVM.

Re: Bridged networking just quit!

laleger2 — Mon, 21 Apr 2008 22:18:17 GMT

Glad to see some activity on this thread again although it's a shame that VMware remains silent on the matter... I recently put new Intel PRO 1000/PT Dual Port adapter in one of my servers that was experiencing this problem and it's been running stable for the past 48 hours or so, that isn't enough time to get excited though. The 1 processor solution seems to be promising but I can't afford to lose processing power on my guest OS's.

Re: Bridged networking just quit!

jsnavely — Mon, 21 Apr 2008 20:44:36 GMT

I thought I'd posted something to this effect, but going back, I can't find it.

I switched my problem server to one processor back in February and it stopped failing. Ran a month or so with no issues. I switched it back to two processors and it failed twice that week. I've since moved it to a physical machine.

I'd gotten a little motivated to start looking at this again, but when the 1.05 "upgrade" trashed things even more than normal I got fed-up and dropped it again.

Re: Bridged networking just quit!

cbak999 — Mon, 21 Apr 2008 20:00:07 GMT

This is what I did. My guest server network card was failing EVERY night during the backup. Since going to 1 proc, has not failed yet - it's been over 55 days! I think that's the proof in the pudding right there! However, the real test would be to go back to 2 procs and retest. Haven't done that since I don't want my Exchange server going down again!

Re: Bridged networking just quit!

mike.laspina — Mon, 21 Apr 2008 19:05:05 GMT

That sounds like an interesting possiblility.

Remove the MS bindings from any secondary interface and use it.

This would rule out layer 3 interaction from the MS host instance and stop any false DOS prevention activity etc. from the MS host side.

The issue appears to be centered around layer 2 MAC/arp so I am not sure that's the answer but it's worth a try.

Re: Bridged networking just quit!

devzero — Mon, 21 Apr 2008 18:26:15 GMT

hi mike,

thanks for the hint.

i`m think this won`t help, because the bridge driver is only needed for bridged connection.
but the problem remains if i switch from bridged to host-only connection, and host-only interface has no binding to vmware bridging protocol, so i assume it isn`t related to vmware bridge driver.
if the problem re-appears, i will mind your hint and try disable/re-enable - but i don`t give much hope that this resolves the issue.

thanks again

roland

Re: Bridged networking just quit!

mikem2002a — Mon, 21 Apr 2008 18:33:07 GMT

Question: has any turned off VM Bridge Protocol from the server's primary interface and still had this problem?

While this may be re-hashing old suggestions, our workaround (other than install Linux like we did) was to turn off the VMBridge Protocol from the Windows server's interface, then turn off any MS file sharing & MS Client bindings on the remaining interfaces and leave the VMBridge Protocol on those same remaining interfaces.

This means, of course, that you'll have to change your default VM bridge, but it seems to work nicely w/o any problems in our environment which is mostly Dell boxes.

Re: Bridged networking just quit!

rage1605 — Mon, 21 Apr 2008 17:43:23 GMT

OK, so we took the following steps and the server has not had delayed write errors nor the bridged networking failed:

Recap:

We are using Dell 1950s & 2950s with dual onboard Broadcomm Nic's.

64x 2003 SP2 R2 with all patches as the host

64x 2003 SP2 R2 with all the patches as the guests

I have edited both guest servers with the following settings:

Changed all of them from two processors to one

Removed the CD-ROM device

This seems to have solved the problem (knock on wood) I have been running our coprorate servers (SQL) with these settings and they have not dropped off yet (it used to happen once every 3-4 days),

Re: Bridged networking just quit!

mike.laspina — Mon, 21 Apr 2008 16:05:19 GMT

Hi devzero,

Great fault trap by the way.

I think the fault could lie in the VMware Bridge Driver. I believe this is where the MAC table is maintained.

You could try stopping and starting the bridge driver to see if that changes/clears the MAC table state.

It is in located within Device Manager -> Show hidden

Re: Bridged networking just quit!

devzero — Mon, 21 Apr 2008 15:25:31 GMT

i didn`t see the network issue for a long time, but a customer had it today and i had a chance to analyze.

one VM he was using for admin purpose (cisco tools) lost it`s BRIDGED network connection.

i could not make it work again, reboot of VM didn`t help.

i switched vm from bridged network to host-only-network and set a different IP inside VM, but that didn`t help either.
i also changed the VMs nic from AMD PCNet to e1000 , but that also didn`t help.

here are some more details:

ping
from host (192.168.109.1)
to VM (192.168.109.10)
connected via vmnet1

ping request times out

C:\Programme\VMware\VMware Server>vnetsniffer /e vmnet1
len 74 src 00:50:56:c0:00:01 dst 00:0c:29:07:db:b4 IP src 192.168.109.1 dst 192.168.109.10 ICMP ping request
len 74 src 00:50:56:c0:00:01 dst 00:0c:29:07:db:b4 IP src 192.168.109.1 dst 192.168.109.10 ICMP ping request
len 74 src 00:50:56:c0:00:01 dst 00:0c:29:07:db:b4 IP src 192.168.109.1 dst 192.168.109.10 ICMP ping request

as we can see, the packets from the host appear on vmnet1 - but no response from VM.

now - vice versa

ping
from VM (192.168.109.10)
to host (192.168.109.1)
but ping in VM tells that "destination host unreachable"

let`s take a look:

C:\Programme\VMware\VMware Server>vnetsniffer /e vmnet1
len 42 src 00:0c:29:07:db:b4 dst ff:ff:ff:ff:ff:ff ARP sender 00:0c:29:07:db:b4 192.168.109.10 target 00:00:00:00:00:00 192.168.109.1 ARP request
len 42 src 00:50:56:c0:00:01 dst 00:0c:29:07:db:b4 ARP sender 00:50:56:c0:00:01 192.168.109.1 target 00:0c:29:07:db:b4 192.168.109.10 ARP reply
len 42 src 00:0c:29:07:db:b4 dst ff:ff:ff:ff:ff:ff ARP sender 00:0c:29:07:db:b4 192.168.109.10 target 00:00:00:00:00:00 192.168.109.1 ARP request
len 42 src 00:50:56:c0:00:01 dst 00:0c:29:07:db:b4 ARP sender 00:50:56:c0:00:01 192.168.109.1 target 00:0c:29:07:db:b4 192.168.109.10 ARP reply
len 42 src 00:0c:29:07:db:b4 dst ff:ff:ff:ff:ff:ff ARP sender 00:0c:29:07:db:b4 192.168.109.10 target 00:00:00:00:00:00 192.168.109.1 ARP request
len 42 src 00:50:56:c0:00:01 dst 00:0c:29:07:db:b4 ARP sender 00:50:56:c0:00:01 192.168.109.1 target 00:0c:29:07:db:b4 192.168.109.10 ARP reply
len 42 src 00:0c:29:07:db:b4 dst ff:ff:ff:ff:ff:ff ARP sender 00:0c:29:07:db:b4 192.168.109.10 target 00:00:00:00:00:00 192.168.109.1 ARP request
len 42 src 00:50:56:c0:00:01 dst 00:0c:29:07:db:b4 ARP sender 00:50:56:c0:00:01 192.168.109.1 target 00:0c:29:07:db:b4 192.168.109.10 ARP reply

as we can see, VM sends ARP to vmnet1, packets pass vmnet1, reaching vmnet1 virtual host interface and host is giving arp reply.

i can see, that host has learned correct MAC/IP of VM (arp -a) , but it seems that VM never receives those arp replies.

VM doesn`t receive ANY packet, as the interface statistics tell.

that may explain, why VM is sending arp request again and again.

now comes the best:

If i assign the VM`s network identity (ethernet0.generatedAddress) to a different VM on same host (e.g. linux vm instead of windows), the problem remains and the VM inherits the network issue - the linux vm has now the same symptom as the windows VM.

BUT - if i change the VMs mac adress to a different one, the problem goes away.

If i revert the mac, the problem re-appears.

it seems, that the vmware virtual networking/switch has got "stuck" with that specific mac adress and doesn`t forward any packets into a VM anymore.

so we have:

packet from guest-os -> vmnic -> vmnet1 -> vmnet1-host-nic -> host-os --->OK!
packet from host-os -> vmnet1-host-nic -> vmnet1 --|here must be a problem| -> vmnic ->guest-os --->NotOK!!!

this really looks like an issue with vmware virtual networking, i.e. the virtual hub/switch implementation.

vmware, will you start help finding the root cause of this this serious problem, please ?

regards
roland

Re: Bridged networking just quit!

pkuczynski — Sun, 13 Apr 2008 03:24:04 GMT

I'm also having this issue....

My post is here

www going down when there is no activity on the web server. When I do ssh
into it and ping the router, it comes back up.
I posted this issue to the vmware community blog here:
http://communities.vmware.com/message/913371#913371

funny thing is...the other 3 server dont have this issue. Could vmware tools have someting to do with this, as I didnt have this issue for the one week after migration to vmware before installing tools on this server.

P

Re: Bridged networking just quit!

jsnavely — Sat, 12 Apr 2008 14:10:43 GMT

Well, there's a core group of people that are clearly experiencing the exact same problem. Ignore the obvious differences and it's pretty clear.

The core as I see it:

windows 64-bit host
broadcom NIC
usually a 64-bit guest, but not always
typically a multiprocessor guest

As a side-note, I switched my problem guest from dual processor to single a couple weeks ago. During that time, I didn't have a single network drop. I switched it back to dual processor and had two drops in four days. This is not conclusive though, since my load level was also low during that time. I'm slammed at work and don't have time to mess with it anymore.

This server role is becoming more critical to production, so I moved it to a physical server. I had $20K in the budget this quarter to start the VI3 transition. I'm not really happy about giving that money to vmware at this point, but there's nothing comparable at the moment.

Re: Bridged networking just quit!

Sk8rCai — Sat, 12 Apr 2008 06:07:00 GMT

Hi Tim,

Sorry for the delay, I've been bogged down with production issues.

As for the driver version information. We started with v8.x.x drivers and after a firmware re-flash, we ended up with v10.x.x
I don't have the specifics to hand right now, but I can confirm I have another blade in exactly the same situation. When it comes to installing VMWare on that server I'll dump a complete firmware and BIOS version to this board and confirm that the same course of action worked again (or not!)

I agree with what's been said by others; I'm not convinced we're all suffering from the same issue but rather the same symptoms.

All I can do is post what happened to us and what go us back up and running.

If only it were that simple for everyone else.

In any case, good luck to those who are still experiencing problems.

Regards,

--CAi

Re: Bridged networking just quit!

laleger2 — Sat, 12 Apr 2008 03:38:33 GMT

I've been experiencing the same problem for several weeks now, same story, x64 Guest OS, Dell 2950 server and Broadcom NetXtreme II NIC. I'm suprised that no one else has not seen (or at least not mentioned) delayed write errors which are reported in the event log on my Guest OS every time the problem surfaces, see below. Unforunately, the information revealed by this error has not gotten me any closer to a solution.

Event Type: Warning
Event Source: MRxSmb
Event Category: None
Event ID: 50
Date: 4/11/2008
Time: 12:09:22 AM
User: N/A
Computer: EXCHANGE
Description:
{Delayed Write Failed} Windows was unable to save all the data for the file \Device\LanmanRedirector. The data has been lost. This error may be caused by a failure of your computer hardware or network connection. Please try to save this file elsewhere.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 00040004 00560002 00000000 80040032
0010: 00000000 c000020c 00000000 00000000
0020: 00000000 00000000 c000020c

Re: Bridged networking just quit!

vladpick — Fri, 11 Apr 2008 19:26:18 GMT

NM

Message was edited by: vladpick

Re: Bridged networking just quit!

Marc Desmedt — Fri, 04 Apr 2008 11:31:56 GMT

This issue is certainly not directly firmware related or broadcom related, I have Intel Network Adapters in my servers and experience the same issues.

Also I do not believe that neccesarily all "disconnect" issues are directly the same issue we are experiencing here, though most have this identical issue. Though some have a very simular issue and resolve it via some means, after all disconnects and network issues do exist with drivers, firmware, ... Often though to make out what is this bug or what is a random issue.

Re: Bridged networking just quit!

TimAlsop — Fri, 04 Apr 2008 09:23:38 GMT

Cai and Gav,

Thankyou for this information. Although I have a Dell PowerEdge 2950, the same network cards are used. I suspect that Broadcom Gigabit adapters are common on modern servers, and this might be why so many people are getting the problem.

Would it be possible for you to mention the exact firmware version which you are using now, and what you were using before when you experienced the problem ?

Thanks,

Tim

Re: Bridged networking just quit!

Sk8rCai — Fri, 04 Apr 2008 07:29:31 GMT

Hello All,

This might be a tad long, so pelase bear with me.
We have been experiencing the same Bridged network problem as the comments in this post have suggested and yesterday we seemed to have solved this problem. Your mileage may vary, so this might not work for everyone, but I havent seen anyone cover this in the post, so it might be worth a go.

Just to clarify this is what was happening...

We have an IBM Blade Center (E series) running several HS20 and HS21 blades. We recently decided to trial Virtualisation and selected VMWare Server for this trial. We installed VMWare Server (1.0.5) onto our Blade HS20 running W2k3 and installed a W2k3 Guest and later a W2k Pro guest. Once we started trying to use the network we experienced problems. When first powering up a guest OS we would get brief network connectivity then after about two minutes the network connection would drop out.
We could ping between the guest and host, but not from the guest to the rest of our network or vice-versa.
We tried a number of solutions; all mentioend here (altering the registry on both the host and guest, updating drivers, trying all manner of stuff)

After much agro we determined that it might be best to setup another machine under Linux and try this. We had a new Blade just delivered and installed SUSE 10.3 as the host and installed the VMWare Server 1.0.5 again. We installed the same guest OS's and this time we had no problems what-so-ever. At this stage we purchase paid support from VmWare and logged out Windows Host problem with them.
After a day with no problem we believed the problem to be with Windows and after really liking SUSE and not hearing anythign from VMWare, inspite of logging the call as a Severity One, we took the decision to format the server and reinstall.
We installed SUSE 10.3 again and followed the same procedure to install VmWare Server 1.0.5 as before, this time, moving the working Guest installations to this new Guest server.
To our supprise the network problem reappeared. After about five minutes of head scratch we thought we had it figured.
The blades were all using Broadcom NetXtreme II cards, and at first I thought the new blade might have something different in it, but it was the same card. However what was different was the firmware version.
We checked our versios of the Blades we currently have against the latest downloads at IBM and found some new versions which seemed to match the new HS21.

After flashing the firmware of the Broadcom the problem appears to have been resolved. I've been runnign the server overnight and we have had 100% uptime throughout the night.

I'd though it's share this as I hadnt seen this mentioned and we saw that IBM was releasing a new firmware updater only a few days ago. I woudl encourage everyone to check their vendors download page for a new firmware revision of their Broadcom Network cards and indeed, while you're at it, the BIOS as well.

Hope this resolves the issue for everyone, again I fully expect that this wont solve everyones problem,but hopefully it might solve a few.

Regards,

Cai & Gav

Re: Bridged networking just quit!

TimAlsop — Thu, 03 Apr 2008 14:57:51 GMT

A Dell PowerEdge Energy Smart 2950 III

http://configure.euro.dell.com/dellstore/config.aspx?c=uk&cs=ukbsdt1&kc=305&l=en&oc=SV12952&s=bsd&sbc=pedge_2950_green_config

Thanks,

Tim

Re: Bridged networking just quit!

ilopezc — Thu, 03 Apr 2008 14:38:12 GMT

What dell machine was this for and can you provide a link?

-IL

Re: Bridged networking just quit!

TimAlsop — Thu, 03 Apr 2008 09:42:08 GMT

Marc,

It is good to know that CDROM was causing similar issues for you.

Regarding the single CPU. I was told a few years ago that VMware products don't support more than one CPU very well, so it is better to have multiple CPU's in host and configure the guest to use one CPU than to make the guest think it has access to more than one CPU. I have therefore always configured guest systems with one CPU, even though there is an option for more.

It looks as though the issues described in this thread are caused by more than one issue - e.g. cdrom, number of cpu's.

One thing I also did recently, but not sure if it helped, was to upgrade the network card driver using latest download from Dell website. Even though I only received the server a couple of weeks ago and it came with drivers installed there were more uptodate drivers on the Dell website ...

Thanks,

Tim

Re: Bridged networking just quit!

Marc Desmedt — Thu, 03 Apr 2008 07:35:31 GMT

hmm, I have not tried using only one processor, instead of two cpu cors per Guest VMware machine.

Anyone else tried that with good results ? Basically I can reproduce this dreaded error quite easely, just perform a heavy file transfer via network en boom!

I do not have CDROM's in my Guest machines, though I can confirm that on some servers leaving the CDROM in place on your VM will cause severe VMware issues.

Before I disabled CDROM's, I had massive freezes, I/O instability, etc. Though the "network bridge error" remains dispite trying every single (good) suggestion on this thread.

About time we see some sort of official response, I don's see how VMware can not reproduce this thing when thousands of people have this very issue...

Re: Bridged networking just quit!

cbak999 — Wed, 02 Apr 2008 20:08:13 GMT

I can't remember if that has been said or not in the 27 pages....but since going back to a single proc on my 64-bit guest, the networking has not bombed since....it's been about 30 days. It used to fail almost nightly during across-the-wire backup.
CB

Re: Bridged networking just quit!

andybei — Wed, 02 Apr 2008 19:56:09 GMT

Hi Tim,

I have removed CDROM's from my guests and still experience the issue.

We're on page 23 now, I think it's a good time for a recap of solutions to various issues discussed in this thread:

1. Disable RSS
2. Disable Offloading
3. Check on Port-Security on your switch port

If after applying the above you still experience network loss, then unfortunatly you are in the same boat as the rest of us.

Good luck!

Andy

Re: Bridged networking just quit!

TimAlsop — Wed, 02 Apr 2008 19:30:52 GMT

I had this problem on my Dell PowerEdge 2950, and I noticed in vmware.log file for some of the guest systems that there were messages/errors related to CDROM. I disabled the CDROM on the VM guest config, and the problem has not occured since. I think there must be some sort of issue with CDROM on a Dell PowerEdge 2950 that is causing some sort of I/O issue. I spent ages thinking it was due to network card drivers, but it seems it was caused by something different. I hope others are able to fix this problem the same way ?

I also had same problem on VMware Server 2 Beta 2, but it didn't occur as often, so maybe VMware Server 2 Beta 2 has some CDROM driver improvements, but not quite enough

Good luck.

Tim

Re: Bridged networking just quit!

TimAlsop — Tue, 01 Apr 2008 21:10:52 GMT

Hi,

I have same problem with VMware Server 1.0.5. However, I have something which I haven't seen mentioned in this forum so far.

Initially I built my Dell PowerEdge 2950 (2 x quad core etc.) with Windows Server 2003 x64 and installed VMware 2 Beta 1. I had some fun with Beta 1 of VMware Server and since Beta 2 was due soon I waited and upgraded this last weekend. The problem I had with Beta 1 and Beta 2 are the same, and they relate to this issue being discussed, but I got slightly different results. With VMware Server 2 Beta 1 and 2, when the guest decides to stop networking, it does so for only a minute or two and then networking is restored to the guest, then the guest runs for a few more minutes and then it might stop networking again for a short while, and then run for many minutes before it happens again, etc. etc. Anyway, today I decided to uninstall Beta 2 and try 1.0.5 instead. Now, my guest works for a few minutes, and then networking stops and networking does not return until my system is restarted.

I wondered if anybody was able to install VMware Server Beta 2 and see if they get any results like mine ?

Thanks,

Tim

Re: Bridged networking just quit!

rage1605 — Tue, 01 Apr 2008 15:14:24 GMT

Just to make this more interesting, I am using Windows 2003 x64, R2, SP2 as the host, I have two guests on this machine, both Windows 2003 x64 R2 SP2 and only one is having this problem. However, I did not have this problem until I went to 1.0.5!!!!

Everything has been humming along just wonderfully until I upgraded and then all of the sudden this one VM just loses all network connection....though it believes it's still connected because I never see any errors in the event logs (besides w32 not being able to reach a server).

Does ANYONE have any idea what can be causing this? I am not looking forward to opening a ticket to resolve this issue since a few people have had their tickets open for 5+ months with VMware on this isssue. It figures, when EMC buys a company they completely destroy it.

Re: Bridged networking just quit!

jmandawg — Tue, 01 Apr 2008 00:04:01 GMT

Well, i upgraded to 1.0.5 in hopes of it fixing the problem. Unfortunately the problem still exists. Time to move on. Thanks for the good times vmware, unfortunatley i cannot run a small personal webserver without a working network adapter.

Re: Bridged networking just quit!

mikem2002 — Fri, 28 Mar 2008 21:26:19 GMT

Well, let's not be too hasty in moving to MS Virtual Server. Checkout the topic at Microsoft's newsgroup, microsoft.public.virtualserver, from two days ago titled "VS 2005 Dropping Network Connections Randomly"