Broadcom iSCSI Adapter - BCM 5709 not working with iSCSI

[Tomek24VMWARE]

Hi All

I testing new ESXi on my server Dell R710 with 4 TOE port BroadCom Ethernet adapter.

000:001:00.0 Network controller: Broadcom Corporation PowerEdge R710 BCM5709 Gigabit Ethernet

000:001:00.1 Network controller: Broadcom Corporation PowerEdge R710 BCM5709 Gigabit Ethernet

000:002:00.0 Network controller: Broadcom Corporation PowerEdge R710 BCM5709 Gigabit Ethernet

000:002:00.1 Network controller: Broadcom Corporation PowerEdge R710 BCM5709 Gigabit Ethernet

This ethernet device have support for dependent BroadCom iSCSI Adapter.

when I configure vmk and NIC with vmhba34 - then iSCSI not working

when I configure vmk and NIC with Software iSCSI Adapter iSCSI working for me.

pleas see image:

and esxcli configuration for vmhba34

Does anyone have a similar problem and knows how to solve this?</span>

Tom

[Tomek24VMWARE]

jumbo frames for 5709 will be available in next release of ESXi mayby in vSphere 6

[irvingpop2]

Looks like VMware finally pushed out an updated driver that supports Jumbo Frames with the BCM 5709 hardware initiator:  https://my.vmware.com/group/vmware/details?downloadGroup=DT-ESXi50-Broadcom-bnx2x-17254v502&productI...

From the release notes:

Version 2.72.10 (Mar. 13, 2012)

==========================

   Enhancements

   ------------

   1. Change: Default MAX MTU support to 9000 for iSCSi offload

              for ESX5.0.  (Approved by VMware)

Anyone feeling brave?

[realdreams]

I am going to test the latest driver on BCM 5709C tomorrow.

[irvingpop2]

Looking forward to seeing your results!

[Tomek24VMWARE]

Yes, new broadcom work excelent with iSCSI and support MTU 9000. I have tested last driver.

If You have connection issue (flapping connection) You must add options for iSCSI module:

from SSH console

esxcfg-module -s 'error_mask1=-1 error_mask2=-1' bnx2i
or 

esxcfg-module -s 'error_mask1=4294967295 error_mask2=4294967295' bnx2i

and reboot ESXi serwer, then iscsi issue should be resolved.

my server boot up in 2.5 minutes thanks to the new drivers from broadcom.

Tomek

[tsimons]

Tom-

What was your boot time before?

How many targets do you have listed in your discovery?

~Todd

[scerazy]

It does certainly boot way quicker. On a single target, or on 3 targets

And in addition it also works on BCM 5708 (onboard in PowerEdge 2950 with iSCSI key enabler), as there is the same driver!

Seb

[realdreams]

I installed the latest driver and set vswitch and vmkernel for iSCSI to 9000.

Jumbo frame enabled on Promise M610i. I can successfully ping iSCSI port on the SAN with jumboframe(8970) but iSCSI data is not using jumboframe per port statistics on the SAN. I see only 12 jumboframes which were from ping......

Is there anything else I need to do to enable jumboframe for iSCSI?

And I am trying to set up SAN boot. Broadcom MBA cannot find the iSCSI target (Boot LUN on the SAN). I use the same settings on Windows Server 2008 R2 iSCSI initialator and it can mount the LUN with no problem. I am going to update Broadcom MBA and try again. Any ideas why SAN boot cannot find target?

HP ProLiant DL385 G6

NC382i(bcm5709C)*2

Thanks

[realdreams]

esxcfg-module -s 'error_mask1=-1 error_mask2=-1' bnx2i
or

esxcfg-module -s 'error_mask1=4294967295 error_mask2=4294967295' bnx2i

What does it do? I don't remember seeing this in release notes.

[Tomek24VMWARE]

esxcfg-module -s 'error_mask1=-1 error_mask2=-1' bnx2i
or

esxcfg-module -s 'error_mask1=4294967295 error_mask2=4294967295' bnx2i

if your have iscsi issue (connection error to target in /var/log/syslog.log) You must set params for iscsi bnx2i driver. Reboot ESXi.

If Your initiator have error in boot process and boot take long time probably You must modify params for resolv issue.

I got this info from broadcom support for supporting HP P2000 3G storage with broadcom 5709.

Before new driver and no set params error_mask ESXI boot up aprox. 7-15 minuts, after update driver and set params ESXI boot up in 2,5 minuts.

Support Broadcom help me very much.

[lexalex]

Useless

http://communities.vmware.com/message/2196258#2196258

[scerazy]

Are you talking about yourself in this instance?

[lexalex]

I'm talking about

esxcfg-module -s 'error_mask1=-1 error_mask2=-1' bnx2i
or

esxcfg-module -s 'error_mask1=4294967295 error_mask2=4294967295' bnx2i

[scerazy]

None of this was needed to get the jumbo frames with new driver in my test (with my arrays)...

But then again, I did not test it on ESXi4 (only on 5, where it works perfect)

And the parameters are for a specific case (connection error to target in /var/log/syslog.log) only