MPIO iSCSI Config Questions Multiple vmKnics and Multiple pNics...

[s1xth]

Hey guys...I am trying to wrap my head around this whole MPIO and iSCSI configuration. I have read every document out there (from dell to vmware) and I am still trying to wrap my head arounda couple things and I am hoping someone could take a few minutes and just explain it a little clearer for me.

First. When you create a vSwitch and add vmknics to the vswitch, the configuration states you can only have ONE active adapter. So you go in and remove that second adapter and put it in unused.Why do you need two adapters when the second is just going to be configured as unused?? Even if that pNic fails the secondary is set for unused and no failover so what is the point of configuring the second one?

Second. When you create a vSwitch and you add multiple vmknics and then bind them to the iSCSI initiator (which I understand and have done in lab) why do some manufacturers (dell equalogic) say to create as many vmknics as pNics that you have? Is that so you as much throughput through that 1GB pipe as possible? Some walk throughs show ONE vmknic per iSCSI connection, what is the best practice?

Third and final. I read that you should have as many vswitch's as you do networks to your iSCSI network. So if I have two separate iSCSI networks going to my storage I will need two separate vSwitches and then bind those vmk nics on both vswitches to my iSCSI initiator. Sound correct?

Thanks in advance for any more detailed explanation!

[balacs]

The one key difference between 3.5 software iSCSI and 4.0 iSCSI is how load balancing is done. In 3.5, load balancing is done at the vswitch level. In 4.0, load balancing is done at the iSCSI software initiatior level.

First. When you create a vSwitch and add vmknics to the vswitch, the configuration states you can only have ONE active adapter. So you go in and remove that second adapter and put it in unused.Why do you need two adapters when the second is just going to be configured as unused?? Even if that pNic fails the secondary is set for unused and no failover so what is the point of configuring the second one?

You need to have one active physical adapter PER vmknic (and not per vswitch). For example, let say you have two physical NICs - vmnic0 and vmnic1. Create a vSwitch0 with two vmknics: vmk0 and vmk1. vmk0 will have vmnic0 as active and vmnic1 as unused. vmk1 will have vmnic1 as active and vmnic0 as unsed.

Another way to configure the above example is to create two vswitches. vswitch0 will have vmk0 and vmnic0 and vswitch1 will have vmk1 and vmnic1.

Second. When you create a vSwitch and you add multiple vmknics and then bind them to the iSCSI initiator (which I understand and have done in lab) why do some manufacturers (dell equalogic) say to create as many vmknics as pNics that you have? Is that so you as much throughput through that 1GB pipe as possible? Some walk throughs show ONE vmknic per iSCSI connection, what is the best practice?

The above example should also answer this question. You only have one active vmknic per physical adapter. Hence you are creatin as many vmknics as pnics.

Bala

Dell Inc

[balacs]

You might have already read these documents.Here it is anyway

http://www.equallogic.com/resourcecenter/assetview.aspx?id=8453

http://content.dell.com/us/en/enterprise/d/business_solutionsengineering-docs_en/DocumentsBRC_vSphere4_Blades_EQL.pdf.aspx

Bala

Dell Inc

[s1xth]

Bala... thanks for the explanation. What you said really makes better sense now. How would this setup change if I was using two physical switches? Would it not change? I wouldnt need two different subnets correct? I know the Equal does things a lot differently then other iSCSI sans and make it all easier because it automatically load balances across the controllers. I am sure once I get the unit here I will have a better understanding of how to get it to work for me. Also, a lot of the documents I am reading are based off the PS5000/6 and not the 4000 that has two 1x gigbit ports and one 100mb management port.

My goal is to have the following setup:

2x PowerConnect 5424 switches stacked together (last four ports on the switch).

1x R710 server with a total of 6 network pnics in it. I will only be using two of these for iSCSI (the others I may use later).

1 EqualLogic PS4000

What would be the best recommended network setup for this kind of setup?? Just curious, why do the ports need to be stacked on the two switches when you are just connecting to one group Ip address of the SAN?

[Andy_Banta]

How would this setup change if I was using two physical switches? Would it not change? I wouldnt need two different

subnets correct? I know the Equal does things a lot differently then other iSCSI sans and make it all easier because it automatically

load balances across the controllers.

With an EqualLogic system, no. You should interconnect the switches and make sure that all of the vmknics are on the same subnet.

EqualLogic requires all prots are on the same subnet to make port redirection work properly.

You can find out more about multipath iSCSI setups from this blog:

http://virtualgeek.typepad.com/virtual_geek/2009/09/a-multivendor-post-on-using-iscsi-with-vmware-vs...

I think the 4000 works the same, but only uses two ports (and has some limitation on grouping, I believe). Just make sure its ports and all of the vmknics on ESX are on the same subnet.

Andy

[balacs]

Bala... thanks for the explanation. What you said really makes better sense now. How would this setup change if I was using two physical switches? Would it not change? I wouldnt need two different subnets correct?

You dont need two subnets.

Bala... thanks for the explanation. What you said really makes better sense now. How would this setup change if I was using two physical switches? Would it not change? I wouldnt need two different subnets correct? I know the Equal does things a lot differently then other iSCSI sans and make it all easier because it automatically load balances across the controllers. I am sure once I get the unit here I will have a better understanding of how to get it to work for me. Also, a lot of the documents I am reading are based off the PS5000/6 and not the 4000 that has two 1x gigbit ports and one 100mb management port.

Configuration for PS4000 should be exactly similar to PS5000/PS6000 (execpt for one reduced port).

What would be the best recommended network setup for this kind of setup?

We are working on a reference architecture for a similar setup. If you can wait a week, I will send you the document. It will be posted on www.dell.com/virtualization.

Just curious, why do the ports need to be stacked on the two switches when you are just connecting to one group Ip address of the SAN?

Stacking has two main purposes. 1) You can manage multiple physical switches as one logical switch 2) You have high speed interconnect between the switches.

Bala

Dell Inc

[s1xth]

Bala...

Thanks for the information. I came across another document that explained more indepth the configuration parameters for the subnet question and found out that you only need one which is AWESOME.

It would be great if you could send me that document once it is available, I will keep an eye on that page for it though.

I understand the Stacking concept now,although I dont have the money in the budget at this time to purchase stackable switches, so I may end up with two 5424 swtiches and configure them for LAG for 4GB connectivity between each other. For now I am only going to have one array and maybe another later next year, at that time I can look at changing the switches. I read on the Dell specs page for the 5424's that you can create a LAG group with up to 8 members, does that mean i could have a 8GB LAG or is there a speed limit in the 5424 that only allows 4GB lag?

Thanks for the great information!

[balacs]

I read on the Dell specs page for the 5424's that you can create a LAG group with up to 8 members, does that mean i could have a 8GB LAG or is there a speed limit in the 5424 that only allows 4GB lag?

I think it means you canhave 8 Gb LAG (let me confirm). Did you see the 4Gb limitation somwhere documented?

Bala

Dell Inc

[s1xth]

Bala..

No I didnt see it anywhere in the documentation, I actually couldnt find too much documentation on the boxes to begin with. I tried looking for a guide on them and couldnt find much. I know some manufactures say you can have 8 members but limit the speed to 4, so you would have to create multiple LAGS to (via vlans etc) to utilize all eight. But I know with others it means 8GB, but I cant seem to find anywhere if there is an actual limitation or not.

Thanks!!

[jgoh4]

Hi and Bala,

Can you give me the instruction to stack the 2x 5424 together?

I also need to:

1] turn off iscsi optimizer

2] turn on flow control and jumbo frame

, on 5424.

Can I do stacking on 2x 5424 online?? any interruption to current iscsi traffic when i enable it?

Thanks/

[s1xth]

There was a document that Dell had that I came across that walked you

trough the 5424 configuration steps but I can't seem to find it.

Maybe Bala could take a look, I am trying to find it but I am not

having much luck.

The document was titled something along the lines "configuring a Dell

powerconnect 5424 switch for an equallogic env" or something along

those lines.

I'll keep looking.

Sent from my iPhone

On Nov 5, 2009, at 4:33 AM, jgoh4 <communities-emailer@vmware.com

[balacs · Accepted solution]

Is this the document that you are refering to? http://attachments.wetpaintserv.us/Hb2187XrZtZHUBMwbBrD4A%3D%3D1015900

I also confirmed that there is no 4Gb limitation on LAG for PowerConnect 5424. You can LAG 8 ports and get 8Gbps pipe.

Bala

Dell Inc

[jgoh4]

Can LAG of 2x 5424 be done online??? Any interruption to MPIO iscsi traffic??

[s1xth]

Thanks Bala...that is the document! Do you happen to know of any document that descirbes how to configure a LAG group via the CLI for the 5400 switch? Thats all I am looking for now.

Thanks!

[balacs]

Here is the CLI guide: http://support.dell.com/support/edocs/network/54xx/en/CLI/HTML/index.htm.

You can find PDF here: http://support.dell.com/support/edocs/network/54xx/en/index.htm

Setting up LAG should not be interrupt your existing traffic and can be done online. But you should be aware that there is a good chance of a misconfiguration, that might affect the other traffics.

Bala

Dell Inc

[balacs]

Here is the reference architecture document: http://content.dell.com/us/en/enterprise/d/business_solutionsengineering-docs_en/DocumentsVMware-vSphere-Reference-Architecture-SMB.pdf.aspx

Bala

Dell Inc

[s_buerger]

.