duhaas
Enthusiast
Enthusiast

ESX 3.0.2 Network Config

Jump to solution

Just wanted to get some advice on my current network config to clarify what I would think would happen. I currently have a Proliant DL360G5 with 6 NICS

VSWITCH0 - VMNIC0/VMNIC1 - This services ISCSI/Service Console

VSWITCH1 - VMNIC1/VMNIC2/VMNIC3 - This services my network connectivity

VSWITCH2- VMNIC4 - This services VMOTION *not actually using yet

My question with the above config is if I loose network connectivity from one of the two links for VSWITCH0, should I assume I will maintain my network connectivity to my ISCSI luns? I guess the same question would apply to VSWITCH2 as well. Here is a shot of what my VSWITCH0 looks like:

0 Kudos
1 Solution

Accepted Solutions
depping
Leadership
Leadership

even if you setup an etherchannel it will only go down 1 nic, because it's ip-hash on the esx side. with just 1 target there's only 1 ip, and because of that there's only 1 route. you can check this on the commandline with esxtop.

Duncan

My virtualisation blog:

View solution in original post

0 Kudos
16 Replies
johnswb
Enthusiast
Enthusiast

I have a simmeller setup, you should not loose connectivity if one of your vmnic's stop responding. One way to test it is to unplug vmnic0 or vmnic1 one at a time. Which I'm guessing you have done. One other thing, the switch the physical connections are connected to needs to be setup as well in order for it to work.

I would only question why you have vmnic1 on both switches? Why not just use vmnic 0 & 1 for sw0 and vmnic 2 & 3 for sw1?

Will Johnson VCP on VI3 / VI4 vSphere 5
dominic7
Virtuoso
Virtuoso

The only change I would make to your vSwitch config is to create 2 port groups, one for the service console and a second one for the iSCSI connections. Then set vmnic0 as primary for the service console and vmnic1 as a standby for the service console, and then set vmnic1 as the primary for the storage network and vmnic0 as as standby for the storage network. That will guarntee that unless you have a nic failure that you will have a dedicated nic for the console and for the iSCSI

0 Kudos
duhaas
Enthusiast
Enthusiast

that was just a typo in my current network setup, i dont have that nic in both switches

0 Kudos
duhaas
Enthusiast
Enthusiast

This might be an obvious question, but I will assume that service console is so important since there is no way to manage esx if the service console is unreachable. Can you also expand when you say make sure you have the switch config setup properly? Currently I have two cisco 3560's for my iscsi network and each of my two interfaces going into each switch, this way I can loose one switch and still have access to the disk.

0 Kudos
dominic7
Virtuoso
Virtuoso

Edit: sorry, the ascii pictures are turning out horrible. Hint Hint Rob/John, why is it so hard to preformat text now?

Service Console (portgroup)----


\

----


/

-


vmnic0 ( physical switch 1 )

vSwitch 0

----


\

-


vmnic1 ( physical switch 2 )

iSCSI (portgroup)----


/

The Service Console portgroup should be assigned vmnic0 as an active adapter, and vmnic1 as a standby

The iSCSI portgroup should be assigned vmnic1 as an active adapter, and vmnic0 as a standby

So, as long as everything is working normally, the Service Console and iSCSI port groups will each have their own

physical nic and seperate data paths. Should either vmnic ( or physical switch ) fail, they will be collapsed onto a

single physical nic until the nic is replaced or physical switch is brought back online.

0 Kudos
duhaas
Enthusiast
Enthusiast

Appreciate the followup advice. Any recommendation on what if anything would need to be configured on the back end 3560 switch? I have two currently, port 1 in switch 0 goes to one, and port 2 in switch 0 goes to the other. Currently I have each port configured for 1000FULL, MTU set @ 9000, although I realize 3.0.2 or any version doesnt support jumbo frames, I guess I'm holding out hope.

0 Kudos
johnswb
Enthusiast
Enthusiast

Our network team has to put the ports we use in a channel port group. This keeps the ESX server connections from flopping. We have 6500's and 3500's. The only time a channel port has to be created on the switch is when there are more then on cable teamed. From what I understand they have to be in pairs. (2.4.6 and so on).

I've team all of our console connection for the redundancy. The ESX server will continue to work if you loose connectivity to the console you just won't be able to manage it.

I hope this helps

Will Johnson VCP on VI3 / VI4 vSphere 5
0 Kudos
duhaas
Enthusiast
Enthusiast

what puzzliing to me is when i go into the following:

It would appear that in ISCSI's case it has two active connections, but am I right in that it only uses one path currently, so I should really have one of them as a standby adapter? here is my config on the service console:

Any advice on best config will be greatly helpful. Currently I dont have my ports trunked since im spanning two seperate switches.

0 Kudos
johnswb
Enthusiast
Enthusiast

duhaas, looking at your fist image you have two active connections. Looking at your second image you have two vmnics but on one nic is active. If you won't your console to have a standby nic select one of your vmnics and move it to "Standby Adapters".

Again, the best way to test your configuration is to unplug one cable at a time. This will give you the real results regardless. Here is an image on how our is setup. I hope I've been some help?duhaas, looking at your fist image you will have two active connections. I wouldn't put one of you connections in standby unless you thin Your second one does not.

Will Johnson VCP on VI3 / VI4 vSphere 5
0 Kudos
johnswb
Enthusiast
Enthusiast

Sorry for the delay in replying.

Will Johnson VCP on VI3 / VI4 vSphere 5
0 Kudos
duhaas
Enthusiast
Enthusiast

Appreciate the response, one more thing to clarify. Even though i have TWO nics in my ISCSI "TEAM", its really only a team from a Network failover standpoint and not a aggregation standpoint.

0 Kudos
johnswb
Enthusiast
Enthusiast

From what I understand if you have two vmnic and one is in standby that would be a "Network failover" and two vmnics in a active config would be an aggregation.

Will Johnson VCP on VI3 / VI4 vSphere 5
0 Kudos
depping
Leadership
Leadership

depending on the usage it would be teamed. aggregation in network terms is only with teaming based on IP-Hash, and that would require a channel on the physical switch side.

anyway, with iscsi you only connect to a single destination... so it's never load balanced unfortunately. and multipathing isn't supported yet. active/standby setup should be fine. and be sure to mix several types of nics in a team. for example a onboard + pci, this way you have the most redundancy.

Duncan

My virtualisation blog:

johnswb
Enthusiast
Enthusiast

depping, that was very well put!

Will Johnson VCP on VI3 / VI4 vSphere 5
0 Kudos
duhaas
Enthusiast
Enthusiast

so are you saying then if i setup etherchannel on the switch side, and plug my two nics for iscsi into them, i should see traffic flowing down both pipes for iscsi if both are active?? and i guess really what i am trying to ask is if active standby is really your only option with iscsi, since having two doesnt seem to send down both paths?? or am i wrong?

0 Kudos
depping
Leadership
Leadership

even if you setup an etherchannel it will only go down 1 nic, because it's ip-hash on the esx side. with just 1 target there's only 1 ip, and because of that there's only 1 route. you can check this on the commandline with esxtop.

Duncan

My virtualisation blog:

View solution in original post

0 Kudos