Vmware ESX Network configuration

[RAINBOW007]

Hi Guys,

I'm configuring ESX server on Cisco networks.

Here is my configuration details

I got two VLan's for the ESX server 6 and 7

VLAN 6---LAN--192.168.3.0/24

VLAN 7---SAN server,ESXHost, are installed on this vlan--192.168.4.0/24

No. of Physical NIC's(VMNIC's) on ESX Host---- 6

I have VLAN Routing Enabled on the Network, so all VLAN can communicate with each other. But i want to follow the best practices, so please help me in securing every part of the Network(such as SAN,VMWARE ESX HOST Network , Vmotion e.tc)

Current Network configurations on the First ESX Host are as followes:

============================================

Vmnic0----

---

||--

---

VSW0VLAN 6 --- Virtual Machines

Vmnic1----

---

Vmnic2----

---

||----

---

VSW1 --VLAN 7 - VMKernel-iSCSI---Ip:192.168.4.20

Vmnic3----

---

Vmnic4--

---

VMotion--VLAN 10 ......192.168.10.10

Note: I don not have VLAN 10 on the Network Yet, should i have to create a VLAN 10 for the VMotion? also i noticed that the Default Gateway for both Vmkernal(iSCSI) and VmKernal(Vmotion) is same, which is in VLAN 7 (Ip :192.168.4.2). Will this work for Vmotion? I'm bit confused here

SAN IP:192.168.4.1 (VLAN 7)

First thing is, have i configured everything right?

Second, should i have to create VLAN 10 and enable the routing between VLAN?

As the routing between VLAN's is already enabled on my Network, will the above setup creates lot's of traffic on my Network?

What is the best way to configure this? should i have to create any more vlan's or create any more network configuration on VMWARE ESX server to have redundancy or to achive a best practices?

I'm really desperate here...

[fhrivers]

Must all your VLANS talk to one another? There's no reason why your user LAN should be able to communicate with your storage network? Don't make it more complicated than it needs to be. Each storage controller should preferably be in a separate broadcast domain.

With 6 NICs I'd use nics 0-1 for VM traffic and nics 2-5 for iSCSI traffic. You can carry vMotion traffic on two of the iSCSI nics.

[RAINBOW007]

Hi,

thanks for the quick reply.

There is no need for inter vlan communication between LAN and the rest of the VMWARE&SAN.but as i have no idea how i can create these network configurations, i started creating as above.

say suppose, if i create a iSCSI (vmkernal) with one subnet and when try creating vmotion on the same subnet , i'm getting error saying i can not have same subnet for both of them. can u please give configuration tips here for the 6 NIC's i got.As this is really urgent i really appreciate your help and quick reply .please help me.i'm really getting confused with this networking configuration. can u please give step by step if u can please

thanks a ton

[RAINBOW007]

Can i also clear me on where and which one should e on the same VLAN and which one can be on a different VLAN? i mean which VLAN should the following can be on

VMware ESX host-SCVLAN no? -

---

Vmnic No?

VMotion(Vmkernal) -

---

VLAN no ?----

---

Vmnic No?

ISCSI(Vmkernal)--VLAN no?--

---

Vmnic No?

VM's-----VLAN no?--

---

Vmnic No?

Also do u know how i can stop the VLAN access between each other please?

[fhrivers]

As I suggested earlier, you can use two nics per host for Virtual Machine traffic. These vmnics would be connected to your regular default network with your other servers. The other four NICS could be teamed and used exclusively for iSCSI and vmotion traffic. If you can do two vlans I would team vmnic 2, 4 on one vswitch and send them to one VLAN and vmnic 3,5 on another vswitch and send them to another vlan. I would then enable vMotion on one of the NIC teams.

[fhrivers]

Can i also clear me on where and which one should e on the same VLAN and which one can be on a different VLAN? i mean which VLAN should the following can be on

VMware ESX host-SCVLAN no? -----------------Vmnic No?

VMotion(Vmkernal) --VLAN no ?-------------Vmnic No?

ISCSI(Vmkernal)--VLAN no?-----------Vmnic No?

VM's-----VLAN no?-----------------Vmnic No?

Also do u know how i can stop the VLAN access between each other please?

frivers:

I don't know the IOS commands specifically but by default, VLANS shouldn't be routable. You simply create a VLAN and assign it to a group of ports and that's it. You have an isolated broadcast domain. If you can post your switch configuration, I can probably see what needs to be changed.

>

[RAINBOW007]

I have modified my networking configuration as attached, but i don't know where to add vmotion? i mean under vswicth?but at the movement according the config i did i could see both vswitch1 and vswitch2 are in same VLAN 7, this means i should create a separate vlan say suppose vlan 10 and i then assign my ESX host machine to VLAN 10 and then create Vmotion in VLAN 10. Am i correct. sorry to be a pain.please correct me. i could write an ACL to stop the communication between LAN VLAN and iSCSI VLAN.but does the SC and iSCSI should be on the same subnet to see a SAN? my understanding is that the SAN should be iSCSI subnet(in this case VLAN 7).

so if i create another VLAN 10 and place the ESX host in that VLAN 10 and then i should be OK with the rest of the configuration and would be able to see the SAN which will be in VLAN 7(if i'm correct)?

Please advise

[RAINBOW007]

please also look at the attached proposed diag.i want to know one thing here please.after creating a vmkernal, which enable the communication for iSCSI, should i enable the vmotion on the same portgroup or should i have to create another portgroup of type vmkernal and enable vmotion on the newly created one, in which case i will have to go for another subnet(VLAN). please clarify me.

[RAINBOW007]

If i understand correct

vsw-1-vlan 6-vmnic2,4 -

---

PhysicalSwitch(enabled VLAn's 6 and 7)----

---

on this vmkernal port-group i can enable iSCSI

vsw-2----vlan 7 -

---

vmnic3,5---Same PhysicalSwitch(enabled Vlan's 6 and 7)--

---

on this vmkernale port-gorup i can enable Vmotion

but my question is what about Service console? should i create a separate VLAN for this?sorry i can not understand this as you have not mentioned about SC's vmnic at all out of 6 nics i got. all looks confusing. can you be bit more clear on these please.

also, if the SC and iSCSI are in different VLAN's should the routing be enabled between these VLAN's? or not? OR both the SC and iSCSI should be on the same VLAN?

please advise on this as i need to configure this on monday please.

[fhrivers]

I'm using ESXi and I assume you were as well. As you may or may not know, ESXi doesn't have a service console. You should enable the SC on the same vSwitch as vmkernel if I remember correctly.

[RAINBOW007]

i'm using esx 3.5 enterprise edition not i, as we have purchased the esx 3.5 enterprise license.

[depping]

I'm assuming the following: You will be using the software iSCSI initiator and have got 6 physical nics.

vSwitch0 -> vmnic0(first onboard) + vmnic2(first pci port) -> virtual port id load balancing

Service console

VMkernel + VMotion

vSwitch1 -> vmnic1 + vmnic5 -> virtual port id load balancing

Service console for iSCSI

VMkernel for iSCSI

vSwitch2 -> vmnic3 + vmnic4 -> virtual port id load balancing

Virtual Machine portgroups

Now, I would most definitely have the VMkernel + VMotion on a seperate VLAN, the first Service Console on a seperate VLAN and the Storage SC+VMkernel on a seperate VLAN? Why? Security / Broadcasts. The VMotion VLAN doesn't need to be routed to the rest of the network. No point in doing so as it's traffic will only be directed to the other hosts.

btw check these excellent articles by Ken Cline on this topic on his blog: http://kensvirtualreality.wordpress.com/

Duncan

VMware Communities User Moderator

-

---

Blogging: http://www.yellow-bricks.com

Twitter:

If you find this information useful, please award points for "correct" or "helpful".

[RAINBOW007]

thanks for the reply .if i put service console on to different VLAN to iSCSI, i think i need to enale inter VLAN routing between these two VLAN.am i correct. please correct me.when i try creating vmkernal iSCSI, it will have a default gateway of that VLAN, but when i try creating vmkernalvmotion, i could only see the same default gate, which doesn't make sense, as the ip add of the vmotion will be in different VLAn to iSCSI.how to overcome this?please clarify me on VLAN routing, especially what needs to be on the same VLAN and what needs to be on completely different VLAN and between what VLAN's the routing should be enabled and which VLAN's should be kept isolated from all.i really apreciate your help, as we have the vmware esx enterprise license and needs configuring on monday(tomorrow).i don't know where i'm going wrong? i think the SAN and iSCSI will be in the same VLAN, also Service Console for the host, hence the ESX host must be on separate VLAN, but the VLAN routing must be enabled between these VLAN's(am i correct?)?.i should create a VmkernalVmotion on a different VLAN, which should be completely isolated from all the vLAN's(am i correct?) . sorry to be a pain.please correct me.i looking forward for your help

[RAINBOW007]

Hi Duncan,

If i got it right.

vsw0- vmkernal+VMOTION---VLAN-1-no need of routing to any other VLAN's

vsw1--vmkernal+iSCSI

---

VLAN-2--

vsw2--Virtual machines--VLAN-3-----connected to internal LAN

But having assumed the avove, what about the first Service console for ESX host? which VLAN it should be in?does it goes to separate VLAN or the same VLAN as vmkernal+iSCSI(which in this example VLAN-2)?should i have o create any additional Servce Consoles? if so which VLAN it goes to. this where i'm confused.please advise

many thanks

[depping]

you need to add a service console to your first vswitch0, and this would be the one that you want to have access to from the network.

and add a service console to vswitcht, this one would only need access to your iSCSI san and can be on the same VLAN as your iSCSI is.

Duncan

VMware Communities User Moderator

-

---

Blogging:

Twitter:

If you find this information useful, please award points for "correct" or "helpful".

[RAINBOW007]

Hi Guys,

i managed to configure as attached. can someone have a look at the config let me know if anything is wrong?one thing i don't understand is when i try creating the second service console port in VLAN 7, (same VLAN as iSCSI), the default gateway for this one is showing the first SC port which is in VLAN 30.is it OK are should i have to do anything? please advise