VMware Cloud Community
baabujatin
Contributor
Contributor

ESXi6 - New deployment network architecture best practices

friends .. this is my first post to this and very new to VMware .. need your help please to guide me to the best practices for the network architecture. Can you please help

My infra currently is

2 X IBM server with 8 LAN ports 1GB each -

1 X IBM SAN with 4 Ports per controller to be connected as iSCSI..

now i am not sure how will i service

1. Storage and how many ports

2. Management

3. vMotion

4. VMs network

5. FT

those IBM servers wil connect to two cisco switches

please help me with architecture.

0 Kudos
6 Replies
Mattallford
Hot Shot
Hot Shot

Hi there,

In all honesty, it is going to be hard for someone here to recommend the best architecture without knowing the workloads and requirements.

Ideally this would have been planned prior to the purchase of the hardware and then you can purchase the hardware as per the requirements, so it seems things have been done a bit backwards here.

Firstly, do you need FT in your environment?

Secondly, given there are only two hosts, can we make the assumption the vMotion traffic will be minimal, and likely only be used when maintenance is being performed and a host needs to be placed into maintenance mode?

Will you be using vLANs in your environment?

Assuming dedicated NICs for FT isn't required, you could break them up evenly into the 4 remaining services as per below. But I'd question the benefit on have 2 NICs sit there for vMotion presumably not doing much at all. Then again, the same could be said for ESXi Management.

2 x iSCSI

2 x ESXi Management

2 x vMotion

2 x Virtual Machine

Or you could combine the vMotion and Virtual machine traffic into two NICs and give iSCSI 4 adapters:

4 x iSCSI

2 x ESXi Management

2 x virtual machine and vMotion

Or you could leave some dedicated for vMotion if the requirement needs that, and place ESXi management on the same adapters as virtual machine traffic:

4 x iSCSI

2 x ESXi Management and virtual machine

2 x vMotion

I guess what I'm getting at, is there is a range of combinations, many more than the above, that would be suitable for the environment without knowing anything about it. Are the virtual machine workloads storage heavy? Network heavy? Neither? Both? How often will vMotion be used? Are you using standard or distributed switches?

Cheers, Matt.

VCP6-DCV | VCAP6-DCV Deploy @mattallford If you found my answers useful, please help me by marking them as Helpful or Correct!
0 Kudos
baabujatin
Contributor
Contributor

hi Matt thanks .. actually what you told me i already knew most of it but what i am not clear is ...

say if you saying 4 lan cards for iSCSI means two going to switch one and two to switch two... and i can do a bonding ... but how it will work in case of host servers... if i have only two lan cards per host .. then one will go to switch one and one will go to switch two .. which means my VM network is  eventually connected on 1Gb connection only ... So whats the point of giving two bonded uplinks for storage.

same with Vmotion also .. if i give two network cards per server then it means one will connect to one switch and another to switch 2... correct me if my understanding is wrong.

0 Kudos
Mattallford
Hot Shot
Hot Shot

same with Vmotion also .. if i give two network cards per server then it means one will connect to one switch and another to switch 2... correct me if my understanding is wrong.


That is good guidance. If your server have 2 network cards with 4 adapters each, then you are best to cable 4 ports from card 1 to switch 1 and 4 ports from card 2 to switch 2. You would then use at least one port on each card for each 'service' (such as VM traffic, iSCSI etc).


So whats the point of giving two bonded uplinks for storage.


Again, this is really a requirements discussion. If your workloads were storage heavy but not necessarily network heavy, then more connectivity to the SAN may be of benefit.


If you aren't sure, I'd probably suggest using 2 vmnics for each service (ESXi management, vMotion, iSCSI and VM traffic). You can monitor and adjust if required down the track.


say if you saying 4 lan cards for iSCSI means two going to switch one and two to switch two... and i can do a bonding


On a side note, ensure you investigate how to correctly set up the ESXi networking for iSCSI multipathing. IBM should have some guidance here on how to best configure ESXi networking to connect to their SAN.

VCP6-DCV | VCAP6-DCV Deploy @mattallford If you found my answers useful, please help me by marking them as Helpful or Correct!
0 Kudos
baabujatin
Contributor
Contributor

Thanks Matt for the help..

Can you please help me clear my concept…

I have two lan cards with 4 ports each.. now if I take 4 ports 2 from each lan card for iSCSI will it be a good idea… as eventually I will not get 4Gb but theoretically 2Gb max from each switch. Now question is .. for the VM to VM communication or VM to internet communication … the network which gets used is VM network.. so load is on this network and not iSCSI or somewhere I have a wrong understanding.. this concept is actually not clear with me.

0 Kudos
Mattallford
Hot Shot
Hot Shot

Now question is .. for the VM to VM communication or VM to internet communication … the network which gets used is VM network.. so load is on this network and not iSCSI

Yes, that is pretty much correct. VM to VM communication (lets say one VM on each host for now) will happen over the NICs that you bind the VM traffic portgroup to.

So, for example, lets say there was some network transmitting happening between two virtual machines, they would no use the same NICs on the host that your iSCSI network uses. They will use the same switch, assuming you are plugging all 8 NICs off each host into the same switch?

If you were copying a 4GB file from VM1 to VM2, the network transmit between VMs would happen over the VM network, but then VM2 needs to write that file to disk as it receives it, so the disk writing action would happen over your iSCSI network. So from VM2's perspective in this scenario, it is receiving packets over the VM network, and writing them over the iSCSI network, which ideally are different NICs on the host. It isn't a great idea to share a NIC for iSCSI and other traffic, especially on a 1GB network.

I have two lan cards with 4 ports each.. now if I take 4 ports 2 from each lan card for iSCSI will it be a good idea… as eventually I will not get 4Gb but theoretically 2Gb max from each switch.

This depends on how IBM want you to configure the ESXi hosts to talk to their SAN, and here I'm mostly talking about the Path selection Policy (which route/method should the host use to send data to the SAN).

Some policies do round robin, so it will send data down all NICs configured for iSCSI on a particular rotation. Others will use policies such as fixed, where is will use the same path every time unless it can no longer use that path.

I'm not familiar with IBM sorry so I can't help much further.

Cheers, Matt.

VCP6-DCV | VCAP6-DCV Deploy @mattallford If you found my answers useful, please help me by marking them as Helpful or Correct!
0 Kudos
baabujatin
Contributor
Contributor

Matt, please tell me one thing...

so i have 8 Lan cards .. Lets say L1...L2....to .. L8

so lets say i bond L1 and L2 together and give it iSCSI to switch one

and L7 & L8 bond and give it to iSCSI to switch Two...

so in total i am left with 4 Lan cards... again if i bond then in two each ... say L3 & L4 to switch 1 and L5 & L6 to switch 2...  and i chose this for my all other traffic ... like Mgmt + vMotion + VMNet ... is this is good practice .. and can i even do this ..

other thing which i am not clear is ... can i have VLAN done for all 4 types of traffic ... like with the above planning i want iSCSI to be on seperate VLAN (10) .. MGMt on 20 .... vMotion on 30 ... VMNet on 40... is this is possible as i am able to do this on HyperV... and at last if this is possible .. i want all my client VMs to have their own VLAN.. like if i have client 1 .. he will have his independent VLAN .. so will client 2 and 3...

please advise if above makes any sense

0 Kudos