Network Configuration

[biolog5]

Good afternoon All,

I am quite new to vmware, so please bear with me :~}.

I  am in the process of setting up a small vmware cluster for my  institution. The cluster will consist of 2 Proliant DL380G7s connected  to P2000 G3 iSCSI SAN and one DL180G6 used mainly for DR and backups (on  another site). All hosts will run ESXi 4.1 and will be managed by 1 vCenter Server.

Each of DL380s has 8nics. I am planning to use 4 for iSCSI, 2 for vm traffic and 2 for management/vmotion/special vm traffic (active-standby configuration on 2 port groups)

My main concern is with the management network. Ideally I would like the hosts to live on the management vlan using private addresses to talk to vCenter server and syslog servers. To test this, I set up 1 host with vswitch0 with vmk0 with right ip (say 10.10.0.50), netmask (say 255.255.255.0) and no gateway. I can connect to host via vSphere client and ssh and all is fine.

However, since there is no gateway, I cannot use ntp or dns (both of which are on production network, with real IP adresses). I think I will be able to get hosts to syncronise time with vCenter server, and possibly with some other vms' that will be on the management network. But, I wonder if absence of gateway and dns will cause me problems elsewhere (assuming that all vms' will have manually configured ips')? If it does, what would be the most appropriate way to set up a management network, without inter-vlan routing and without exposing hosts to the production network?

Many Thanks for your help!

S.

[weinstein5]

Welcome to the Community - You can make it work without a gateway but I would not recommend since it makes you life easier if you had access to DNS and if you plan to set up HA it by default uses the management gateway address to test the health of the network - I would set up some form of intra-vlan routing -

[kjb007]

If you really want to isolate, as needed, you can configure your vcenter server to forward ip, and set a static route on your esxi hosts to the IP's / services you need access to.

-KjB

[biolog5]

Ok, Thank you for this - very useful to know about these ways of doing it!

Second question is it possible to set up a second vmk (vmk1) on vSwitch0, with with production ip address/netmask/gateway, and then disable management on it? I tried, but unticking Management box does not stop ssh / vSphere client from connecting, and I could not find a way to disable management via cli.

Alternatively will using esxcli to set up a firewall to block all inbound connections to production vmk be a good idea?

Apologies for barrage of questions - just trying to see all the options.

Best,

S.

[kjb007]

Unchecking management means ESX will not use it for management purposes, meaning no inter-host communication/hb/etc, it does not disable the NIC.

With regards to firewall, you must now be referring to ESXi 5, your post mentioned 4, no worries, you can set fw rules there if you want, but I would change the allowedip instead of disabling all, but that's up to you how you want to customize it.

-KjB

[biolog5]

Oh ok - do you have any more information about management tickbox? Intuitively, I thought that unticking it would disable vSphere access and ssh... Would be really good to know if that is possible!

With regard to firewall - I was infact talking about 4.1, but I misread one of Vmware KBs, which was talking about ESX rather than ESXi. Apologies for that!

Thanks you,

S.

[kjb007]

Good question, I've never actually seen an exact definition for that management traffic, other than little tidbits, such as it is used for hb, etc.

As far as unticking the checkbox, it will definitely not do what you would think.  I can attest from experience, that adding the additional vmkernel port, even without checking the box, does nothing to stop use of that IP if you know it for ssh/vsphere client connections.

-KjB

[biolog5]

Ok, Thank you.

In that case - will this be a good option, security-wise:

Have vSwitch0 with 2 vmks', one on management network (vmk0) and one on production network (vmk1).

Use vCenter Server on management network to manage the hosts. Also use Management network for vMotion (this will be used extremely rarely - mostly for when a host needs to be rebooted).

Use the production network for ntp and dns.

Once this is set up, enable the lockdown mode, so that noone can connect to ssh (which will be switched off anyway) or vSphere client on. the server. Obviously, inbound access to hosts will be blocked at main firwall level. If I need to access the console I will either do it locally or via iLO (also on management network).

vSwitch0 will have vmnic0 active and vmnic1 on standby for these two networks. It will also have a vm network for a few special machines (one being a student nat box and another a monitoring box, to which all traffic is mirrored), with opposite configuration - so vmnic1 active, vmnic0 on standby. This should provide me with needed redundancy.

What do you think?

Best Regards,

S.

[kjb007 · Accepted solution]

That sounds like a very workable solution to your specific needs, and limits your exposure on both networks, I like it.  Lockdown mode is something I haven't played with much as my environment requires me to access the hosts directly often, but it should work for what you want it to do.

-KjB

[biolog5]

Brilliant, thank you.

Only two more questions:

1) Why would I want to access the host directly, providing vCenter Server is accesible

2) What Happens with HA if the main gateway should become inaccesible? Will it trigger?

Many Thanks!

Stan

[kjb007]

Aside from vCenter losing acess, there are times when network connectivity is loss to the host as well, and at those times, if the server is still up, you can still access the host via the console.

HA can be configured with additional IPs to validate isolation.  Also, your isolation response has to be configured to shutdown/poweroff the virtual machines fot the response to trigger.  If you fear your gateway is not as reliable, configure another IP address for HA to test against, see hthe availability guide for those advanced options http://www.vmware.com/pdf/vsphere4/r41/vsp_41_availability.pdf.

-KjB