LUN with multiple IP addresses

[vidkun]

I have two questions actually. The first being: I have a machine running OpenFiler with 3 iSCSI targets. This machine has two NICs each with its own IP address. There are two ESX hosts. Currently in each I have added a new VMFS datastore from one of these LUNs and then added the other two as extents to that VMFS Datastore. This is mirrored on both hosts and all using only one of the IP addresses from the openfiler SAN and Vmotion working between both hosts. Could I take one of the hosts and add the LUNs to it through the second IP address and have ESX still see it as shared storage so that each host would be accessing the same LUN(s) through separate IP addresses and still have Vmotion working across them? If so, how exactly would this be done and would there be any forseeable performance/stability issues?

Secondly, what sort of issues would we run into if I setup a second physical machine running OpenFiler and added the LUNs on this second box as extents on to the same datastore? This should theoretically create on massive datastore spread across two physical machines and accessible as a shared datastore by both ESX hosts. Thoughts?

NOTE: I know we could bond the two NICs in the SAN, but we had lots of performance/stability issues when we had the machine running a standard linux distro with bonded nics and an NFS share as the datastore. I also ran into some reliabillity issues when trying to bond them in OpenFiler where I could access the web interface and then it would randomly become unreachable and then a little bit later come back online and back and forth. Either way, we are seeing a DRASTIC increase in performance on this single nic with openfiler iSCSI LUNs over any other scenario we have had setup.

[vidkun]

Anyone have any ideas on either of these issues?

[cmacmillan]

The two NICs would need to be on different subnets or you will get into problems if one fails. Assuming you do that, look into multipath and choose the "preferred" path for the initiator and you'll get what you're after.

1. You'd need TWO iSCSI IP interfaces (vmkernel) on the ESX side to match the subnets of the SAN.

2. EachLUN will have two discovery addresses, one for each subnet, and identified by a unique LUN.

3. They will show-up only ONCE in the "select disk/lun" chooser when you "add storage" wizard them into your ESX storage pool.

4. Once storage is formatted, select "properties" from the available storage list for the volume you created and click on "manage paths" to select the path you want to "prefer"

As for your other question: you can probably find a lot of advice AGAINST using extents in this way. Essentially, you have two non-redundant iSCSI targets dependent being made to be upon each other. One fails: ALL storage goes bad. This approach will INCREASE your failure chances, not decrease them...

--Collin C. MacMillan

SOLORI - Solution Oriented, LLC

[vidkun]

This sounds like you are talking about setting up both paths on both hosts. Using one path as the preferred and the other as a failsafe. Is this correct? If so, then can the preffered path be set differently for each host? For example, set up both IPs on each host then set the preferred path for Host A to IP A and the preferred path for Host B to IP B? The main goal was simply to breakup the traffic from Host A and B going to the same SAN so that Host A was accessing the SAN on IP A and Host B was accessing it on IP B, but with both of them still using the same shared datastore on the SAN. Also, is there no way to achieve this without the two IP addresses being on separate subnets?

That's a good point about the extents failing.

Thanks for the help!

[cmacmillan · Accepted solution]

Essentially, yes. Make sure the IP addresses are on different subnets for A and B interfaces:

ESX-VMK1=192.168.100.5 talks to SAN-NIC1=192.168.100.254
ESX-VMK2=192.168.200.5 talks to SAN-NIC2=192.168.200.254

Under NO circumstances should you use the same subnet for multiple interfaces...

If you have TWO ESX hosts, you can cross-provision the NICs so HostA "prefers" subnet-1 and HostB "prefers" subnet-2:

ESX-A-VMK1=192.168.100.5 talks to SAN-NIC1=192.168.100.254 (manually preferred in LUN path)
ESX-B-VMK1=192.168.100.6 talks to SAN-NIC1=192.168.100.254

and

ESX-A-VMK2=192.168.200.5 talks to SAN-NIC2=192.168.200.254
ESX-B-VMK2=192.168.200.6 talks to SAN-NIC2=192.168.200.254 (manually preferred in LUN path)

This arrangement gives you "dedicated" bandwidth without losing redundancy in case one NIC fails on either of the three hosts... If subnet-1 becomes unuseable (i.e. your D-Link switch dies) then fail-over to the other switch will happen auto-magically.)

If you found my comments helpful, please award them points. Thanks.

--Collin C. MacMillan

SOLORI - Solution Oriented, LLC

http://blog.solori.net

[vidkun]

Thanks for the help! Now if I could just figure out how to keep LUN Snapshots on a different filer.