iSCSI Multipathing - does each path refer to a particular target IP address?

[pinkerton]

Dear Experts,

I've just configured one ESXi 6.0 host and a Synology ds2015xs NAS for MPIO/Multipathing. On the ESXi side I created two VMkernel adapters with a dedicated physical NICs each and all other NICs set to unused. I then added the adapters to the iSCSI software adapter in ESXi. On the NAS side I configured two interfaces within the same subnet as the VMkernel adapters. I then created a target that allows multiple iSCSI sessions, added LUNs and allowed access to the target for the IQN of the iSCSI software adapter of ESXi.

Back on the ESXi side I added a static discovery entry by manually adding one of the NAS devices IP addresses and the target IQN. After rescanning the storage, ESXi showed two paths to the NAS interface for which I entered the IP address as static discovery entry:

With this configuration, multipathing does not work - once I shut down the interface on the NAS, ESXi does not automatically fail over to the second interface of the NAS. After adding a second static discovery entry with specifing the same target but the second IP address of the NAS, ESXi showed all four paths - two to each interface on the NAS:

With this configuration, multipathing works as expected. If one interface on the NAS goes down, ESXi automatically uses a path to the second interface.

Actually, this seems logical. At first I was wondering about whether it is correct that I have to enter each interface of the target manually to get the corresponding paths to it. First I thought that I only need to enter one IP address of the target and that MPIO uses some kind of magic to automatically discover the paths to other available interfaces on the NAS. Later I've seen that this is the case when using dynamic discovery instead of static discovery. However, even dynamic discovery adds all interfaces of the NAS as static discovery entries. Once I remove one of those entries from the list, the corresponding paths are removed as well.

So is my understanding correct that the path information always refers to the corresponding IP address of the target and that a path will stop working once its corresponding IP address is not available anymore?

Thanks,

Michael

[a_p_ · Accepted solution]

When configuring storage access, it's always important to follow the vendor's recommendations. Not only for how to setup the IP configuration, but also for the Path Policy (e.g. Fixed, MRU, Round-Robin).

Regarding your question, a lot of storage arrays are configured with an additional VIP (virtual IP address), or group IP address, which are then used to configure dynamic targets on the ESXi (or other) hosts. When an initiator connects to such a VIP or group IP, the target will respond with all of its associated IP addresses. If this is not an option for your storage device, you need to provide all target IP addresses manually (as you did).

André

[pinkerton]

Hi André,

thanks, I guess this means that in order to use an arrays IP address for an iSCSI connection / addtional path it must be included in the static discovery list as a separate iSCSI server entry - either by adding it manually or through dynamic discovery. If a given IP address it not known to ESXi it will not be used for iSCSI connections.

BR Michael

[a_p_]

Yes, ESXi (as well as other operating systems) will only connect to known targets.

André

[pinkerton]

Thanks André.  I have one additional question - do you know how to find out which "Controller" (that is part of the runtime name on the iSCSI adapter) belongs to which VMkernel adapter/NIC:

I know that the values T and L are refering to the Target/LUN respectively but don't know what the controller information (C) is related to.

BR Michael

[a_p_]

Interesting question, but I can't answer it at the moment. Btw. the "C" refers to a Channel rather than to a Controller.

André

[PePierias]

@pinkertonI'm really thankful for your thread since it's resolved a couple of notions about setting up multipathing. Since you also have a Synology model, I'd be grateful if you could provide some info on the way things are set up from the NAS side:

1) I understand that you've allocated two interfaces from the Synology side for storage. Can I presume that you've configured the interfaces separately (ie without any link aggregation like active/failover or adaptive load balancing)?

2) There are various options on configuring the storage on the Synology: btrfs/ext4 format, thin or thick provisioning, jumbo frames (off or specific value). Could you suggest which one(s) you used on your setup?

3) Finally, assuming that multipathing is active, do you see performance improvements when uploading a single large file? How would you go about benchmarking this thing?

Thanks in advance for any information provided!

M.-