Add iSCSI to existing Cluster

[Brades511]

I have a VSphere v6.7 cluster running on VxRails.  It has 4 Nodes.  Each node has 2 physical 25 Gb NIC's.  Our existing cluster has some storage restrictions, and limitations of VMware do not allow shrinking the provisioned storage of a VHDX file without being able to move it to another storage.  A simple solution seems to be to add cheap lower cost iSCSI storage to this cluster to allow moving a VM and them moving it back once it is shrunk down.  

My question is how do I create the proper iSCSI networking elements.  Each host only has 2 physical NIC's and they are all in use for all the VM's running in the cluster.  All the documents I've read on how to create the iSCSI components keep ending up at the VMKernel step which seems to be an issue.  Can iSCSI user the same 2 physical NIC's with Virtual Switches?

https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.vsphere.storage.doc/GUID-28C3CFF8-AE86-413F...

This guide states to create a VM Kernel adapter, which on step 5 states to select network adapters.  All our adapters are in use, so I can't do this.  Can iSCSI share the NICs like all other network areas using Virtual Switches or Dynamic Switches?  I'm kind of out of my element here on iSCSI side on the VxRail cluster.  I have a physical server already setup to be the Target iSCSI server with TB's of capacity.  The main issue is now, how do I setup the iSCSI elements on my cluster that is already in use without needing additional physical Network hardware?  I assumed given the fact I have 50 GB of networking on each host, that I would be able to share part of that to do some minimal iSCSI work.  

Any help would be appreciated.  

[scott28tt]

VLANs?

[IRIX201110141]

It always depends on what kind of iSCSI Storage you have. There is no one config which is right for all because it depends on the need for Multitarget, Single Subnet which ends up for swISCSI Binding or not.

Regards,
Joerg

[Brades511]

Yes, multiple VLAN's are configured in Distributed Switches.  Main issue is creating the VMKernel for iSCSI.  All documentation seems to require free NIC's, which seems odd given that I just want Software iSCSI and should have plenty of bandwidth for temp iSCSI moves and connections.  

[Brades511]

I can put the iSCSI storage target server on any VLAN, so that isn't the concern.  

[IRIX201110141]

Its not a matter of VLAN or not. For sure they are Best practices and add a layer of security.

We also have one pair of NICs for vSAN, LAN and iSCSi (CMPL, EQL)  because we are low on 10/25G Ports. Its the used iSCSI SAN which dictates how to setup iSCSI on ESXi.

Regards,
Joerg

[Brades511]

Joerg, I appreciate the response, but that doesn't help.  I will need more info to understand what you are saying.  You stated you have 2 NIC's, and use them for vSAN, LAN, and iSCSI.  How did you setup and configure you iSCSI to get it to work?  Sounds like you have an environment similar to what we have.  Based on what I described, is there a way to setup iSCSI to get my external iSCSI server storage setup?  I merely want iSCSI for doing data moves to shrink over provisioned storage since VMware lacks the ability to shrink an existing VHDX drive without moving it to another storage location.  

My question, given my hardware and environment, is how can I create the Software iSCSI setup.  The steps I've found all imply a VMKernel for iSCSI is needed so I can finish setting Software iSCSI storage.  Creating the VMKernel is not possible as it seems to want physical hardware.  Hence the question, can I use Dynamic Switches for the network portion?  To my understanding, as long as vSphere can see the Target iSCSI device is all that matters to the VMWare cluster. 

What the vendor is, or type of hardware the iSCSI device is behind it shouldn't matter.  iSCSI is pretty generic and universal.  If I have hardware, regardless of what it is, if it can support being made an iSCSI device or present itself as iSCSI to other devices, that should be all.  If vSphere can see the target, by IP or Hostname, then that should be all that is really considered.  Speed, etc. will all have impacts on performance, but those aren't related to my question or concerns. 

Is it possible to setup a Software iSCSI adapter, which requires a VMKernel, for iSCSI using Dynamic Switches or some other shared network element to complete the virtual configuration for Software iSCSI adapter, as all Physical NIC's are already defined and used within the cluster?  

[IRIX201110141]

There is no Dynamic Switch it called "Distributed" aka vDS. For sure you can create a VMkernel on a vDS or simply reuse one of your existing ones if this in the same Subnet as your iSCSI Storage and matches the requirement for simply iSCSI, but if possible create a new one and use a new IP subnet.

A VMKernel never points directly to a phys. NIC because it sits on a vSwitch and this one offer the Uplinks. Since you only have 2 Uplinks in your setup all have to be on the already existing vDS and the two 25G Uplinks.

Yes you can enable the swISCSI module as a storage adapter and leverage the existing vDS and is 2 Uplinks.

1.  Create one VMKernel and let it use only one vmnic in your Failover rule
2. Try to  "vmkping <IP_OF_STORAGE>" to test simply connectivity to the Storage
3. Configure the Storage with a Proper ACL(IP,CHAP,IQN) or what ever it takes to give permissions to let a Host connect to a LUN
4. Enable the swiSCSI on ESXi and add your Storage under Dynamic Discovery and perform a rescan. If you see a device you can create a VMFS Datastore on it

Yes, we only have one vDS for all the networking stuff and yes this vDS use both two of our vmnic's. A external iSCSI Storage is connected together with a NAS trough NFS and the vSAN of course.

Regards,
Joerg

PS: Just for history and ALL other people it will come down to the answer of configuring swISCSI Binding or not which depends on the used iSCSI SAN. Also swISCSI Binding is not supported if you use LACP on your one and only Uplinks. For sure VLANs, Jumbo Frames, Special iSCSI Settings (LoginTimeout,DelayedACk and so on), MultiPathing are part of the game.