iSCSI vmk vs Service Console vmk?

[ekisner]

I'm trying to add an old iSCSI SAN that I have, which was previously no longer in production... a dev environment.

I have it added to 4 ESX 4.1 hosts, and I'm now trying to add it to an ESXi 5 host.

I've added an iSCSI VMK, configured the teaming per the documentation (the storage adapter doesn't complain when I add discovery targets).

The iSCSI has a separate VLAN (and ofc network) from the management network.  My management runs on vlan 2, 10.0.2.0/24, iSCSI on vlan 99, 10.0.99.0/24.  Default gateways are 10.0.*.254 (star being network specific).

I cannot reach the iSCSI VMK, nor can it reach the SAN.  The reason is obvious, the VMK has an IP in 10.0.99.0/24, and it thinks its default gateway is 10.0.2.254... never going to get out like that.

But here's the catch.  When I try and change the iSCSI VMK's default gateway, it ALSO automagically changes my service console default gateway to the same value - the only way I can recover is to walk over, and locally change the management gateway back to the desired value (since I have no PCs with access to that subnet).

I must either be doing something really dumb, or I must really be sucking at my google searches today, because I cannot see what I'm doing wrong and I cannot find any mention of this online.

The steps I follow, in case it matters:

1) Config > Networking > vSwitch0 properties

2) Select iSCSI VMK and Edit

3) IP Settings >  Edit VMK DG

4) Apply correct DG, and encounter problem immediately after applying

Any thoughts?

[a_p_]

You can only have a single gateway on a host. For iSCSI you have to configure the vmk network in the same subnet as the storage system to avoid the need of a gateway/routing.

André

[ekisner]

Perhaps I'm misunderstanding what you mean, but are you saying that my management and iscsi need to be on the same subnet?

Since that sounds like a strange thing to do, I'm going to assume you mean that the iSCSI VMK should be on the same subnet as the SAN (that is, have the SAN in the 10.0.99.0/24 subnet).  To which I can only answer that it already is.  Since this is my first attempt at setting up iSCSI on an ESXi5 host, perhaps I should troubleshoot further into why it's not able to reach the SAN (which shows no connection attempts by the host).

I would be concerned, however, about the 'Single Gateway' you mention... in ESX4.1 for example, I can set each VMK up to it's own default gateway - why would they remove that capability?  My network has always been configured such that there is a separate VMK for each function (fault tolerance, iSCSI, SC, vMotion), with each function located in its own subnet for nice clean lines of communication.  Is that no longer a possibility in 5?

[rickardnobel]

ekisner wrote:

My network has always been configured such that there is a separate VMK for each function (fault tolerance, iSCSI, SC, vMotion), with each function located in its own subnet for nice clean lines of communication.  Is that no longer a possibility in 5?

It is certainly possible in ESXi 5 as well. The point is that all of the different vmkernel functions you mention should be local subnet only, with the exception for the Management vmknic. This means that there typically is not an "issue" with the single default gateway, because the other features does not need communications outside their specific networks - which should be separated in different LANs / VLANs and different IP networks.

[ekisner]

That is very comforting to hear.

Since I plan on running iSCSI at a DR site(connected by fibre), and the alternate building iSCSI will be in a different subnet, I will at some point need to be able to set up a proper default gateway for my iSCSI VMK... perhaps I'll poke around with an SSH session and see what I can accomplish from there.  It'd be nice to be able to access the DR storage from the production environment.

[ekisner]

Well, I got my iSCSI on the local subnet working, but I didn't really make any ground changing the default gateway for an individual VMK.

Ah well, I'll cross that bridge when I get to it.