ESX 3.0.2 Build 61618 - iSCSI Software Network Setup

[scotte]

Hi,

I have read various forums on this subject and cannot get my configuration to work.

Hope someone can help please.

2 x ESX Hosts

Trying to setup iSCSI on the first host

My Cisco 3750G is configured as follows

VLAN Name Status Ports

---- -

---

-

---

-

---

1 default active Gi1/0/1, Gi1/0/2, Gi1/0/8

• Gi1/0/11, Gi1/0/12, Gi1/0/16*

• Gi1/0/20, Gi1/0/21, Gi1/0/22*

• Gi1/0/23, Gi1/0/24, Gi1/0/25*

• Gi1/0/26, Gi1/0/27, Gi1/0/28*

2 vmotion active Gi1/0/3, Gi1/0/4

3 iscsi active Gi1/0/5, Gi1/0/6, Gi1/0/7

4 management active Gi1/0/9, Gi1/0/10

10 vmw_vlan_10 active

20 vmw_vlan_20 active

30 vmw_vlan_30 active

40 vmw_vlan_40 active

1002 fddi-default act/unsup

1003 token-ring-default act/unsup

1004 fddinet-default act/unsup

1005 trnet-default act/unsup

----

---

GARGRAVE-SW1>sh ip int br

Interface IP-Address OK? Method Status Protocol

Vlan1 192.168.0.10 YES NVRAM up up

Vlan3 192.168.2.1 YES NVRAM up up

Vlan4 192.168.3.1 YES NVRAM up up

Vlan10 192.168.10.1 YES NVRAM up up

Vlan20 192.168.20.1 YES NVRAM up up

Vlan30 192.168.30.1 YES NVRAM up up

Vlan40 192.168.40.1 YES NVRAM up up

GigabitEthernet1/0/1 unassigned YES unset up up

GigabitEthernet1/0/2 unassigned YES unset up up

GigabitEthernet1/0/3 unassigned YES unset up up

GigabitEthernet1/0/4 unassigned YES unset up up

GigabitEthernet1/0/5 unassigned YES unset up up

GigabitEthernet1/0/6 unassigned YES unset up up

GigabitEthernet1/0/7 unassigned YES unset up up

GigabitEthernet1/0/8 unassigned YES unset down down

GigabitEthernet1/0/9 unassigned YES unset up up

GigabitEthernet1/0/10 unassigned YES unset up up

GigabitEthernet1/0/11 unassigned YES unset down down

GigabitEthernet1/0/12 unassigned YES unset down down

GigabitEthernet1/0/13 unassigned YES unset up up

GigabitEthernet1/0/14 unassigned YES unset up up

GigabitEthernet1/0/15 unassigned YES unset up up

GigabitEthernet1/0/16 unassigned YES unset down down

GigabitEthernet1/0/17 unassigned YES unset up up

GigabitEthernet1/0/18 unassigned YES unset up up

GigabitEthernet1/0/19 unassigned YES unset up up

GigabitEthernet1/0/20 unassigned YES unset down down

GigabitEthernet1/0/21 unassigned YES unset down down

GigabitEthernet1/0/22 unassigned YES unset down down

GigabitEthernet1/0/23 unassigned YES unset down down

GigabitEthernet1/0/24 unassigned YES unset down down

GigabitEthernet1/0/25 unassigned YES unset down down

GigabitEthernet1/0/26 unassigned YES unset down down

GigabitEthernet1/0/27 unassigned YES unset down down

GigabitEthernet1/0/28 unassigned YES unset down down

Port-channel10 unassigned YES unset up up

Port-channel20 unassigned YES unset up up

GARGRAVE-SW1>

-

---

# esxcfg-vswitch -l

Switch Name Num Ports Used Ports Configured Ports Uplinks

vSwitch0 32 3 32 vmnic4

PortGroup Name Internal ID VLAN ID Used Ports Uplinks

Service Console portgroup4 0 1 vmnic4

Switch Name Num Ports Used Ports Configured Ports Uplinks

vSwitch1 64 3 64 vmnic5

PortGroup Name Internal ID VLAN ID Used Ports Uplinks

VMotion portgroup3 0 1 vmnic5

Switch Name Num Ports Used Ports Configured Ports Uplinks

vSwitch2 64 5 64 vmnic3,vmnic2,vmnic1

PortGroup Name Internal ID VLAN ID Used Ports Uplinks

VLAN10 portgroup5 10 1 vmnic1,vmnic2,vmnic3

Switch Name Num Ports Used Ports Configured Ports Uplinks

vSwitch3 64 5 64 vmnic0

PortGroup Name Internal ID VLAN ID Used Ports Uplinks

VLAN3 portgroup10 3 1 vmnic0

iSCSI Service Consoleportgroup8 3 1 vmnic0

iSCSI VMkernel portgroup9 3 1 vmnic0

-

---

# esxcfg-vswif -l

Name Port Group IP Address Netmask Broadcast Enabled DHCP

vswif0 Service Console 192.168.3.10 255.255.255.0 192.168.3.255 true false

vswif1 iSCSI Service Console192.168.2.10 255.255.255.0 192.168.2.255 true false

#

I have a letfhand networks vm appliance running on this esx host which has an IP address of 192.168.2.12, which is the iSCSI network VLAN3 on my switch.

I can ping 192.168.2.12 from the esx host itself but not from any other network device.

I assume this is a routing problem as the iSCSI Service Console Default Gateway automatically sets itself as default Service Console gateway, which is 192.168.3.1

When I try to change the DG of the iSCSI console to 192.168.2.1 I lose connectivity to the esx host as it changes the DG on the default Service Console.

Can anyone describe in simple terms how to configure this correctly please ?

Thanks

Scott

[MattMeyer]

Can you please post the output of "esxcfg-vmknic -l" and "esxcfg-nics -l" ?

Thanks.

[weinstein5]

Can you ping your iSCSI target form the vmkernel port? Using the vmkping command- which I believe will not wqork since I do not see a vmkernel port configured - you need a vmkernel port configured to access IP based storage - easiest way is to place it on the iSCSI segment -

[scotte]

Hi Matt,

Here is the output

root]# esxcfg-vmknic -l

Port Group IP Address Netmask Broadcast MAC Address MTU Enabled

VMotion 172.16.1.10 255.255.255.0 172.16.1.255 00:50:56:66:4e:62 1514 true

iSCSI VMkernel 192.168.2.11 255.255.255.0 192.168.2.255 00:50:56:6a:a8:f0 1514 true

-

---

# esxcfg-nics -l

Name PCI Driver Link Speed Duplex Description

vmnic0 01:00.00 e1000 Up 1000Mbps Full Intel Corporation 82571EB Gigabit Ethernet Controller

vmnic1 01:00.01 e1000 Up 1000Mbps Full Intel Corporation 82571EB Gigabit Ethernet Controller

vmnic2 03:01.00 e1000 Up 1000Mbps Full Intel Corporation 82546EB Gigabit Ethernet Controller (Copper)

vmnic3 03:01.01 e1000 Up 1000Mbps Full Intel Corporation 82546EB Gigabit Ethernet Controller (Copper)

vmnic4 04:00.00 tg3 Up 1000Mbps Full Broadcom Corporation NetXtreme BCM5721 Gigabit Ethernet

vmnic5 05:00.00 tg3 Up 1000Mbps Full Broadcom Corporation NetXtreme BCM5721 Gigabit Ethernet

#

#

[scotte]

Hi Weinstein5,

I have 2 vmkernel ports configured.

1 is configured on 1 vswitch with vmotion enabled to isolate vmotion traffic the other is configured on another vswitch which also has an iSCSI port group & VM Network portgroup for Vlan 3 which has the 192.168.2.x network addressing scheme configured on it.

I have uploaded a screenshot of my networking config.

Thanks for your prompt assistance guys.

[MattMeyer]

When you use "ping" it's using the service console connection. But if you "vmkping" it will use the vmkernel connection. From what I can tell though, the routing should not be any kind of a problem since you have a SC connection and a vmkernel connection on the local subnet as the VSA.

What happens when you use "vmkping" to ping the VSA?

[MattMeyer]

I just read your original post. Are you trying to connect to the VSA from outside the 192.168.2.x subnet? If you are, it's not the vmkernel default gateway that needs set. It would be the VSA's.

[scotte]

Hi Matt,

When I use vmkping I get a reply from the VSA.

The problem is trying to connect to the VSA outside of the 192.168.2.x network

i.e. my mangement machine is on 192.168.0.4 and I cannot ping the VSA from here

VSA:

IP: 192.168.2.12

SM: 255.255.255.0

DG: 192.168.2.1

I have tried changing the gatewway of the vsa to the IP address of the iSCSI Vmkernel & iSCSI SC and I still cannot ping the vsa ip address 192.168.2.12

Any ideas Matt ?

Thanks

Scott

[MattMeyer]

I'm not famaliar enough with Lefthand products to give a definitive answer,but I'm pretty sure a VSA is nothing but a regular VM acting as a software iSCSI target. Is it possible there is a firewall service running on the VSA that allows ping from the local subnet only and blocks everything beyond?

[Texiwill]

Hello,

Unless VSA has 2 network addresses or you use a gateway to go from 192.168.0.0 network to the 192.168.2.0 network then your management node will not be able to reach the VSA node. In order for iSCSI to work the SC and the iSCSI vmkernel device needs to be able to reach the iSCSI Server (in this case VSA). So does 'ping VSANodeByIP' and 'vmkping VSANodeByIP' both work?

If so then iSCSI should work. However if you need to configure VSA then your management node must also be able to reach the 192.168.2.0 network, you may need a virtual gateway to make this happen if you can not make this happen within your physical network or VSA can not have 2 IP addresses, etc.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[scotte]

vmkping works fine.

I can ping other devices on the 192.168.2.x network from my management node.

It's just the VSA that I cannot ping.

I have added persistent static routes on my management node.

I am becoming puzzled by this.

As you say I am sure iSCSI will work but I cannot manage VSA to configure the iSCSI target.

Not sure what to do next other than put the VSA on the 192.168.0.x network or put my management node on the 192.168.2.x network.

Any other ideas guys ?

Thanks for your input so far.

[weinstein5]

Can you ping other devices on the 192.168.2.x network other than the VSA? if not perhaps there is a problem with the default gateway on the 192.168.2.x network - could something on the VSA be blocking the icmp packet like a firewall -

[Texiwill]

Hello,

I believe you should contact Lefthand NEtworks support for further help on this. It really appears to be their appliance/management node....

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[scotte]

There is no firewall on the VSA.

I have used VSA successfully on a flat network before.

I can ping other devices on the 192.168.2.x network and I can ping the DG 192.168.2.1 from anywhere on my network so I know my network/switch/rotuing is configured correctly.

This has to be something to do with the iSCSI SC DG being 192.168.3.1, which is being inherited from the first/main SC. I would aqssume the routing will work if the DG was 192.168.2.1.

However, as I said before changing this disconnects the host as the first/main SC DG is changed to 192.168.2.1.

I suppose I will just have to keep testing trying other things.

Thanks

[MattMeyer]

Assuming the management machine DG is 192.168.0.1, can you ping it from the VSA? From what I can tell, the vmkernel DG and the COS gateway have no effect in this equation. There is a routing problem from the VSA to an outside network. This point me to the network config in the VSA, or an external routing problem in the switch. I would start with double-checking the physical connections, then the port VLAN config on the switch. It seems like the switch is not routing between 192.168.2.0 <-> 192.168.0.0. If it is routing according to the config, then check the physical connections to make sure the cables are plugged into the right ports that operate in that correct VLAN.