iSCSI on ESX 3.5

[soonblue]

Afternoon all,

I'm pretty new here, but I've been banging my head against the wall regarding the following problem for a few days now. First, some background info:

I work for the University of Delaware in Newark, Delaware as a systems programmer/administrator. We've recently decided to virtualize some of our servers. We've got a mix of operating systems: Windows, Linux, Solaris, AIX etc. To start our production VMware cluster we bought three Sun Microsystems 4140s and one of the new Sun Microsystems 7110 storage devices. We also have a second environment being run by another department that I'm administering the storage device for, another Sun 7110.

I successfully managed to get two of our machines hooked up to the Sun 7110 storage device through the software iSCSI Initiator provided in ESX 3.5. As per ESX's recommendation, I rebooted both VMware machines. After they came back up, I did a rescan of my storage adapters to make sure I could still see the LUNs on the 7110. Suddenly, no LUNs were available. I scaled down the configuration on both the VMware side and the 7110 side(removed CHAP, any network interface teaming etc), and tried again. Still nothing. Dropped into the Linux side of things via SSH and did a little more digging through the logs. First saw:

1. tail -f /var/log/messages

Mar 17 11:32:43 vs2 vmkiscsid[1725]: iSCSI poll session ioctl failed for iqn.1986-03.com.sun:02:7743756e-0b3a-407d-ef67-8bfcdc7f3905, rc 0

Mar 17 11:32:43 vs2 vmkiscsid[1727]: iSCSI poll session ioctl failed for iqn.1986-03.com.sun:02:11cb99de-fdc2-443e-f63b-dd4782fc0ea9, rc 0

This was a bit vague so I killed the vmkiscsid process and re-started it with higher log level:

1. /usr/sbin/vmkiscsid -d 10 &

1237316027.242752 >> iscsid[2969]: iSCSI poll session ioctl for iqn.1986-03.com.sun:02:7743756e-0b3a-407d-ef67-8bfcdc7f3905 returned Device or resource busy

1237316027.262742 >> iscsid[2970]: iSCSI poll session ioctl failed for iqn.1986-03.com.sun:02:11cb99de-fdc2-443e-f63b-dd4782fc0ea9, rc 0

1237316027.262905 >> iscsid[2965]: child 2970 exited with status 100

1237316027.262909 >> iscsid[2965]: session process 0x87f12d0 with pid 2970 exited, no retry

1237316027.262913 >> iscsid[2965]: main daemon polling 0 pipes

If anyone could shed any light on this I'd be extremely grateful. We're trailblazing a bit at UD with this environment as we're apparently one of the first groups to use the new Sun Fishworks storage with VMware.

Thank you!

Fraser

blue@udel.edu

University of Delaware

IT Network & Systems Services

Systems Programmer III

[ericsl]

So, Fraser, can you connect to those iSCSI targets from any other initiators?

Eric

[mike_laspina]

Hi,

How are you connecting the s7110 to ESX 1 path 2 paths, shared network switch, separated etc?

Did you assign a portal group to the s7110 interfaces?

vExpert 2009

[soonblue]

Yes, a Linux iSCSI initiator and the Windows iSCSI initiator can both connect.

[soonblue]

The VMware servers are connected to the 7110 through a gigabit switch. Because of the cost of these switches, we've only bought one for testing, but to simulate two switches I've separated the paths from the two servers to the interfaces on the 7110 using two different VLANs.

Not sure what you mean by portal group...

Fraser

[mike_laspina]

Make sure you enable the multipath checkbox for interfaces on the fishworks gui.

Also do you have a Service Console defined for both subnets?

vExpert 2009

[soonblue]

Multipath is enabled and I have a service console defined for both subnets.

Fraser

[mike_laspina]

ioctl indicates a network connection failure.

Use vmkping and ping from the ESX server SC's to find whats not established.

vExpert 2009