zenking
Hot Shot
Hot Shot

All iscsi volumes show up as same size on host

I have 2 hosts with iscsi connections to openfiler san. On san, I have 3 volumes of different sizes (6, 8 and 15 GB). Both hosts see all 3 volumes. However, under storage adapter details for the iscsi software adapter:

  • Host 1 sees 3 volumes with correct names, but all show up as 6 GB

  • Host 2 sees 3 volumes with correct names, but all show up as 8 GB

I can't migrate a vm from one host to the other, and this odd size display is probably a clue. When I try to migrate with vmotion, I get the "virtual disk 'hard disk 1' is not accessible on the host" error. What have I misconfigured?:) Thanks.

VMWare Environment: vSphere 6.7, EQ PS6210 SANs, Dell R730 Hosts, dedicated Dell switches w/ separate vlans for vmotion and iscsi.
0 Kudos
13 Replies
JaySMX
Hot Shot
Hot Shot

In your ESX iSCSI setup, are you using static discovery or send targets? Not knowing anything about your iSCSI SAN it's difficult to determine if it could be a problem. Have you tried adding a Windows box to the storage group and using the MS iSCSI Initiator to see how the Windows box would emunerate the LUNs? That could help you figure out of the problem is with the SAN or ESX.

The reason you get that error with VMotion is because your VMs reside on local storage at the moment, you cannot VMotion a VM which is on a local VMFS datastore. In order for VMotion to work, the VM must reside on a datastore which both ESX hosts have access to.

-Justin
dingding
Enthusiast
Enthusiast

please double-check your iscsi array configuration, can you provide the snapshot in VCMS|configuration|storage adapter, or the three lun name just like vmhba0:0:2

---- Idleness is not doing nothing. Idleness is being free to do anything.
zenking
Hot Shot
Hot Shot

Thanks for the reply. I'm using the send targets under the dynamic discovery tab in the iscsi storage adapter section. I have 1 iqn and 3 targets showing up. Each target is pointing to a separate volume on the openfiler san. The 3 volumes are all on one partition on the san.

I know vmotion requires shared storage, so that's why I set up the san. I used vmware converter to move VMs from the local datastore to iscsi. When I moved the VMs, the free space on the iscsi volumes dropped accordingly, but in some sense it seems like esx is still treating it like local storage. The vmotion error is "virtual disk 'hard disk 1' is not accessible on the host", so the destination server can't see the hard disk, like you say because it's actually local. In the VM properties, hard disk 1 is listed as a virtual disk and the disk file section shows server2.vmdk. The virtual device node is SCSI (0:0) Hard Disk 1. So bottom line is some info indicates local disk and other info indicates iscsi connection. Argh!

I've thought about blowing away all VMs and iscsi storage on esx and starting over, but wanted to see if I could fix what I have first.

VMWare Environment: vSphere 6.7, EQ PS6210 SANs, Dell R730 Hosts, dedicated Dell switches w/ separate vlans for vmotion and iscsi.
0 Kudos
JaySMX
Hot Shot
Hot Shot

Are both ESX hosts able to see the same iSCSI LUN during VMotion? In Virtual Center, if you click on each ESX host, on the Summary tab can you browse the same datastore and see the VM's files?

If it does come down to it, you could use the VMWare Converter to clone your VMs to a datastore you know both can see.

-Justin
0 Kudos
zenking
Hot Shot
Hot Shot

DD,

Sure, thanks. All 3 iscsi connections under one iqn. The target names are iqn.2006-01.com.openfiler:iscsi_1.### where ### is the san volume name. Both hosts see the same targets, but not the actual sizes. That's the weirdest part. Host 1 reports all 3 targets as 6 GB and host 2 reports all 3 as 8 GB. When I created the volumes on the san, I made the first one 6 GB, the second one 8, and the 3rd one 15 GB. So esx was able to see the targets and grab the names, but not the actual sizes. Frack!

VMWare Environment: vSphere 6.7, EQ PS6210 SANs, Dell R730 Hosts, dedicated Dell switches w/ separate vlans for vmotion and iscsi.
0 Kudos
zenking
Hot Shot
Hot Shot

Jay,

I didn't realize you could browse the datastores in VI, so thanks for that. When I browse my iscsi datastore in each host, I can only see the files for the VM guest on that host. I can't see files for any VMs on the other host.

VMWare Environment: vSphere 6.7, EQ PS6210 SANs, Dell R730 Hosts, dedicated Dell switches w/ separate vlans for vmotion and iscsi.
0 Kudos
JaySMX
Hot Shot
Hot Shot

If you go into the Configuration tab, Storage, and click Add Storage..., click Next, do you see any other LUNs available? If not, it would seem that the hosts are not all being presented the same LUNs by the storage device. Did you configure any sort of storage groups or LUN masking on the SAN?

You also mention "volumes" on the SAN instead of LUNs. Did you create one LUN with multiple volumes or three separate LUNs?

-Justin
0 Kudos
zenking
Hot Shot
Hot Shot

When I try to add storage, I only see the local datastore and one iscsi datastore. In openfiler, I have one iscsi partition that I divided into 3 volumes (openfiler term), so does that sound like one LUN with multiple volumes? ESX shows me all 3 volume names under the storage adapter iscsi section, but appears to have access to only one volume, so that probably explains why they all show up as the same size on either host. I'm starting to think I need to blow away all VMs and iscsi storage on ESX and all volumes on openfiler and create one volume. But it still seems like both hosts should be able to see at least one of the current volumes.

VMWare Environment: vSphere 6.7, EQ PS6210 SANs, Dell R730 Hosts, dedicated Dell switches w/ separate vlans for vmotion and iscsi.
0 Kudos
zenking
Hot Shot
Hot Shot

DD,

In my earlier response, I forgot to mention that under the iscsi storage adapter info in ESX, the 3 targets are:

SCSI Target 0 - Path is vmhba37:0:0, Canonical path is vmhba37:0:0

SCSI Target 1 - Path is vmhba37:1:0, Canonical path is vmhba37:0:0

SCSI Target 2 - Path is vmhba37:2:0, Canonical path is vmhba37:0:0

So I guess all I'm really seeing is the canonical path.

VMWare Environment: vSphere 6.7, EQ PS6210 SANs, Dell R730 Hosts, dedicated Dell switches w/ separate vlans for vmotion and iscsi.
0 Kudos
zenking
Hot Shot
Hot Shot

Good news! I whacked all my VMs and the 2 smallest iscsi targets from esx and openfiler. After resetting the iscsi service on the hosts, I was able to create a datastore that can be accessed by multiple hosts. I'm creating a new VM from a template , so I'll know more when I actually get a successful migration. Will post results. Thanks for the input.

VMWare Environment: vSphere 6.7, EQ PS6210 SANs, Dell R730 Hosts, dedicated Dell switches w/ separate vlans for vmotion and iscsi.
0 Kudos
JaySMX
Hot Shot
Hot Shot

Interesting. I've been looking at info on that product but haven't found much about creating multiple LUNs (or volumes I guess) for ESX servers to access simultaneously. Let us know what happens if/when you create more volumes and present them to the hosts.

-Justin
0 Kudos
zenking
Hot Shot
Hot Shot

I just successfully migrated a vm between hosts. :smileycool: I originally set up multiple volumes because I wanted to keep VMs separate, but that's probably a flawed strategy since it limits flexibility. I might try to set up another volume for another VM again just to see if it works, but I need to catch up on drs, ha, etc.

I will definitely be setting up a separate shared volume for my MS server guests on a cluster, but that's a whole different animal.

Thanks again for the help.

TR

VMWare Environment: vSphere 6.7, EQ PS6210 SANs, Dell R730 Hosts, dedicated Dell switches w/ separate vlans for vmotion and iscsi.
0 Kudos
dingding
Enthusiast
Enthusiast

it seems the three target are the same lun from diffrent path, so it must be the problem of your iscsi array!

---- Idleness is not doing nothing. Idleness is being free to do anything.
0 Kudos