How do I delete a volume on an EqualLogic SAN without generating alerts

[Box293]

We recently installed an EqualLogic PS6010XV in our environment.

I went to delete a volume the other day and it didn't matter what way I went about it the SAN sent multiple error emails about the ESXi hosts not being able to contact the target. This generally occurs when I do a rescan of the storage adapter. And this occurs for every ESXi host that is connected to the volume. The emails only occur once.

ERROR event from storage array EQLSAN01

subsystem: MgmtExec

    event: 7.4.3

     time: Fri Feb 25 06:35:31 2011

iSCSI login to target '192.168.102.30:3260, iqn.2001-05.com.equallogic:0-8a0906-d74e39c07-78700184cdc4d587-vc-238-99' from initiator '192.168.102.91:2291, iqn.1998-01.com.vmware:localhost:268107216:34' failed for the following reason:

                Requested target not found.

I logged a support request with EqualLogic about this, their response was:

Before deleting the volume on the array you need to disconnect from this volume on all hosts that are configured to access it.

I responded:

My current thinking is that I need to remove the static target from each iSCSI initiator on each ESXi host (these hosts use Dynamic Discovery).

We are using hardware offloading iSCSI initiators so there are two of these iSCSIinitiators per ESXi host and we have 5 ESXi hosts.

However I feel that by the time I do these steps on all 5 ESXi hosts the dynamic discovery will redetect the disconnected target and add it in as a static target again (even with scripting).

It feels like a never ending loop.

Extract taken from: http://www.vmware.com/pdf/vsphere4/r40/vsp_40_iscsi_san_cfg.pdf

Dynamic Discovery Also known as Send Targets discovery. Each time the initiator contacts a

specified iSCSIserver, the initiator sends the Send Targets request to the server.

The server responds by supplying a list of available targets to the initiator. The

names and IP addresses of these targets appear on the Static Discovery tab. If

you remove a static target added by dynamic discovery, the target might be

returned to the list the next time a rescan happens, the HBA is reset, or the host

is rebooted.

Their response to this was:

In this case it needs to be done this way:

• Under group configuration -> notifications -> email notifications -> clear check mark for errors
  
  
• Put volume offline -> delete it
  
  
• Disconnect from this volume on all hosts
  
  
• Under group configuration -> notifications -> email notifications -> place back check mark for errors
  
  

And this is where I am at. Disabling email notifications before deleting a volume and then re-enabling notifications afterwards seems very clunky. We've never had problems like this before with our existing EMC SAN (FC).

How do other EqualLogic users delete their volumes without receving email alerts?

[FROGGYJ]

I typically delete the volume from the ESX servers and then delete it from the Equallogic SAN. I've never had an issue with a host with either an an HBA or onboard NIC rescanning too quickly before I have deleted it from the Equallogic box. Worst case do 1 host at a time I guess, but I'm sure you can do 5 hosts quick enough...I've done 4 hosts all day long.

[Box293]

Thats how I did it the first time.

When I deleted the volume on the ESXi host first (through vCenter) it then automatically created a rescan task for all the other ESXi hosts that had this volume. However the target still exists in the iSCSI initiator, it's just an unformatted volume at this stage.

Does your EQL SAN have email alerts enabled (I know it's a obvious question)?

[FROGGYJ]

Yes I have email alerts setup...here are the exact steps I do which I just tested again.

-delete the volume from vcenter > host config > storage

-vcenter auto rescans other hosts VMFS datastores and removes the volume (still shows an an iSCSI target on the initiator)

-go to equallogic box set volume offline and then delete

-go back to vcenter and re-scan initiators on all hosts (removes the iSCSI target from the initiator)

doing it this way I never receive any email alerts.

[FROGGYJ]

haha I take that back....I just got an alert. Hrm guess you can't beat it.....maybe your only option is to turn off notifications temporarily or just delete the alert email you receive.

[Box293]

LOL

Thanks for quickly testing.

[tjcarst]

I have the same problem.  Did you ever get the email alerts to stop?

[Box293]

Not at this point in time, I need to disable the alerts on the EQL Group and SANHQ.

However when I was at VMWorld 2011 I spoke to Dell and they informed me that there is a plugin coming out for vCenter that will allow you to delete volumes from within vCenter, you won't need to log into the Group manager. I would assume that this would circumvent the problem.

There is also KB 1015084 that has a step by step guide on how to remove a VMFS volume, however I had a read of it and it seems overally complex.

One little tip if you decide to disable the alerts, immediately create a calendar reminder to re-enable the alerts. I shudder to think about missing actual important alerts all because I forgot to re-enable them.

[NTC99]

Hello,

I always have this issue. I was wondering if this issue has been fixed. Where can I download the plugin for vCenter?

Thanks,

[Box293]

I have not re-vistited this for some time now, I've just put up with it.

The plugin should be available from the EqualLogic support website.

I'll be upgrading to vSphere 5 soon so I'll be testing this plugin (if available) and other EqualLogic ones as I'll have some time for test and dev before putting it into production. If I find anything out I'll report here.

[FROGGYJ]

The Dell vCenter plugin is a purchased product that you purchase based on the number of physical hosts you want to manage. I am using it now in production with vSphere 5 if anyone was curious. You can find some basic info here.

http://www.dell.com/us/enterprise/p/d/business~solutions~virtualization~en/Documents~dell-management...

[Box293]

Cheers, I'll be testing that one as well.

However the plugin I am talking about here is an EqualLogic specific one which is different to the one you are talking about.

[NTC99]

Thanks for reply it so quickly. I will try to find the plugin download at the EqualLogic site.

Thanks!

[NTC99]

Just an update on this issue.  I have upgraded to vSphere5 update1, and the same alerts are still generated.  I have heard that vSphere5 will fix the issue, but it's not for me.  I will be at the VMWorld 2012 in year, and I will address this issue to both EqualLogic and VMWare.  Thanks!

[Box293]

I'm on the same page as you, I'm also using vSphere 5 Update 1.

I did happen to try out the Host Integration Tools for VMware (HIT/VE). This is an EqualLogic virtual appliance that communicates with the EqualLogic SANs and vCenter. However it is only aimed at small deployments for the following reasons:

• HIT/VE can only talk to a single instance of vCenter
  
• HIT/VE can only talk to a single PS series group
  

The ability to only talk to a single PS series group is a real show stopper for me. I have multiple groups and I don't want to have to deploy multiple vCenter servers just to get around this stupid limitation. It's a pity because the HIT/VE shows a lot of promise and the possibility to fix our issue (there is a Datastore Manager component in HIT/VE).

I did try implementing a HIT/VE appliances for each PS group however when the second appliance registers with vCenter the first appliance is de-registered. Doh!

I too will be at VMworld this year so I'll also be addressing this issue with Dell (I was on their case about it last year as well).

I don't see it being a VMware issue, our older SAN is an EMC SAN and it never has this issue when deleting luns.

[Box293]

I had a chat to Dell at VMworld about HIT/VE and they said they have a new version coming out shortly that will allow it to talk to multiple EQL groups.

[dwilliam62]

If you are not using the HIT kit for VMware, then the procedure to delete an EQL volume w/o generating "no such target/volume" errors is as follows:

1.)  In the ESX GUI, delete the datastore.

2.)  In the EQL GUI, set that volume OFFLINE

3.)  Rescan all the storage adapter on all servers.  (or Right mouse over cluster icon and select Rescan)

4.)  In the EQL GUI, you can now delete the volume.

FYI:  The HIT kit for VMware has been renamed to Virtual Storage Manager (VSM) and v3.5 is available on the Equallogic support website.  It now supports multiple groups.

Regards,

Don

[DeleteUser]

Probably solved by now, but for the googlers out there, I found that I had to additionally delete the iSCSI target in the Static Discovery tab under Storage Adapters - {iSCSI software Adapter} -Properties -> Static Discovery tab - Remove target with EqualLogic volume in question.