VMware Cloud Community
seijirou
Contributor
Contributor

[Bug] Unable to remove legacy static software iSCSI target

EDIT:

I am now thinking that this is a bug.  When ESXi is installed to an iSCSI target, the static iSCSI target is created automatically.  Even after the initial iSCSI target is no longer in service (as a new iSCSI target is created connecting to the same LUN), the initial iSCSI target still cannot be removed.

What I do:

From the vSphere client;

Configuration -> Storage Adapters -> iSCSI Software adapter vmhba38 -> Properties -> Static Discovery -> Highlight the iSCSI Server Location that I want to remove and click Remove.

I receive the following error.

Call "HostStorageSystem.RemoveInternetScsiStaticTargets" for object "storageSystem" on ESXi "192.168.2.10" failed.

Operation failed, diagnostics report: iScsiLibException: status(80000101): SCSI Busy; Message= IMA_RemoveStaticDiscoveryTarget

What I am trying to accomplish:

Originally I had a single connection between my hypervisor and my LAN (192.168.2.0/24) that all traffic flowed over.  All of my virtual machines live on iSCSI Luns on my SAN.  Even my hypervisor boots over iSCSI.  Now I would like to have a dedicated link between my hypervisor and my SAN.

I've created a new vmkernel port configured for a new network (172.16.1.0/24) and added an available physical adapter to it. I added a direct cable connection between the hypervisor and the SAN and configured the adapter on the SAN to be on the same new network.

I then added a new static iSCSI target using the new network, rescanned all HBAs, and I now have a new iSCSI connection to the SAN.  I also reconfigured bootup so the iSCSI connection used to boot ESXi uses the new network.

All of that works.  What doesn't work is removing the old iSCSI target and I don't know why.  The error is always the same.  I have rebooted many times, and I've disconnected the SAN from my LAN such that I know the legacy iSCSI connection cannot be established upon boot.

What can I do?

Thanks,

Donald

Message was edited by: seijirou

10 Replies
seijirou
Contributor
Contributor

Please let me know if there is any more information I need to provide in order to help bring this problem to resolution. 

Reply
0 Kudos
amiterjain
Contributor
Contributor

Hi Donald,

I need to understand the problem so could you please check your setup and let me know these things.

1. After configuring the new setup are you sure that your system is booting through new iSCSI target and not the old one.

2. Have you cleared/modified iSCSI boot settings in adapter BIOS ( just to make sure server does not boot through the old target)

3. Please check the NIC boot sequence if you are booting through the desired NIC and it is not same as previous one.

4. Please remove the network connection of the NIC from which your system was booting up and then reboot the system and then try to remove the entry.

Reply
0 Kudos
seijirou
Contributor
Contributor

Hi Amiterjain,

I will try my best to answer your questions.  Thanks for your reply.

1. After configuring the new setup are you sure that your system is booting through new iSCSI target and not the old one.

Yes I am sure.  My network cards are not natively capable of creating the iSCSI connection necessary to boot.  I use the iPXE application (http://ipxe.org/) booted from a USB drive to enable iSCSI booting.  In using iPXE I manually configure the iSCSI target to boot from, and once I added the direct cable connection I modified the USB boot image and changed the iSCSI target to the new network target, 172.16.1.1.

2. Have you cleared/modified iSCSI boot settings in adapter BIOS ( just to make sure server does not boot through the old target)

Because I am using iPXE the adapter BIOS does not apply.  iPXE is a pre-boot environment loaded from the USB drive, so my motherboard BIOS is configured to boot from USB, then iPXE takes over, creates the iSCSI connection, and initiates boot from iSCSI.  At no point is the actual network adapter BIOS involved.

3. Please check the NIC boot sequence if you are booting through the desired NIC and it is not same as previous one.

Part of the iPXE configuration requires initializing the network adapter before assigning an IP and creating the iSCSI connection.  In order for me to successfully create the iSCSI connection I had to change the network adapter I was initializing.  This process was "guess and check" in order to find the correct network adapter.  Until I got it right, the iSCSI conection would just time out.  So I am certain that the correct network adapter with the direct cable connection to my SAN is being used.

4. Please remove the network connection of the NIC from which your system was booting up and then reboot the system and then try to remove the entry.

I have done just this, although I may not have been clear about it in my first post so I apologize.  My SAN normally has 2 network connections, 1 goes to my switch carrying the 192.168.2.0/24 network, and the other goes directly in to my hypervisor.  On the direct cable connection my SAN has the IP address 172.16.1.1, and my hypervisor has 172.16.1.2.

     I removed the network cable between the SAN and the switch, and booted the hypervisor.  When this is done and the hypervisor has booted, at the vpshere console I see 1 iSCSI connection with 1 path (with both cables connected I would see 2 paths).  However I am still unable to remove the static software iSCSI target to 192.168.2.20, with the exact same error message given in my first post.

Thanks for your time,

Donald

Reply
0 Kudos
amiterjain
Contributor
Contributor

Hi Donald,

Could you please try out removing the network connection of 192.168.2.0 network and after that boot the Hypervisor from 172.16.1.2 and then trying remove the Static discovery address.

If you have already done this then your setup should work fine.

Thanks

-- Amit

Reply
0 Kudos
seijirou
Contributor
Contributor

Amit,

I have removed the 192.168.2.0/24 network connection, and then rebooted the hypervisor from 172.16.1.2.

I am still unable to remove the Static discovery.  The error remains the same.

Thanks,

Donald

Reply
0 Kudos
seijirou
Contributor
Contributor

I now think this is a bug so I am updating the thread accordingly.

When ESXi is installed to an iSCSI target, the static iSCSI target is created automatically.  Even after the initial iSCSI target is no longer in service (as a new iSCSI target is created connecting to the same LUN), the initial iSCSI target still cannot be removed.

I have been in the documentation trying to find out if removing an iSCSI target can be done via esxcli, but so far I have not found it.

Regards,

Donald

Reply
0 Kudos
rakeshbitsindri
Contributor
Contributor

I have also encountered this problem and there is no workaround for that, and now I have to remove the ESXI and install it from the fresh!

It would be great if Vmware could provide a workaround for this.

Observe that in the image the remove/add option is completely greyed out after adding a software iscsi adapter.

software_adapter.jpg

Reply
0 Kudos
syedak
Contributor
Contributor

Hi Guys,

I am also facing the same issue. Any solution found Smiley Sad

Reply
0 Kudos
griffinboy
Enthusiast
Enthusiast

Usually after removing a iscsi target or making config changes to the software iscsi initiator,  the host should be rebooted.

Have you bound your iSCSI vmk interface to the Software iSCSI initiator (vmhba38) at all?

If so it may need un-binding before you can remove the target (although i have to say i don't think i've had to do that before)

(i'm using the vMA to run all of the following commands so vicfg or vmware-cmd commands may not work directly on the host unlless you have installed vcli)

I would put the host in maintenance mode:

vicfg-hostops --operation enter > /dev/null 2>&1

Not sure if you have tried before at the cli as root but I would remove the iscsi initiator and re-add it:

esxcli iscsi software set -e false

esxcli iscsi software set -e true

To un-bind an interface you can use this command:

esxcli iscsi networkportal remove -n vmkX -A vmhbaXX

Here's a small sample from one of my scripts that configures my hosts after a fresh install. Some of the commands may help you.

############# snip ##############

#enable the Software iSCSI initiator on the host

esxcli iscsi software set -e true
#configure iSCSI port binding
esxcli iscsi networkportal add -n vmk1 -A vmhba32
esxcli iscsi networkportal add -n vmk2 -A vmhba32
#add send target(s) portal for iSCSI SAN
esxcli iscsi adapter discovery sendtarget add -A vmhba32 -a $SAN_IP":3260"
#set multipathing frim default to round-robin
esxcli storage nmp satp set --default-psp VMW_PSP_RR --satp VMW_SATP_DEFAULT_AA
#rescan for storage devices and new volumes
esxcli storage core adapter rescan -a
esxcli storage filesystem rescan

############# /snip ##############

Good luck

VCPID: 40118 (VCP310, VCP4)
Reply
0 Kudos
zyman2008
Contributor
Contributor


I also facing the same issue on my lab.

After some tray and error, I remove the vmhba to solve this problem.

Here my step,

1. Go to properties of the software adapter.

2. On General tab, click Configure

3. Then uncheck the Enable.

4. Reboot the ESXi host adn then the vmhba interface gone.