VMware Cloud Community
ArrowSIVAC
Enthusiast
Enthusiast

Unable to Delete Offline VMFS Volume: Storage I/O Control Removal Error

Environment: 20 servers, VMWare Farm, v5.0.

2TB Fibre attached LUN mapped to all hosts in the cluster as VMFS3.  LUN was destroyed.  It was not (as seen below) under the storage control pool "Production Disk". Need to remove this dead volume so I can add a copy of the volume back with the same name.

Right click Volume -> Delete

Error

Call "HostDatastoreSystem.RemoveDatastore" for object "datastoreSystem-606" on vCenter Server "vc01.aessatl.arrow.com" failed.
Cannot remove datastore 'vmfs3' because Storage I/O Control is enabled on it. Correct it and re-try the operation

2021464.png

Google gets hits on NFS versions (http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=103739... ) of this removal process but no "properties" option can be accessed with the volume offline. So their is no way to remove it from Storage I/O control.

VMWare Volume Offline
2021464_1.png

Tried to SSH into the system directly and as noted in entries related to this error, force unmount and removal of the volume.

SSH Removal Attempt
~ # esxcli storage filesystem list
Mount Point                                        Volume Name  UUID                                 Mounted  Type                   Size                 Free
-------------------------------------------------  -----------  -----------------------------------  -------  ------  -------------------  -------------------
/vmfs/volumes/48ae6b3e-36d0cdaf                    labfiles     48ae6b3e-36d0cdaf                       true  NFS           1449551462400         146889039872
/vmfs/volumes/4c51b3b7-bfb2faee-55bb-001a6459f874  vmfs2        4c51b3b7-bfb2faee-55bb-001a6459f874     true  VMFS-5        1992596389888         149034106880
/vmfs/volumes/4c4e0eb7-561ea784-d8bc-00145ed56dd2  vmfs0        4c4e0eb7-561ea784-d8bc-00145ed56dd2     true  VMFS-5        1992596389888         142396620800
/vmfs/volumes/4c4e0ed5-7337d28a-c88b-00145ed56dd2  vmfs1        4c4e0ed5-7337d28a-c88b-00145ed56dd2     true  VMFS-5        1992596389888         159938248704
/vmfs/volumes/4f3a83ea-68bca8ed-9465-e61f133b5277  vmfs3        4f3a83ea-68bca8ed-9465-e61f133b5277     true  VMFS-5  7021786302599163750  1248136196516808178
/vmfs/volumes/4f7b4da5-3080997e-4b0c-00145ed58288  vmfs4        4f7b4da5-3080997e-4b0c-00145ed58288     true  VMFS-5        3298266447872        2419947732992
/vmfs/volumes/4f2be6e2-558576c3-5234-001a64760061               4f2be6e2-558576c3-5234-001a64760061     true  vfat             4293591040           4257939456
/vmfs/volumes/113d2a77-f5150d49-9073-4f748fd9134d               113d2a77-f5150d49-9073-4f748fd9134d     true  vfat              261853184            122101760
/vmfs/volumes/52a15d7d-2a7dd8b9-96dd-e20ce415e5f0               52a15d7d-2a7dd8b9-96dd-e20ce415e5f0     true  vfat              261853184            108527616
/vmfs/volumes/4f2be6cb-edc71c48-ca6f-001a64760061               4f2be6cb-edc71c48-ca6f-001a64760061     true  vfat              299712512            114843648
~ # esxcli storage
Usage: esxcli storage {cmd} [cmd options]

Available Namespaces:
  core                  VMware core storage commands.
  nmp                   VMware Native Multipath Plugin (NMP). This is the VMware default implementation of the Pluggable Storage Architecture.
  vmfs                  VMFS operations.
  filesystem            Operations pertaining to filesystems, also known as datastores, on the ESX host.
  nfs                   Operations to create, manage and remove Network Attached Storage filesystems.

~ # esxcli storage vmfs remove vmfs3
Error: Unknown command or namespace storage vmfs remove vmfs3

Question: What is the process to unbind this volume from Storage Control and then remove it?

Reply
0 Kudos
1 Reply
bobbygedwards
Contributor
Contributor

Stupid Question:

Is it possible to put one of the hosts in maintenance mode and reboot it to see what happens?

Maybe look over this post:

http://communities.vmware.com/message/1640132

Reply
0 Kudos