VMware Cloud Community
clizarr
Contributor
Contributor

ESXcli 5.5 UNMAP and PowerCLI script?

Has anyone been able to get powercli to run the esxcli command for unamp

esxcli storage vmfs unmap -l Datastore name -n unit

any example would be appreciated

Tags (1)
59 Replies
vkev
Contributor
Contributor

HI,

Try this for an individual server:

connect-viserver -Server "hostname here"

$esxcli = Get-EsxCli -VMHost "hostname here"

$esxcli.storage.vmfs.unmap(200, "datastore name here", $null)

The unmap will return true if it's worked

kev@vkev.co.uk

0 Kudos
znip
Contributor
Contributor

You may also want to disable timeout if your operation takes more than 300 seconds:

Set-PowerCLIConfiguration -Scope Session -WebOperationTimeoutSeconds -1

0 Kudos
LarsB13
Contributor
Contributor

See attached script. The script loops through all datastores (one at a time), and runs the unmap command if the datastore is accessible and thin-provision capable. 

I have been working on this script for a customer; it works well at reclaiming space on our LUNs.     I ran into issues with the esxcli unmap command timing out after 30 minutes.  It timed out even if the 'WebOperationTimeoutSeconds' was set to no timeout or  something higher than 30 minutes.   Even though the timeout occurred in the powerCLI script, the unmap operation continued through the ESXi host and could be tracked to completion by tailing the hostd.log file on the ESXi host.   I presented the timeout issue on Server Fault; as of this posting there isn't a resolution.  vmware esxi - ESXcli command timeout at 30 minutes - Server Fault.

R/

Lars

0 Kudos
MMcDonald_EWS
Enthusiast
Enthusiast

I'm trying to use this script but every datastore it processes errors out with the following error.  Thoughts?

DEBUG: 7/17/2014 4:32:57 PM Get-View

VERBOSE: [ Storage-Datastore1 ] - Refreshing Datastore Data...

VERBOSE: [ Storage-Datastore1 ] - Gathering statistics...

VERBOSE: [ Storage-Datastore1 ] - Running unmap, can take 30+ minutes

VERBOSE: Message: Cannot find VMFS volume with label Storage-Datastore1;

InnerText: Cannot find VMFS volume with label Storage-Datastore1EsxCLI.CLIFault.summary

VERBOSE: 7/17/2014 4:33:02 PM Get-View Started execution

VERBOSE: 7/17/2014 4:33:03 PM Get-View Finished execution

0 Kudos
MMcDonald_EWS
Enthusiast
Enthusiast

I figured out the issue.  At some point in the script, an arbitrary ESXi host is selected to run the unmap workload.  With this is an assumption that this ESXi host can see all datastores that are present in the entire vCenter.  In my case, I have one vCenter with multiple datacenters/clusters configured that can only see certain datastores.  So this method of selecting the ESXi host will not work for me.

I've edited the script to take into account my scenario by iterating through each datacenter and cluster within, selecting a host from each cluster (which is expected to see all datastores within said cluster).  I have attached the edited script to this post.  I'm running it now (will take a long time) and it seems to be doing the trick.

Message was edited by: mcdonamwION

0 Kudos
iforbes
Hot Shot
Hot Shot

Hi
mcdonamwION. I also have multiple clusters and hosts within a cluster can only see their own datastores. When I run your script I get:

Get-Datastore : The input object cannot be bound to any parameters for the comm

and either because the command does not take pipeline input or the input and it

s properties do not match any of the parameters that take pipeline input.

At C:\users\sp_studentlabvcadmin\Downloads\vmware_unmap_datastore.ps1\vmware_un

map_datastore.ps1:113 char:46

+         foreach ($ds in ($VMCluster | Get-Datastore <<<< )) {

    + CategoryInfo          : InvalidArgument: (LABS-DESKTOP-JHE:PSObject) [Ge

   t-Datastore], ParameterBindingException

    + FullyQualifiedErrorId : InputObjectNotBound,VMware.VimAutomation.ViCore.

   Cmdlets.Commands.GetDatastore

Any idea why I'm getting that error?

0 Kudos
CNI0
Enthusiast
Enthusiast

Hi

I have the exact same problem as LarsB13. Anyone found a solution for this?

Regards

Claus

0 Kudos
CNI0
Enthusiast
Enthusiast

If i log in to the ESXi with SSH and run the same command everything runs as expected.

esxcli storage vmfs unmap -l <Datastore label>

The command completed successfully in about 75 minutes. I guess there is some kind of flaw in PowerCLI version of esxcli that makes it timeout after exactly 30 minutes 😞

And PowerCLI is not covered by GSS so no result from the SR i created.

0 Kudos
LucD
Leadership
Leadership

Did you change the timeout before running the command ?

See the Set-PowerCLIConfiguration cmdlet above.

Did they actually tell you that PowerCLI is not covered ?

Because last time I checked it was.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
CNI0
Enthusiast
Enthusiast

Yes, I did set the timeout with Set-PowerCLIConfiguration like this:

Set-PowerCLIConfiguration -WebOperationTimeoutSeconds -1 -Scope Session -Confirm:$false

This is an exact copy/paste from my SR:

"Thank you for your Support Request.

Through GSS we do not support Power cli problems."


Regards

Claus

0 Kudos
alanrenouf
VMware Employee
VMware Employee

Let me assure you that PowerCLI as a product is supported but as you can imagine the line between scripts and the actual cmdlet can get blurry, looks like we have a miscommunication somewhere internally here, I apologize for that.

Would it be possible to post the SR number so I can fix the communication issue here and investigate the issue more.

Thanks

Alan

Blog: http://virtu-al.net Twitter: http://twitter.com/alanrenouf Co-author of the PowerCLI Book: http://powerclibook.com
0 Kudos
CNI0
Enthusiast
Enthusiast

Hi Alan

The SR is 14535091209.

I am glad to hear that I can support Smiley Happy

If I run the "esxcli storage vmfs unmap" command through a SSH session the command completed successfully in 2:34 hours.

Regards

Claus

0 Kudos
alanrenouf
VMware Employee
VMware Employee

Thanks Claus, I will ask the support engineer to follow up on this.

Blog: http://virtu-al.net Twitter: http://twitter.com/alanrenouf Co-author of the PowerCLI Book: http://powerclibook.com
0 Kudos
bollipa
Contributor
Contributor

To get the script to work you must replace the following line:

foreach ($ds in ($VMCluster | Get-Datastore)) {

replace ist with:

foreach ($ds in ($ESXiHost | Get-Datastore)) {

bye

CODJ
Contributor
Contributor

Thanks bollipa, this worked:

To get the script to work you must replace the following line:

foreach ($ds in ($VMCluster | Get-Datastore)) {

replace it with:

foreach ($ds in ($ESXiHost | Get-Datastore)) {

0 Kudos
MaxB
Enthusiast
Enthusiast

Hey Guys,

i found this little powershell function http://www.zerrouki.com/invoke-ssh/

i modified this script a bit that it works with esxcli unmap.

The benefit of using this is that it doesn´t run in a 30 minutes timeout, because it use puttys plink.exe!!!

But before you can run the unmap.ps1 you have to run the collect_ssh_keys.ps1 to store all the hostkeys (never know with host will be taken)  in your local registry.

So when running the unmap script no "would you like to store the hostkey" questions are coming up.

I know the logging is not the best but its a beginning... feel free to modify!

I´m sorry for the scriptformat... so i attached it also as pdf for formatting it readable.

Bye,

MaxB

0 Kudos
HippoCoar
Contributor
Contributor

I know that my efforts in this have me standing on the shoulders of giants, as my work is just another modification to those who posted both the original script, and the one that allowed for some datastores not being visible to all clusters/datacenters in a vCenter.

I will also preface this by saying that I was led here looking for a way to completely automate the vmfs unmap of all my datastores. Originally, I was looking to set it up as a series of scripts distributed across my ESXi 5.5 hosts, and run those scripts as cron jobs, but that effort was proving fruitless. I couldn't get the script to run properly, much less get the cron job set up. As a member of a two-member team responsible for the entire server/storage/network/VDI workload for a school district, we thrive on automation. If it can be automated, it should be. If it can be scripted to reduce the amount of time I have to spend on it in the future, it should be. Period.

I digress. I made it to here.

One of the situations I have going on in my environment that made the original version of this script not work for me was the lack of datastore visibility to all my hosts. We have two vCenters (one for VM servers, one for VDI), and our server vCenter has, at present, two different clusters, one for "normal" servers, and one for servers related to VoIP (the hosts in the VoIP cluster are MUCH less powerful). Additionally, all the ESXi hosts are Cisco UCS blades, all booting from iSCSI SAN storage. Having come from a military background, when we re-designed our entire storage policy for disaster recovery, I put in place restrictions on most of my datastores to only be visible to the vCenter clusters that needed to access them, and restrict the iSCSI boot datastores to only be visible to their respective hosts.

As a result, I took a look at the second version of the script that was posted that did the VMFS unmapping by connecting to all the clusters individually, and painstakingly (my PowerShell skills are weak so I had to look up most of the commands as I went) modified it to connect to ALL the hosts in ALL the clusters to do the unmapping. I realized this was going to create many redundant unmappings as many of my datastores are visible to more than one host, so I added a separate sub-process to keep track of the datastores it had already processed, and not process them again.

I ran this modified version, and found that it mostly worked, but, like many others, I hit the 30 minute timeout on the unmap process on one of my datastores, even running it with a block count of 800 against a 10TB VMFS5 datastore. Not liking the "SSH into the server from PowerShell" option that relies on keys, etc, as I try to do things as natively as possible, I tried to increase the block count again, this time to 2000, but I found that my boot datastores, the ones visible to only one host, had an unused block count of 1936, just enough under my limit to create problems. This led me to the realization that all this was scripted, and other parts of the unmap command were variablized (such as the datastore to run the unmap command against), so why not variablize the block count as well? If PowerCLI can find that there aren't enough free blocks to run the command, then there has to be a command set I could run to find how many free blocks there were, and set my block count for the unmap command as a function of that.

With this concept in mind, I started doing research on the impact of running the unmap command with a higher block count than the default of 200, and since it was always my goal to automate it running in the off-hours, but never to degrade performance significantly even if it were run during working hours, I settled on a block count that was 0.5% of the datastore freespace in MB (as the default block size for VMFS5 volumes is 1MB/block). I rounded this number as a function of Powershell, and injected it back in.  The end result was that the script ran in about 25 minutes total, instead of the 1.5+ hours it ran at previously. And remember that 10TB datastore that failed to process with the block count at 800? It completed successfully in three minutes, and ran at a block count of 16172.

Having already read this thread, and knowing there were others running into the same issues, I now present you with my modifications to this script. I have included commented blocks to allow it to run without any user input AND connect to multiple vCenters so it can be run as a scheduled task from a Windows server. Be advised that as of the time of this posting, I have not yet run it as a scheduled task, but the theory is sound and the script runs.

Snow days are fun, aren't they? Without it, I'd have had real work to do today!

G2SO
Contributor
Contributor

Hi HippoCoar and all,

This sounds great and thanks for working this!

I'm not a script or powershell guy but get around with some simple commands. I too stumbled on to this thread to find a way to automate the unmap commands. For awhile now, I have been running the original script from this thread on a "as needed" bases. But I would rather have this automated with no prompts and run from a windows scheduled job.

I have not tested your script yet. I just ran the original one and cleaned it all up. I will do so soon.

A couple of questions

- did you get it run completely automated?

- I have a simple VMware environment with 3 hosts, 1 vCenter, 1 cluster and datacenter. I would imagine the script should work just fine for this type of environment.

Thanks for the help here and for all the folks that contributed!

0 Kudos
HippoCoar
Contributor
Contributor

I apologize for taking so long to get back to you on this. Yes, by setting it as a scheduled task using an account that has the proper permissions in vCenter, it runs completely automated. The glory of this script is that it automatically finds all hosts in all clusters after pointing it at the vCenter server(s), and, by extension, all datastores connected to those hosts. Like I said, it only takes having an account that has the proper vCenter permissions - the script does the rest.

As we prepare to make the migration to ESX 6, I have found that the script continues to work there as well, with no tweaks. For anyone using datastore clusters, I cannot confirm that it works, as we don't run any in our environment, but even as I write this, I think I may set up a small one just to test it out. I seriously doubt that it would be vendor specific, as iSCSI is rather established, but we currently have EqualLogic and Nimble for storage, and there have been no problems with it working on either without any special treatment. Your mileage may vary.

0 Kudos