VMware Cloud Community
godbucket
Enthusiast
Enthusiast
Jump to solution

Way to script and automate vmkfstools command?

Extremely long story short, running vSphere 5, I need a way to run the "vmkfstools -y 60" command against my datastores to reclaim free space on my Compellent SAN array.

This article explains the command and the purpose: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=201484...

So from either one or multiple hosts, I need to go into each VMFS datastore and run the "vmkfstools -y XX" command to reclaim space. Otherwise the Compellent array will see SVMotion jobs and Snapshots as "new writes" and never release the free space.

To automate this, I'd like to use PowerCLI and maybe just run the script as a scheduled task from my vSphere server.

I also understand the VMA appliance can do this kind of stuff but I havent figured out how to automate/schedule tasks with it, plus I'd rather use PowerCLI.

Anyway, sorry for the noob question, and thanks in advance. Any help is much appreciated!!!

1 Solution

Accepted Solutions
brettcarr1
Enthusiast
Enthusiast
Jump to solution

The problem is we use Thin Provisioned Volumes on a 3par Array, with T10 Unmap turned off (Which is now the default) freeing space up by deleting a VM or doing a storage vmotion means the space gets freed up at the VMWare/Datastore Layer and can be reused by Vpshere but it is not reclaimed at the Array Level, issuing a vmkfstools, writes a file to the datastore and then deletes it and sends T10Unmap back to the Array, the 3par then returns this space to the free pool. This works when we do it manually we just want to automate it until VMWare decide to turn on UNMAP again.

Brett

View solution in original post

0 Kudos
12 Replies
godbucket
Enthusiast
Enthusiast
Jump to solution

Bump? Smiley Sad If I posted this question in the wrong catagory, someone please feel free to move it. Thanks in advance!

0 Kudos
iw123
Commander
Commander
Jump to solution

Hi there

You could do this using the vmkfstools.pl script included as part of the vsphere CLI tools. This is essentially what you can do with the vMA but you can install it on a windows machine if you wish. I guess you could then create a batch file to use the script, stating the required parameters.

You can find more on this here http://www.virtuallyghetto.com/p/vmware-vcli-rcli.html

Cheers!

*Please, don't forget the awarding points for "helpful" and/or "correct" answers
0 Kudos
godbucket
Enthusiast
Enthusiast
Jump to solution

The "-y" switch/option isn't available in the vmkfstools.pl script, and I'm not sure how to invoke it even if it is supported. Cant seem to find syntax for that script.

Regardless, if there is a better way please someone let me know. I've tried using Orchestrator, but for some reason you cant change directories into a datastore and run vmkfstools. It just says "directory not found".

All I really need to do is find a way to automate these 2 simple lines:

cd vmfs/volumes/datastorename

vmkfstools -y 60

Does anyone have any ideas? Thanks in advance, any help is appreciated greatly

0 Kudos
brettcarr1
Enthusiast
Enthusiast
Jump to solution

Did you find a way to do this, we have the same issue (Different Array) and need to automate this, I'd (much) rather do it using powercli if i can.

0 Kudos
godbucket
Enthusiast
Enthusiast
Jump to solution

Oh, yes I did! Sorry, meant to repost what I did but I'd forgot. I've drank since then...

I ended up installing vCenter Orchestrator and using scheduled workflows to automate the process

(I owe credit for this idea to my good friend Sean Howard at VMware)

There's a few catches though, few things you have to do before creating workflows in Orchestrator:

- you have to enable SSH on a host (or a few; depending on how many hosts you want running the workflows)

- you have to/should suppress the SSH security warnings on those hosts

- you have to enable SSH password authentication on those same hosts

- you have to enable use of the UNMAP commands for space reclamation (VAAI primitives)

- then you can issue the UNMAP comands against the datastores/LUN's

Please bear in mind, the whole reason I wanted to automate vmkfstools commands is for the purpose of running UNMAP commands against my datastores so my Compellent SAN's would reclaim unused space caused by snapshots, backups, and SVMotion jobs. Below are the complete list of instructions for what I did to accomplish this.

Enable SSH

SSH is required for command-line access to ESXi hosts.

  • Using the vSphere Client, select the ESXi host, then the “Configuration” tab, then under “Software” click “Security Profile”.
  • In the top-right corner, click “Properties”, then select “SSH”, then click “Options”
  • From here you can start or stop the SSH service and set the startup type to manual or automatic.

Disable SSH security warnings

If SSH is enabled, it will generate warnings on the summary tab of each host. These warnings can be suppressed.

  • Using the vSphere Client, select the ESXi host, then the “Configuration” tab, then under “Software” click “Advanced Settings”.
  • Locate “UserVars” on the left-hand pane, then change “UserVars-SuppressShellWarning” to 1.

SSH Password Authentication

To allow for applications such as vCenter Orchestrator to have root or shell access to an ESXi host, Password Authentication must be enabled.

  • Log into the console with SSH (putty).
  • Navigate to /etc/ssh and edit sshd_config using vi
  • Type vi sshd_config and press Enter.
  • Now you are in the vi text editor. You can move around using page up, page down, and the arrow keys.
  • Enter insert mode. You can press i to start editing where the cursor is, you can also press shift+o to start editing on a new line above the cursor, or o to start editing on a new line below the cursor.
  • Change the line "PasswordAuthentication no" to "PasswordAuthentication yes".
  • To save and quit press Esc, :, w, q.
  • To quit without saving, press Esc, :, q, then press Enter.
  • Restart the SSH daemon on the ESXi host under “Configuration” tab, and “Security Profile”

Enable use of UNMAP commands for space reclamation

vSphere 5.0 introduced VAAI Thin Provisioning Block Space Reclamation (UNMAP) Primitive. This feature was designed to efficiently reclaim deleted space to meet continuing storage needs. ESXi 5.0 issues UNMAP commands for space reclamation during several operations.

  • Log into the console with SSH (putty) and issue the command below
  • esxcli system settings advanced set --int-value 1 --option /VMFS3/EnableBlockDelete
  • This is a per-host setting and must be issued on each ESXi 5.0 host in your cluster.

Issue UNMAP command against VMFS datastore (manually)

The actual space reclamation command is as follows. This writes a “balloon file” to the top blocks of the datastore to persuade the SAN into relinquishing free space

  • Log into the console with SSH (putty)
  • Change directories into the VMFS datastore you wish to reclaim space from
  • cd vmfs/volumes/datastorename (where datastorename is the actual name of the datastore)
  • Issue the UNMAP command
  • vmkfstools -y 60 (where 60 is the percentage of space the balloon file will attempt to reclaim space from)

So what you can do is create custom scheduled workflows in vCenter Orchestrator and run something like this:

cd vmfs/volumes/LUN-NAME;vmkfstools -y 60

Where "LUN-NAME" is the VMFS datastore name.

That's pretty much it!!! I hope this helps someone. I may repost these instructions in a separate post with a better title, for easier searching.

RParker
Immortal
Immortal
Jump to solution

Brett Carr wrote:

Did you find a way to do this, we have the same issue (Different Array) and need to automate this, I'd (much) rather do it using powercli if i can.

I suggest you contact your storage vendor first.. most if not all have affiliation with VMWare and it's clearly documented on how to free up space.

The reclamation is tied to files within a data store itself, if that datastore isn't setup right (with permissions to delete files for instance) then the data isn't removed, so the datastore still shows it as used, which is true.

The problem therefore is with either your permissions or the way it was configured to work with your ESX host.  If you MOVE a file, don't care if it's zeroed or white space or not.. a move is a move, that file is marked for delete, which in turn tells the file system to reuse it...

I have seen many issues with EMC, Clarion, Netapp, and Compellent, but reclaiming of space means there is a problem with ESX and the storage configuration the storage will reclaim space that is UNUSED.. don't care how old or what vendor, that will work 100% if that storage device is certified for VMware...

Something else is the cause.

Also I have done many disk store cleanup, within a storage array and datastore, never touch vmkfstools... no need to do command line, there are many other ways to handle it.

0 Kudos
godbucket
Enthusiast
Enthusiast
Jump to solution

brettcarr1, please see my other post if you have any questions. I hopefully set the record straight there:

http://communities.vmware.com/message/2199370#2199370

Hope this helps you somehow. Please feel free to ask me questions, I'd be happy to help.

0 Kudos
brettcarr1
Enthusiast
Enthusiast
Jump to solution

The problem is we use Thin Provisioned Volumes on a 3par Array, with T10 Unmap turned off (Which is now the default) freeing space up by deleting a VM or doing a storage vmotion means the space gets freed up at the VMWare/Datastore Layer and can be reused by Vpshere but it is not reclaimed at the Array Level, issuing a vmkfstools, writes a file to the datastore and then deletes it and sends T10Unmap back to the Array, the 3par then returns this space to the free pool. This works when we do it manually we just want to automate it until VMWare decide to turn on UNMAP again.

Brett

0 Kudos
godbucket
Enthusiast
Enthusiast
Jump to solution

Exactly! Well put! That's the exact same problem I was having, only your wording is much better (and condensed! lol)

So yeah, vCenter Orchestrator man, all the way. The Perl script mentioned earlier I'm sure is very helpful but I dont know Perl. Orchestrator is great, quick spin-up, schedule the workflows, and you're done.

0 Kudos
brettcarr1
Enthusiast
Enthusiast
Jump to solution

Weirdly all my experiences of Orchestrator have been confusing and well I just don't get on with it, If you know of any good intros/tutorials I would be happy to try it again.

My intention for this however was to use PowerCLI to do the following

Turn on SSH/Root Access to the Host

call plink to login to the host via ssh

run the vmkfstools command for each Datastore via plink

Turn off SSH/Root Access to the host.

I'll probably add something to report the datastore usage (at the SAN Level) before and after aswell.

Brett

godbucket
Enthusiast
Enthusiast
Jump to solution

Originally I had wanted to do this too, but there isn't a native PowerCLI command for vmkfstools I don't think, and I suck at using PLink, so I gave up early on that. You're right, scheduling tasks with Powershell scripts would be the ideal way to go. Could run them all from the vCenter Server. If you find a way to do this, please share. I'd love to see it work and would consider using it myself. Thanks!

0 Kudos
vkev
Contributor
Contributor
Jump to solution

Depends,

For ESXi 5.5 this is easy and can be initiated with PowerCLI as below as the unmap command is now in esxcli rather than vmkfstools:

$esxcli = Get-EsxCli -Server <host>

$esxcli.storage.vmfs.unmap(200, <datastore volumelabel>, $null)

For ESXi 5.0/5.1 vmkfstools is used, to schedule this you could either:

- Add this plugin ( http://www.v-front.de/2013/01/release-esxcli-plugin-to-run-arbitrary.html )that would allow you to run vmkfstools from PowerCLI ( this has implications for security, and allowing community supported vibs on the host )

- Create a script for the vmkfstools then modify local.sh to create the script and cron job each time the ESXi server boots

Regards

vKev

0 Kudos