vSphere/Storage Monitoring Tools for Thin Provisioning

[Brad_P]

We are a completely VMware and NetApp shop, and I am in the process of evaluating a number of enterprise monitoring solutions, each with its own virtualization+storage monitoring module or component.

We are also currently 100% iSCSI on the VMware storage side, and so we thin provision many of our LUNs in order to take advantage of NetApp's ASIS, which does a wonderful job of block deduplication at the volume level. But what ends up happening is someone requests a VM with an absolute requirement of say 65GB disk space. They only end up using 35GB for the first few months...NetApp's ASIS does it's thing, and that remaining empty 30GB is no longer taken up on the volume utilization. The datastore usage is still there, but since they're thin provisoned, that's not a hindrance to creating more. After a while, you don't know what space savings you're getting from empty Datastore space, empty VMDK's, or identical OS blocks.

SO...two months later, that person starts filling up that 30GB with unique stuff, and the volume space dwindles quickly.  I would love to see a monitoring tool which can monitor and report on 3 things:

1. Available NetApp Volume Space
   
2. Available VMware Datastore space provisioned on that volume
   
3. Available VMDK space within those Datastores
   
4. Report on the vulnerabilites:
   
   1. Any thin provisioned VMDK which could fill up your data store
      
   2. The sum of all available datastore space on a per NetApp volume basis
      
   3. The sum of all available VMDK space from a guest perspective on a per NetApp volume basis (space that has been provisioned/reserved in the datastore, but NOT in the underlying volume).
      

This way, you'd be able to not provision too much storage which has the potential to come back to bite you, but still take advantage of deduplicating all of the common OS blocks by KNOWING the difference between the former and latter. This would enable you to make an educated decision as to how much you can safely over provision your volumes.

I have asked every monitoring sales engineer about how to accomplish this with their tool, and surprisingly no one had a good answer. This functionality has to exist, in an enterprise monitoring solution, right?  Does anyone have any suggestions for a tool that can do that?

Thanks in advance for any suggestions.

[ChrisDearden]

If you are prepared to get your hands dirty with some powershell I suspect there is a good ammount you coudl get form that ?

http://virtualcurtis.wordpress.com/2010/09/24/thin-provisioning-and-storage-capacity-on-vsphere/

[Brad_P]

Thanks for the reply.

I know that at least most of this is possible with PowerShell, I've used the NetApp PowerShell SDK and VMware PowerCLI to do some pretty cool things. This is why I figurethat it's possible...but I would like the functionality in a monitoring tool itself, rather than just running some PowerShell scripts for a few reasons:

1. The data would be stored so historical reports can be run on it from the same interface as the rest of the monitoring data
   
2. Trending analysis could be done in the background and alerted on
   
3. Single pane of glass for environment health analysis
   
4. I can imagine how slow this script would be to run, connecting to each of the platforms to gather this information in realtime.
   
5. I am getting ready to spend real money on an enterprise monitoring solution. And frankly, having to tack on some powershell scripts for such a seemingly basic requirement is irritating.
   

Am I being unreasonable?

[ChrisDearden]

if its a single pain of glass you are looking for then I guess the next place to look would be somethign like Microsoft System Center Operations Manager with data fed by the Veeam management pack ( you could use Veeam monitor if you wanted the same data without SCOM , but SCOM would be a usefull place to aggregate data from other systems )

[Brad_P]

I've looked at Veeam Monitor, but that simply takes data from vCenter, which is not sufficient in this case. I have also looked at Quest vFoglight Storage,  and SolarWinds Profiler.

The solution also needs to talk directly to NetApp to get volume information, as vCenter is completely unaware of the underlying volumes.

It needs to start at the storage array and work backward into the virtual infrasture, i.e.:

Report:

• NetApp /vol/example_vol is 1TB in size, with 200GB free
  
  • LUNs provisioned on /vol/example vol total 2TB
    
    • Datastores SUM free space 120GB <---Volume vulnerability to VMware snapshots, VM provisioning, etc. VI Admin has relative amt good control
      
    • VMDK SUM free space 250GB <----Admin may have least control over. Trending report very useful. Alerts/Alarms very important.
      
• /vol/example_vol is vulnerable to over commit by 170GB
  
• Top X Datastores w/ free space in /vol/example_vol
  
• Top X VMDK w/ free space in /vol/example_vol
  

[ChrisDearden]

You are correct - Veeam Monitor will only pull in stats from VC.

However if you use Microsoft System Center Operations Manager ( SCOM ) - then you can use the NetApp Management Pack to pull in Netapp Data.

http://www.systemcentercentral.com/tabid/145/indexId/58922/Default.aspx

and the Veeam Management Pack to pull in the VC data

I believe you should be able to use the 2 Mps to pull the data together into the framework you mentioned.

EDIT:

I spoke with an ex collegue of mine who is a SCOM MVP - he was pretty confident that some nice reports could be built using the 2 management packs to bring it all together.

[Brad_P]

That could definitely be doable...thanks for the background. I'll check out SCOM and see if I can get this level of reporting/correlation of SAN/VI.

I appreciate the research, and I'll report back my findings. 

[regnak]

Hi,

We're using Solarwinds Profiler to monitor a vCenter and NetApp environment, it gives good visibility over Storage, Zoning mapping etc:

http://www.solarwinds.com/products/profiler/storage_management/

Mike

[VeraxSystems]

Give a try to VMware vSphere plugin for Verax NMS (http://www.veraxsystems.com/en/products/nms). You can monitor NetApp Volume Space, VMware Datastore space provisioned, VMDK space and define summary reports on what you need.