NetApp Scaling?

[vmrulz]

We just performed a POC using a couple NetApp 3170 filers. We setup thin provisioned FC connected volume's/LUN's both local to the NetApps and some on one of our EMC clariion arrays (fronted by the netapps). We are planning to buy these devices which also include their spendy PAM cards to allow for improved READ performance.

Our goal is to stop the bleeding when it comes to the continuous provisioning of FC storage for our burgeoning VMware infra. We plan to go through a lengthy migration where by we migrate everything on one clariion to thin provisioned netapp storage (both netapp and the clariion) all FC based. We'd like to do NFS at some point but our 10GE infra is lacking to say the least.

Our hope is to be able to de-duplicate most if not all of our VM's (after the perfunctory disk alignments as required) to the tune of 1400 VM's and growing. I've not been able to get a clear picture on how well the netapp's scale and whether we can consider de-duping everything including a large number of production VM's. We are on 4.1 vCenter and a mix of 3.5 and 4.1 ESX right now.

The netapp documentation mentions creating dedicated VM swapfile LUN's to improve de-dupe percentages. Has anyone verified whether this is helpful and what kind of performance impact if any they are seeing with potentially hundreds of VM's sharing a LUN for swap?

Thanks for any best practices, dirt and what have you that anyone can share.

Ron

[idle-jam]

check this out http://blogs.netapp.com/virtualization/2009/07/new-tr3749-netapp-vmware-vsphere-best-practices.html

it's a bit dated, but you get the idea 😃

iDLE-jAM | VCP 2, VCP 3 & VCP 4

If you found this or any other answer useful please consider the use of the Helpful or correct buttons to award points

[vmrulz]

Thanks Idle.. I've gone through that good whitepaper over that past couple weeks. Hoping for some unbiased real world now.. :smileygrin:

[FredPeterson]

Probably going to depend upon how much CPU activity is going on - can't really answer that until it starts in with your environment.

[mlubinski]

tr3749 should be your bible

the last comment on separate lun for swap is no longer valid, so don't bother it.

and think if you will really use 3170, as we were told while considering new storage systems based on both 3160 and 3170 that, with 3170 Netapp prices more software, and in general we would have to pay around 100k EUR more for software than for the same software with 3160. And the main difference I found between these heads was cache (3160 with 8GB per controller, in HA 16GB, 3170 16GB per controller in HA 32GB total), so we were considering only 3160 series.

Now they rerfreshed their products with 3200 series, so you can verify how it looks like

If you found this or any other answer useful please consider the use of the Helpful or correct buttons to award points

[vmrulz]

So the "Bible" says that we should use separate swap file luns, yet you're saying it is not valid any longer to use them. Can you elaborate on that comment?

Thanks for the comments

[RParker]

Our hope is to be able to de-duplicate most if not all of our VM's (after the perfunctory disk alignments as required) to the tune of 1400 VM's and growing.

FC requires LUN, LUN's are limited to 2TB, Deduplication Netapp is PER Volume. Remember that, it's NOT Aggregrate wide.

So your dedupe numbers won't be as good as you THINK, trust me, I know. We are getting around 40-45% (MAX) deduplication on Netapp, if we were using NFS it would probably be better.

[RParker]

So the "Bible" says that we should use separate swap file luns, yet you're saying it is not valid any longer to use the

Not sure which "bible" you are reading but like the original BIBLE it's written in 1000 languages, and there are 100 different versions of the so called "bible", so there is a LOT of interpretation.

Swap file LUNS are a waste of time. If your disks are configured right, and Netapp is configured correct, you don't need to separate LUNS for anything.

I have done many a benchmark, the overhead alone MORE than negates negligible performance gains (but mostly they don't exist)

[vmrulz]

Parker,

Thanks for the comments I've seen you weigh in on several NetApp conversations here. I was referring to a prior comment by another user "tr3749 should be your bible :)" regarding the bible comment.

It will be interesting to see what our reseller and NetApp guys have to say about these questions.

Are you guys doing much with NetApp Backup/Recovery? We're hoping we can eliminate the use of VMware snapshots.

Thanks again,

Ron

[RParker]

Are you guys doing much with NetApp Backup/Recovery? We're hoping we can eliminate the use of VMware snapshots.

I assume you mean Netapp Snapshots, and no we aren't using that. They use it for some production stuff but not where VM's are involved.

the VM snapshots allow you to gain access to the files, while they are in use, so you really can't totally eliminate the use of VM ware snapshots. Netapp reserves (and wastes) a lot of space for the sake of snapshot, clone, and volume copies. You have to snapshot an entire Volume, but what if all you need is a few VM's backed up? You snapshot an entire volume and reserve that space. I would rather backup from within VM ware infrastructure, than let Netapp control it.

I am sure Netapp could argue this, because that means another license for them. Vizioncore and Veeam have a different perspective, you get more direct approach. Netapp is getting better, they still have not convinced me they are ready to do this right.

[mlubinski]

Well I just checked this tr, and indeed they have old info in there. I followed this TR in every aspect to avoid problems when getting Netapp support on the line. But I know from where I said so (about swap). Here it is: NFS best practices

and especially:

Some early adopters of VI3 on NFS created some best practices that are no longer viewed favorably. They are:

1. Turn off the NFS locking within the ESX server

2. Not placing virtual machine swap space on NFS storage.

Both these have been debunked and the following section provides more details as to why these are not no longer considered best practices.

And I would really recommend following up this TR document, especially with alignment (MUST HAVE) and network settings (flow control). I know this, as I went through it with Netapp Support, and I never got valid help from them when I had performance issues. Always they said: ow you have misaligned VMDK files, or ow, you don't have flowcontrol enabled on network switches, so first correct this, and then we can speak (so lame from them).

In regards to your question about Netapp snapshots - we use them, and I can tell you that they are different than VMware snapshots. Just whole idea is different. In VMware snapshots, you will use them as you want to upgrade application, instal patches etc, so you can easily rollback to previous state. VMware snapshots are also "dangerous", as if you will forget to delete snapshot from VM, and will keep it for weeks, it will just possibly make your VM not accessible when trying to delete such snapshot (and also it will take a lot of time to process that). That one does not apply to Netapp snapshots, which can be kept for weeks/months, and nothing happens to volumes or VMs itself (apart of growing in size)

Netapp snapshot can be taken as non crashconsistent backup solution, as you have point in time snapshot of your volume including ALL your VMs. I am speaking about NFS volumes, but that applies also for iSCSI (just recovery is different). With NFS you can easily restore your VMDK (single) files or complete VM folder via vCenter gui, or you can speed it up by using SnapRestore (if you have license).

If you found this or any other answer useful please consider the use of the Helpful or correct buttons to award points

[mlubinski]

It depends (with dedupe ratio) how you place VMs. If you place on one datastore (either NFS or iSCSI/FC) only (the same) OS vmdk, and you enable dedupe on it (so you would have only w2k8 OS vmdk files on datastore etc), then your ratio will be like 80-90%. The best thing about NFS is that you see additional space after dedupe (which does not apply to LUNs). It means that if you have like 1TB lun with 900GB used space, and after dedupe it shows you like 90% saving, it allows you to provision more luns within this volume, but you will not be able to put more VMs than 1TB of assigned. If you have the same on NFS, then you will be able to place 1.9TB on 1TB volume.

and BTW, I have one iSCSI lun with size of 1.8TB and it contains the same OS VMs (w2k8) and deduplication ratio for this lun is 80%, so it means that it is physically using 300GB.

If you found this or any other answer useful please consider the use of the Helpful or correct buttons to award points

[mlubinski]

I am sure Netapp could argue this, because that means another license for them. Vizioncore and Veeam have a different perspective, you get more direct approach. Netapp is getting better, they still have not convinced me they are ready to do this right.

Netapp has their own product called SMVI (Snap Manager for Virtual Infrastructure) that integrates with vCenter, and allows you to backup single VMs as well.

I don't know which solution is cheaper (depending on number of hosts), but SMVI price (listprice) is (from storagemojo)

SW-T4C-NS-SMVI

SnapManager VI Bundle,SW,NetApp Select,T

SnapManager VI Bundle,SW,NetApp Select,T4C

$10,430

and Veeam/Vizioncore are priced per physical CPU (like 400€ per 1 CPU or something like that) so if you have like 20 ESX hosts with 2x Quadcore each, then you would already have to pay way more than that

If you found this or any other answer useful please consider the use of the Helpful or correct buttons to award points