Planning for big setup

[siranar]

Hi Everyone,

We are running one of the bigger dating sites on the internet, and are currently around the 200th most visited website worldwide. We have over 1 million unique hits per day, and about 120 servers.

I am considering moving to a fully virtualised infrastructure, based on blades and centralised storage. After reading a lof of information we are heading for the direction of dell/netapp. More precisely we are considering

buying 10 netapp blade chassis with 160 blades in total. I am also considering running vmware on systems that are 100% loaded (we would be running just one VM on one ESX). I would love to hear some feedback about

doing this and if its a good or bad idea. The extra price doesn't really matter - its the comfort and managability that does. Ideally we are going to make all 160 blades exactly the same - dual xeons with 16 GB ram (ram is cheap

nowadays). I will shortly explain what we have right now and what we require:

mysql servers: 20 dual xeons with 4 15k rpm drives configured in raid 0, they are mostly at 100% load, and so are the disks (we need to get more servers)

webservers: 40 webservers, dual xeons with 2 15 k rpm drives, plus an NFS share which they are all connected to - moderate load

30 random machines which are running various things

6 machines that are running load balancers to spread the load for the above mysql and webservers

To repharese my main questions are:

Running vmware one on one - yes or no? (price isnt an issue)

We are getting pricing for a netapp system with about 200 disks (15k rpm drives) - should we use NFS for vmware?

Any remarks on the above and/or recommended best practices? If you think netapp/dell/nfs is a bad choice please let us know. What we really

want is flexibility so that when there is more load on the site and when we grow, that expansion is a breeze. Also, rebooting and debugging remotely is a must

since driving to the datacenter is a waste of time and resource.

Thanks in advance!

[siranar]

We have had a bakeoff with IBM, HP and DELL in terms of servers, and IBM, HP and NETAPP in terms of storage. However we only did tests with running mysql over SAN/NAS. There are

differences in results, but the problem was that they all delivered a minimum amount of disks - so we could not test it fully the way we wanted it.

[williambishop]

Could you share the models you tested?

[Texiwill]

Hello,

THey should be able to bring in systems that have what you need in order to fully test functionality without needing to purchase the equipment. Or provide a location that does have the capability. I know IBM and HP have centers for such higher end testing.

I to have not heard any issues with NetApp.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[siranar]

I will get the test results for you in a zip file, together with the layouts we made. However, since each vendor had different amount of disks you need to take this with a LOT of salt.

What I can give you right now is the quotes I have received (after negotiating hardcore discounts as usual ) - I have removed the name of our company to protect our confidentiality.

I would like to know if these are good prices and if this is a good setup (features, compatibility). It turned out that the extra cost of having fibrechannel enabled on the 160 machines is about 60k euro (about 90k USD).

Attached the quotes, but I will give a short breakdown.

Dell quote totals for 10 chassis, 160 blades with dual quad core xeons, 32GB ram, dual 15k rpm drives, qlogic adapter on each blade and redundant fibre channel switches in each chassis + 2 external fibre channel switches and PDUs for the racks: 514k euro (760k USD)

This comes down to about 3212 euro per server or 4750 USD

Netapp quote for FAS3140 redundant version, 14 shelves with a total of 196 disks 15k rpm 144GB, and an additional 7 TB of sata storage: 167k euro (247k USD)

Not included: vmware licenses and 32 cisco 3130 blade chassis switches + bigger cisco to integrate everything on the network level

Ready to hear your guys suggestions about the setup. Quotes are attached (removed confidential info)

[williambishop]

No one else seems to have responded(and for some reason email alerts to topic aren't running for me anymore)...so, looking at them, my personal pick would be the netapp build. That's a fast monster. I do think you can get that price down a bit if you wrangle, but I'm not familiar with how well that works in europe vs. the U.S.

[RussellCorey]

We're actually doing a 1:1 virtulization gig right now using NetApp NFS (NFS is generally plenty fast for most applications) for ~400 physical servers. One of the things they're looking for is platform abstraction.

In regards to buying NetApp; catch them at the end of the quarter and you might get a better deal.