Clearspace Server Syndication Feed

How to Determine Bandwidth Usage

Virtual_JTW — Wed, 25 Nov 2009 18:51:35 GMT

I gave a presentation at VMworld 2009 on my SRM implementation. Part of this presentation included a discussion on WAN requirements and, more specifically, how to determine the amount of bandwidth you're going to need to replicate virtual machines between sites.

I have decided to wrap-up the main points of this discussion in this article for those that missed the session and for future reference in general.

I've summarized the process in three steps:
Step 1: Determine the size of data that needs to be replicated
Step 2: Determine the change rate
Step 3: Crunch the numbers

Determine the Size of Data that Needs to be Replicated
For SRM, you're going to group protected VMs on the same LUN or LUNs. View your datastores and add up all of the disk in use (i.e. utilized by VMDKs, VMXs, logs, etc). This will be the total size of data that needs to be replicated

Note that the first replication is what I call the "seed" copy as it will replicate all of the data. Most modern replication technologies will replicate only the changes or deltas from that point forward. Because this can be quite a sizable amount of data sometimes adding up to a terabyte or more, I highly recommend replicating over a LAN connection first. I had the good fortune of having both of my SAN nodes in the same location for several weeks. This allowed me to setup replication between the two and perform that first seed copy while connected via high-speed LAN links. If you don't have this luxury, some SAN vendors provide a facility where by you can use alternate media such as tape to perform the first copy. Either way, please keep this in mind as it could dramatically impact the success of your efforts (i.e. replication could consume large amounts of bandwidth causing production applications to fail, it could run for a long time causing doubt that it will ever finish successfully, etc, etc).

Determine the Change Rate
This will be the hardest part for some storage administrators. You'll typically want to determine both daily and peak change rates. There are several different ways of determining your change rate. Here are a few examples:

Obtain from incremental backups
Utilize tools provided by your SAN vendor
- If you haven't purchased your SAN yet, ask SAN vendors if they have tools that will help you manage and monitor bandwidth utilization
Utilize third-party WAN (or LAN) bandwidth monitoring tools
Perform several replications overs at least a full weeks time and measure the results

I used the last two methods to determine our daily and peak change rate. And after moving the second SAN unit to the remote site, we continued measuring the WAN bandwidth utilization over two weeks time.

Crunch the Numbers
Now that you're armed with some raw data, it's time to try to make sense of it all. The general formula we'll use to calculate the change rate is:
(GB change/day)/(GB total size) = % change rate

So, for example, if we had 56GB of data change in one day out of 617GB total, the change rate would be 56/617 = 9% per day. You'll find that a 9% change rate is not that bad. From this you could surmise that you'll need a fractional DS3 (about 6 DS1s).

Alternatively, you could use an "industry average" change rate of 20% and apply that to the total size of data you discovered in the first step. I have seen this figure used as a general guideline in documents from multiple SAN vendors over the years. Hopefully it's overkill for your needs but it's better than nothing. Using this rate on same same example:
20% * 617GB = 124GB/24hrs = 5.2GB/hr = about 12 DS1s

As you can see, this will cause you to provision roughly double the amount bandwidth that you really need.

Caveat Emptor
Please note that every environment is different. Consider these calculations to be estimates! These rates are never linear. For example, after all is said and done you may add several VMs to your replicated datastores dramatically increasing the total size. Consequently, I recommend revisiting your calculations on a regular basis and adjusting accordingly. Use a good network monitoring tool such as NetFlow Analyzer to help keep you on top of bandwidth utilization.

One final note, my WAN utilization is made up of several consumers - SAN replication, Active Directory replication, corporate applications such as warehouse inventory tracking, etc. Make sure you account for these other bandwidth needs and provision WAN links that meet your total requirements. We also added 30% to our total bandwidth requirements to allow for growth and room for some additional utilization that we knew we would need within the next year.

This certainly isn't the final word on the matter so please feel free to post your experiences on determining WAN bandwidth requirements.

Tip: EMC Celerra NS350 iSCSI and vSphere 4 SAN HCL

Virtual_JTW — Mon, 23 Nov 2009 21:50:18 GMT

As part of my vSphere 4.0 upgrade planning, I checked VMware's HCL for all of my hardware. I noticed that VMware's SAN HCL for vSphere 4 includes the Celerra NS350 for NAS but there's no listing for iSCSI unlike many other Celerra models that explicitly state "iSCSI" and "NAS".

Good News: I opened and SR with EMC support who verified that the Celerra NS350 is indeed supported as an iSCSI target for vSphere 4 just as it is for VI 3.5.

For future reference, one can verify this by using the e-Lab Navigator tool found on EMC PowerLink.

VMware TAM Weekly Newsletter Edition 1.13 - vSphere 4 Update 1.0, View 4.0 Available for download and loads more

nisserow — Mon, 23 Nov 2009 08:47:40 GMT

Lots of news, reviews and more in this weeks newsletter, please enjoy.

Kind Regards

Neil Isserow | VCP3 | Technical Account Manager - Queensland |
VMware Australia |

vSphere PowerCLI 4.0 Update1 リリース

kkomatsu — Sun, 22 Nov 2009 23:27:55 GMT

vSphere 4.0 Update1 にあわせて、vSphere PowerCLI 4.0 Update1 もリリースされた。大幅な機能拡張がなされている。
http://www.vmware.com/support/developer/windowstoolkit/

新しく提供されたコマンドレットによって例えば下記のようなことが自動化できる

ESX自体のパワーオン・オフ、パッチの適用が可能に
仮想マシンクローン時等のカスタマイゼーションでIPアドレス等の指定が可能に
ゲストOSのネットワーク設定の制御可能に

また、既存のコマンドレットの拡張によって、下記のようなことにも対応できる

仮想マシンをThinディスクで作成可能に
仮想マシンのライブクローン(パワーオン状態の仮想マシンのクローン)が可能に
仮想ディスクのオンライン拡張が可能に

一方、vNetwork Distributed Switch の構成・管理への対応は見送られているようだ。VMworld の Break out セッションでは、VMotion や FT、NFC トラフィックを Parse する Wireshark プラグインと並んでデモもされていただけに残念。これら、PowerCLI で対応していない機能については、先日も紹介した Get-View を使って、vSphere SDK for .Net のレベルで記述することになる。

なお、同じタイミングでPowerCLI コマンドレットのオンラインリファレンスがついに提供された
http://www.vmware.com/support/developer/windowstoolkit/wintk40u1/html/index.html

MAP-1.1 Mathematica Platform

focaccio — Sat, 21 Nov 2009 20:04:29 GMT

Notations: A-Action, R-Result, Q-Question, C-Cognition/Commentary, X-Exploration research

A MAP-1.1.1 Completed the Mathematica installation.

R MAP-1.1.2 Mathematica installation completed successfully. As far as I can tell at this point Mathematica 7 for Students runs well on an Ubuntu Server 9.10 virtual machine in the ESX Server 3i 3.5 Build 153875 environment.

virtualization.info Vanguards on LinkedIn

JohnTroyer — Fri, 16 Nov 2007 03:17:38 GMT

We last saw our intrepid Web 2.0 adventurer about to delve back into the depths of LinkedIn. LinkedIn is old school, one of the oldest business social networks, and it's not about virtual martinis or superwalls or how many movies you have in common -- it's all about the networking.

I will admit it's been a while since I've been there -- probably since the last time I was looking for a job, which seems to be one of the major use cases for LinkedIn. The principle activity of LinkedIn, aside from the meta-activity of increasing the size of your personal network, is the Introduction -- asking your network to hand your referral from person to person until you reach the object of your affections -- Bill Clinton, a hiring manager, a prospect. I've never sent or received one of these invitations, but I do know folks that have gotten plenty of job inquiries from the site. My profile on LinkedIn is here, and no, I'm not looking for a job.

Spurred on by the current social networking frenzy, they are adding features like a Q&A section and beefier profiles. Here's Bernard Lunn of Read/Write Web on how he recently used LinkedIn and his perception of its business value vs Facebook's.

After logging in and taking care of some pending connection requests, I joined Alessandro Perilli's virtualization.info Vanguards Group (not to be confused with VMware's Virtual Vanguard Awards).

After only a month and with a single announcement, Alessandro has assembled 383 virtualization professionals from across the globe. Not bad! So why do you want to be there, even if you're not looking for a job?

A wide cross-section of the virtualization industry. There are vendors (from VMware and Microsoft on out), consultants of all stripes, very experienced sysadmins and IT experts, and quite a few names you may recognize from communities.vmware.com.

You can see and contact everybody in the Group. The contact piece is configurable on a per-group and per-person basis, but Vanguards is set up by default so that we can all contact each other. Interested in finding a virtualization consultant in Norway? Looking for a contact at a vendor -- either the executive or the engineering kind? Want to compare notes with someone else in your industry? You can probably make that happen here with a quick search.

You won't be spammed. Now, since this is a business network, many people have something to pitch, so LinkedIn groups are not built for spam. In a LinkedIn Group you can contact individuals, but nobody in the Group can globally spam everybody in the group with a pitch for their latest virtualization management appliance.

LinkedIn is still very much a business-to-business network, and so what a Group can do there is still very buttoned-down and oriented at making business contacts -- LinkedIn doesn't actually offer much more to do with Groups yet. I suspect that we'll see other functionality soon: for instance, you can ask a Question to your network and the LinkedIn community at large, but I'd love to see questions just from the virtualization.info Vanguards Group. Although the Groups feature on LinkedIn is at least two years old, they've only recently opened up and become easier to create.

So you can't go wrong adding the virtualization.info Vanguards Group on LinkedIn to your professional online presence. I think since we have such a vibrant community here, I'm not going to start up a LinkedIn group specifically for VMware, but let me know if you have other good ideas.

Virtualization reddit: come on in!

JohnTroyer — Fri, 25 Jan 2008 05:50:34 GMT

There are many social media sites out there. Most of them can be interesting if you have something in common with the crowd that hangs out there. Digg often gets most of the press, but I've always been more partial to reddit.

Reddit works in a similar way to Digg -- people submit items, and everybody votes them up or down. In theory, the most interesting items bubble to the top. Also in theory, as you rate items up or down, the system learns about your interest and starts to show you recommended items.

The reddit crowd has always been a little geekier and a little more interesting -- a bit of lisp, a bit of web culture, and sometimes a funny picture. A quick dip into the programming reddit now and then will help you carry on the conversation at your next party when Erlang or closures come up. (Hmm, I may be going to the wrong parties.)

Reddit just opened up a beta feature to create new topic-specific reddits and I'm very pleased to announce:

virtualization reddit

Virtualization reddit is the place to read news and commentary about virtualization, all chosen by the virtualization community. VMware, Microsoft, Virtuozzo, Xen, whatever. Go ahead, create an account, submit your favorite news article or blog post on virtualization, and rate the others.

I've submitted a few articles, but one man does not a social media site make. Come on in, add your two cents, vote up the most interesting articles, and have fun. Then check back every day to discover today's must-read articles about virtualization technology and the virtualization industry.