Why use VCB over esXpress or vRanger Pro?

[jfields]

Hello,

I know this topic has been covered before to some extent, but I had a more specific question that I don't believe has been addressed. I have VCB, as part of Enterprise edition, and wanted to know if anyone is using it with iSCSI SANs. Also, I have read VMWare's literature on this, but a lot of it is marketing. In real world terms, if I have access to both my SAN and LAN(s), should I select VCB (with Backup Exec 11d) over a product like esXpress? How important is the quiescing feature that is part of VCB? The cost of the two solutions is roughly the same for us.

We have a primary site with two ESX hosts and a SAN. I am on an University campus, so the connection between our two sites is not private. Both sites will be protected with enterprise firewalls. It is necessary to get backups to the secondary site for DR purposes.

Option 1- Install a Win 2003 with VCB at primary site. This is necessary because our SAN infrastructure only exists in our primary data room. Run backups with VCB during business hour and then SSH (for security) backups to remote site overnight.

Option 2- Install esXpress on both ESX hosts. Run backups at night over the public connection using SSH for security to the secondary site.

Which option would you recommend? Thank you for reading my lengthy question and helping me.

JF

[dconvery]

The good thing about BackupExec is that it allows you to put the backups on tape for offsite recovery. If your DR site is far enough away to avoide floods, tornados, nuclear warheads, locusts,fire, brimstone, etc. Then esXpress will work nicely. Take a look at Vizioncore vReplicator too.

Quiescing is very important when dealing with open files. VCB will also give you the ability to run a pre-freeze script before the quiesce (net stop mssql) and a post thaw script after the snapsot is taken. It gives you a more consistent backup.

Dave

[petedr]

Hey jfields.

I'll give you the vendor perspective as esXpress is my companies product but before that I was a user running a virtual Oracle/Linux environment. It is a different approach to VCB as we use a virtual backup appliance (VBA) to perform the backups. One big difference is you don't need the VCB proxy server as esXpress can send the backups directly to your other esx host (ssh as you suggested will work fine), you could even our new simple replication if you want the second host to be a warm standby. If you do have a requirement to write to tape most of our customers do use an external ftp or backup server for that. One thing esXpress can offer you is Delta backups that I don't think your option 1 can provide. If you just want full VM backups we give that away for free.

On quiescing it really depends on what your requirements are in a backup solution. For some it is very important, for other not as much, there is a lot of discussion on vmtn about this topic. When I ran my Oracle environment it was not a requirement. A crash consistent backup was fine for us. With those backups I had never had an issue on on any restore I did.

Good luck with your evaluation, the best is to try both solutions and see what works best for you.

Pete@esxpress

[jfields]

Thank you both for the info. I should mention that I do already have two licenses for Backup Exec 11d (server and 1 agent), but I bought those before I bought VCB. I hadn't planned for it, but now I am wondering if it makes sense to use that. Since I have the server license, I could install the BE server and the VCB on the same server. Unfortunately, I suffer from a severe space issue at my main site. Therefore, I have do not have any more rack space for a physical machine. What do you think of running Backup Exec 11 and the VCB in the same VM? We have a very small install, so I am not really concerned about performance scaling and such. Just if it would actually work. I guess I would then have to SFTP or SSH the backups to my remote site ovenight via shared LAN (servers not really used at night)?

I started this thread because I was principally confused (and still am) about the reasons why someone uses one of these backup solution over the other. Don't esXpress, VCB, and vRanger all basically offer live VM backups to remote machines? If I need to backup my VMs daily and also make sure they get offsite, what is the best solution? Note that I would probably not spring for the Enterprise version of esXpress as it is the most expensive option ($2000+). We would be comparing esXpress Pro or lower and vRanger and VCB. I know this is a complex issue, but I feel I need help with this. I have read tens of forums posts on this and still am confused. Thank you much!

[dconvery]

You can definitly install BE and VCB on the same server. It cannot run in a VM because it needs to connect to the VMFS LUNs via a SAN connection. VCB cannot run on the VC server because of disk mount driver issues. Take a look at http://www.vmts.net/vmbk.htm. It is a script that will run on the console. You can add to it to scp your backups to offsite.

vRanger, esxpress, etc. will accelerate the process and add compression. They also assist if things like DNS are not set up right. They can run stand alon to back up over the network. When you couple vRanger with VCB, it moves the backup off of the LAN and on to the SAN.

That being said, you could certainly use BE and VCB on a physical machine and script an scp to the offsite. If space is an issue, don't forget you can convert the physical systems to VMs and use the physical for VCB. If you can't do that, check out vmbk.

Hope this helps...

Dave

[jfields]

Dave,

Thank you. Unfortunately, I have no more space for any machines over there. I did not know you couldn't run VCB on a VM. That is good to know. If I need a separate physical machine to run VCB, then it may not be an option for me. I guess I can always do esXpress and vRanger without VCB and send the backups to a remote site via the shared LAN connection over SSH. VMBK looks interesting, but I think I would prefer to use a program that gives me a better view of what is going on and reporting features. Both esXpress and vRanger have very nice graphical interfaces. It seems a shame I won't be able to use VCB and BE together, since I already have both licenses.

JF

[kix1979 · Accepted solution]

You can certainly install VCB in a VM, it will just only support iSCSI at that time. If you wait for the 3i release, which supports NPIV, you can use a VM for Fibre channel as well, but since you are using iSCSI you are fine to use VCB in a VM today. I use this in my environment and I know customers that do it as well. As for vRanger Pro vs. esXpress, it depends on how you want to backup your VMs. vRanger Pro can leverage VCB or operate over the network vs. esXpress can only backup over the network. Beyond that I would really recommend that try both and see which works best for you in the environment.

Kix @ Vizioncore

[petedr]

Hey guys,

Just a correction on what was stated about esXpress. While it does support backups over the network ( ftp and ssh ) it also supports backups directly to vmfs ( either local or on a SAN ) which does not go over the network. In fact esxpress can send the backups to both the network targets and vmfs at the same time.

Pete@esxpress

[kix1979]

Hey guys,

Just a correction on what was stated about esXpress. While it does support backups over the network ( ftp and ssh ) it also supports backups directly to vmfs ( either local or on a SAN ) which does not go over the network. In fact esxpress can send the backups to both the network targets and vmfs at the same time.

Pete@esxpress

When sending to two targets, would it go at the slowest of the two speeds or would it be two seperate transports?

[kharbin]

Hi Kix,

It was good to finally meet you at this years VMworld.

As for the split transport, it operates at the slowest of the two speeds. But since we have such an efficient compression and fast delta engine it is not really an issue. A 100GB VM, even with 10% delta extraction will still compress to around 1-2GB. On a 100Mb network, thats about 1.7 to 3.4 minutes transfer time for what is essentially a 100GB VM (under 1 minute on GB net). While FULLs obviously take longer, by default we only have to do this once a month. And FULLs can be stagered by ESX host, VM, day of week, day of month, etc., the system can be set to "load balance" your FULLs so as to not overwhelm the SAN, network, or backup target, making esXpress ideal for remote sites that want to replicate their backup across the WAN.

esXpress holds both the speed and quantity records for backups, with some customers exceeding 565GB/hour per ESX host, and another that averages 5TB/hour across his entire farm (60TB in size) and both were done without adding expensive dedicated hardware. You can read more about that here:

And, currently esXpress is the only product that can not only perform networkless backups, but also networkless restores. While other products can do networkless backups, I believe all others perform their restores via the network. Correct me if I am wrong.

When dealling with tens of terabytes a night, slowest of the two speeds is irrelevant to esXpress. We make any transport mechanism fast, even if it's not.

Ken Harbin

PHD

[kix1979]

Ken,

Calculating speeds based on full size when doing a delta backup seems like shady math to me. Network speeds are based on the amount of data transmitted over the time it takes. You can't take a delta backup and calculate that as a full, it's not a realistic value. If you use that value you get values that exceed the MAX transmission speed of Gbit networks. The most data you can theoretically push over a Gbit network is as follows:

1Gbit/sec = 125MB/sec = 450GB/hour = 10.8TB/day

How many hosts are you saying you are getting 565GB/hour or 5TB/hour? At a minmum you would need at least two nics running at full speed with little to no overhead for 565GB/hr. For 5 TB you are talking about at LEAST 11 network cards that are pushing data will little or no overhead. This doesn't even address the issue that the target for this data would have to have the exact same setup with network cards as well as disk. What type of disk are they pushing 5TB/hour into? That is crazy amounts of Disk I/O on both ends, which would melt most SANs that I have seen in production, not to mention as a target. Then that also doesn't address the fact that writing data is slower then reading data...

In the case of the post you refer to, they are using 8 hosts. If they are actually doing 60TB of VMs, that means a full (even at theoretical max with 1 Gbit NIC per host) would take about 18 hours to actually complete. Even with 4 NICS, running at MAX speed per host on a full you are talking about 3 - 4 hours. Other then that, it's bad math if you are calculating based on deltas as above. Frankly, this is all based on speculation and math at the maximum tranmission level of a gigabit network and assuming no TCP/IP overhead and a host of other things which are not going to happen in any network anywhere on this planet.

Kix @ Vizioncore

[kharbin]

Kix,

Well first let me say I am not trying to start another Vizioncore - esXpress pissing match.

Your math: 1Gbit/sec = 125MB/sec = 450GB/hour = 10.8TB/day

The above is correct if you are running only full backups are they are 100% uncompressed (or using a proxy). But since esXpress uses compression,and does not rely on the proxy bottleneck, that 10.8TB compresses down to about 1.8TB (on average). And if you are performing deltas, that 10.8TB compresses down to about 600GB (toward the end of month when deltas are at their largest). With most esXpress customer averaging about 10 ESX hosts, thats only 60GB per host. Not a very big number. That's the power of distributed processing.

To base ALL of you math and ALL of your examples on uncompressed data transfers is not real world. Your numbers and examples imply esXpress does not perform any compression. This is absolutely false, untrue and misleading.

"Calculating speeds based on full size when doing a delta backup seems like shady math to me." I find this statement not only incredulous, but extremely insulting. Your statement implies an intentional attempt to lie and mislead this forum. I take extreme offense to this unprofessional personal attack against myself. To call my statement "SHADY" and then end with "Frankly, this is all based on speculation" proves your claims are without merit and that you are not willing to backup your claims with any real evidence, but instead only biased opinion and speculation.

Anyone can come to the site, download the software and prove it for themselves. Don't take Thomas' or my word for anything, either way. Prove it to yourself.

This is the last I will post on this subject.

Ken Harbin

esXpress.com

P.S.

We missed you on Thursday at VMworld. We stopped by the VC booth but you were gone. Did David give you the red Swingline?

[ronzo]

Here's a couple of real world backup examples, showing 207 GB/hour

average on Fulls, and better then 400 GB/Hour on deltas. Most people do

not get these speeds, but those with good hardware do. But most people

do average at least 100 to 200 GB/Hour. Now imagine you have 10 hosts

or 25 hosts getting these backup speeds.

2007-10-21 17:04:55a ALL TOTAL: 6 vms 12/12/12 disks, (100%) 342g/342g/342g, Act: 1h:37m:54s 59mb/s (207gb/hr) vs Vrt: 3h:01m:28s 32mb/s (112gb/hr)

2007-10-21 21:12:30o ALL TOTAL: 4 vms 21/21/21 disks, (0.1%) 735m/510g/510g, Act: 1h:11m:20s 122mb/s (428gb/hr) vs Vrt: 2h:12m:18s 65mb/s (228gb/hr), sent 295m

2007-10-21 21:32:27o ALL TOTAL: 5 vms 9/9/9 disks, (3.6%) 8.7g/240g/240g, Act: 31m:22s 130mb/s (457gb/hr) vs Vrt: 1h:07m:06s 61mb/s (214gb/hr)

thanks

ron

[RParker]

It can do either, or both at same time.

[RParker]

Yeah, what would I do to configure ESX to get these speeds? Assuming I have 1G to ALL NICS from the host to ftp, and I am using Intel NICs PCIe, not internal.

I installed a "trial" of esXpress without my license, to see what the performance would be, it doesn't seem to be higher than about 59 gb/hr.

1/1 disks, 400g, 6:29:17s, 17mb/s (59gb/hr) NET3

This is FTP (or maybe I didn't set something up right). No throttle limits, and this is a new install on a new host.

[ronzo]

Rparker-

The speed is usualy disk I/O bound, but without getting your backup logs, hard to know for sure.

Set the backup mode to TEST and run a backup, and you will get the max speeds in the VBA..

Even on my Dell 1950 test box with local storage, my fulls are 50 mb/sec and my deltas are 150 mb/sec (with 2 CPUs in the VBA) and about 80 mb/sec with 1 CPU in the helper.

FULLS

2007-10-17 16:33:44.539O TOTAL: 10 vms 12/12/12 disks, (100%) 139g/139g/139g, Act: 44m:29s 53mb/s (186gb/hr) vs Vrt: 1h:43m:36s 22mb/s (77gb/hr), sent 19.0g

DELTAS

2007-10-21 00:29:59.873O TOTAL: 10 vms 12/12/12 disks, (0.9%) 1.2g/139g/139g, Act: 28m:54s 82mb/s (288gb/hr) vs Vrt: 48m:32s 48mb/s (168gb/hr), sent 311m

The guy with the 400 GB/Hour is on a DMX 3.

thanks

ron

[kix1979]

Ken,

I'm not trying to start anything, I'm trying to understand where your numbers come from. I base my numbers on the amount of data that is actually backed up. This leads for two situations: Full and Deltas. In the case of a FULL, it means that all data is being backed up. This leads for the easiest situation for backup times/speeds. So if a VM is 10G in size and has 10G of data and takes 10 minutes to backup it would be said to backup at 1GB/min or about 17MB/sec. In the case of a delta backup it's no longer backing up all of the data, instead a small subset of blocks. In an uncompressed state let's say for that same VM that 1GB changed and that took 2 minutes. To me that says the backup was 1GB and took a rate of 500MB/min or about 8.5MB/sec. The alternative, which is what I think you are saying, is that since it essentially is delta of the entire full it should be 2 minutes for the delta or 5GB/min or 75MB/sec. Perhaps that is the difference we see in the math?

Also please note, I don't mention compression, but mearly what is possible over network connections. Of course, compression makes a difference in how much data can be pushed over a network, but all things aside I was speaking to the speeds of network and how much data, regardless of state, can be pushed over this. Even in a compressed state, GZIP or LZOP etc..., most data will compress down to about 3 - 5 times plus the truncation of whitespace which can amount to a size that is inconsistent across VMs for the purpose of average calculations. That said it still comes down to how you calculate the times. Again, this is why I am just trying to understand how you calculate times as above in the first paragraph. I'm not trying to start a fight here, there was no personal attacks and I did not call you out etc... so please take what I am asking as just that.

As for Thursday, I had to head out to Asia-Pacific on a red-eye. In fact I just got home after a 45 day stent away from home, and I head back to China next week. Ah the life of a traveller...

Thomas

[kix1979]

> {quote:title=ronzo wrote:}{quote}Rparker- > > The speed is usualy disk I/O bound, but without getting your backup logs, hard to know for sure. > Set the backup mode to TEST and run a backup, and you will get the max speeds in the VBA.. > > Even on my Dell 1950 test box with local storage, my fulls are 50 mb/sec and my deltas are 150 mb/sec (with 2 CPUs in the VBA) and about 80 mb/sec with 1 CPU in the helper. > > FULLS > 2007-10-17 16:33:44.539O TOTAL: 10 vms 12/12/12 disks, (100%) 139g/139g/139g, Act: 44m:29s 53mb/s (186gb/hr) vs Vrt: 1h:43m:36s 22mb/s (77gb/hr), sent 19.0g > > DELTAS > 2007-10-21 00:29:59.873O TOTAL: 10 vms 12/12/12 disks, (0.9%) 1.2g/139g/139g, Act: 28m:54s 82mb/s (288gb/hr) vs Vrt: 48m:32s 48mb/s (168gb/hr), sent 311m > > The guy with the 400 GB/Hour is on a DMX 3. > > thanks > ron

Ken,

This is what I was wondering about. The thing that I find different in my calculations and yours is that on the DELTA is that 1.2G is backed up and this is done in 28:54 for 288GB/hr or 82MB/sec. Since it was 1.2GB backed up, I've always based my calculations based on that size not the 139G size for example. That's the difference I think. 😄

Thomas/Kix @ Vizioncore

[kharbin]

Thomas,

Being that you work for a VMware based backup company, I thought by now that you would have an intimate understanding of full backups, deltas, compression and the associated math.

Rather than state you don't understand my math, or ask me to clarify my math, you instead call my post "shady" and imply deceict on my part, with absolutely no evidence except "speculation". This was not only misleading and unprofessional, but a personal attack on my credibility and character.

Ken Harbin

www.esXpress.com

[oreeh]

Message was edited by: oreeh

I'm now locking this thread.