VMware Cloud Community
MurphyMoif
Contributor
Contributor

Slow Replication is slow.

Hello,

We're trialling SRM 5 and we're having some major headaches with speeds. We have two sites connected via a dedicated 100Mb line. We've been using Veeam in the past which has used the full available bandwidth without any problems but we're finding that SRM is only using a 10th at most. In fact, right now as I type this, we're sending over a single VM to our DR site via SRM and it's only using on average 1000KBps. When we first installed SRM we were getting the full 100Mb but now it's less than a trickle, what gives? The line is dedicated so there's minimal usage, VMs are version 8 and latest VMWare Tools have been installed.

We know the network is good as we can copy and paste files between the sites with no speed issues at all. Switches, VPN and routers have all been checked at least twice. It's not a problem there.

We have an SRM 5 server running SQL 2008 Standard at both sites, a VRMS server at both sites and 2 x VR servers at both sites. We're using replication as opposed to Array Manager.

Any ideas? What are we missing?

Tags (2)
Reply
0 Kudos
5 Replies
harshahosur81
Contributor
Contributor

what sort of Network IO control do you have on your hosts? May be apply Network IO for the VM replication on the vDSwitch level. Also check if there is QoS implemented on the link and what it is configured to provide QoS to. If not you can enable NIOC for vSphere Replication.

i wrote a simple blog post about it you can see it at

http://harshahosur.blogspot.com.au/2012/04/network-io-control-for-vsphere.html

Harsha Hosur. VCDX #135; MCTS; CCA
Reply
0 Kudos
MurphyMoif
Contributor
Contributor

We only have an Enterprise license so this isn't relevant to us, unfortunately.

We've checked all of our vNICs again and can't see anything that stands out. Just for the hell of it we've recreated our SRM setup and whereas before we had the full 100Mbit and it bottomed out at 1000KBps, this time it's come in at 1000KBps straight away and now it drops to around 500KBps.

It certainly does feel like some kind of QOS is in place.

Reply
0 Kudos
MurphyMoif
Contributor
Contributor

After further testing it seems that replication to datastores on iSCSI seem to be fine but if we replicate VMs to datastores attached via Fibre then that's where we see the throttling.

Investigations are ongoing...

Reply
0 Kudos
harshahosur81
Contributor
Contributor

Interesting, keep us posted on what the findings are. I would have assumed that FC would have been given higher QoS than other data traversing the link.

Harsha Hosur. VCDX #135; MCTS; CCA
Reply
0 Kudos
kwerneburg
VMware Employee
VMware Employee

It's possible, if you're only using one VM, that you're running into queuing issues on the single VM.

VR is designed to optimize a whole bunch of VMs and therefore buffers to a maximum size of 8k blocks as determined by the advanced settings.


Specifically, HBR.TransferDiskMaxBufferCount and HBR.TransferDiskMaxExtentCount.

These two will in some cases dictate how much data gets sent for a single VMDK at any point in time.  Obviously as you start replicating more VMs the throughput will go up if these maximums are what is limiting you on a single VMDK. 

The fact that you're seeing differences at the storage subsystem may indicate this is something totally different, but I wanted to make sure you understood that performance of a single VMDK replication may not indicate performance of VR as a whole when you start sending lots more disks across the wire.

Also... please don't change those settings unless you've talked extensively to support about the problem!  I don't think we actually support changing those parameters just arbitrarily.

-Ken

| @vmKen | VMware Technical Marketing |
Reply
0 Kudos