We have upgraded to vSphere (full) and we're seeing a drastic reduction in backup speeds. Where guest backups would previously report 50-70MB/s, we're now seeing things top out at 25MB/s, even for the followup incremental backups.
We struck it up as a fluke on the first datacenter in our main facility, but we're now seeing the exact same performance after installing in our second datacenter.
We're using Veeam Backup 3.1, installed on a fresh Windows 2008 64-bit guest with 2 CPU and 2GB memory. It was previously installed in a Windows 2008 32-bit guest.
The physical server has 8 2.83GHz CPUs, 32GB memory and is showing approximately 20% host utilization at the time backups are occurring.
The hosts are connected via 4GB/s FC to an 8-drive EVA 4400 through a Brocade 200E switch. The backups are writing to a 16-drive RAID50 Samba share. The Samba server isn't experiencing high CPU loads, we're seeing about 6% utilization there during the backups.
The vSphere Service Console is on vSwitch0. The vSwitch has a Service Console port and a VMKernel port. The vSwitch is connected to 2 physical adapters, both Broadcom BCM5708 gig NICs. These are the same NICs that were used under ESX 3.5 and Veeam Backup 3.0.1 with great backup performance.
Have checked Ethernet NICs and switch ports, no errors. Have checked FC ports and switches, nothing odd there either.
The vSphere installations on the two hosts were clean installs, not upgrades. The vCenter install was also on a fresh Windows 2008 64-bit guest. It was previously on a Windows 2003 32-bit guest.
The guests were all updated with the new VMware Tools, part of them were upgraded with the newer VM Hardware. Old or new VM Hardware doesn't seem to make a difference in the backup speed.
Testing the SAN and Samba server via IOmeter get us performance that we'd expect, much higher than we're seeing via Veeam backups.
We have tested backing up via a Samba share (\\server\share) and doing "direct" to the Linux Samba server. Both perform equally as bad.
Have checked the Veeam forums for possible problems, but came up empty. Have watched these forums for the past few days to see if others were seeing this, but didn't see anything there either.
Sort of at the end of my rope as to what is causing this drastic slowdown. A 330GB guest is now taking about 4.5 hours to backup and that's REALLY ridiculous... Sorry for the long post, but wanted to provide the hardware and install steps along with the troubleshooting we've attempted so far.
Would love to hear any suggestions for how to narrows this down and resolve it!
We struck it up as a fluke on the first datacenter in our main facility, but we're now seeing the exact same performance after installing in our second datacenter.
We're using Veeam Backup 3.1, installed on a fresh Windows 2008 64-bit guest with 2 CPU and 2GB memory. It was previously installed in a Windows 2008 32-bit guest.
The physical server has 8 2.83GHz CPUs, 32GB memory and is showing approximately 20% host utilization at the time backups are occurring.
The hosts are connected via 4GB/s FC to an 8-drive EVA 4400 through a Brocade 200E switch. The backups are writing to a 16-drive RAID50 Samba share. The Samba server isn't experiencing high CPU loads, we're seeing about 6% utilization there during the backups.
The vSphere Service Console is on vSwitch0. The vSwitch has a Service Console port and a VMKernel port. The vSwitch is connected to 2 physical adapters, both Broadcom BCM5708 gig NICs. These are the same NICs that were used under ESX 3.5 and Veeam Backup 3.0.1 with great backup performance.
Have checked Ethernet NICs and switch ports, no errors. Have checked FC ports and switches, nothing odd there either.
The vSphere installations on the two hosts were clean installs, not upgrades. The vCenter install was also on a fresh Windows 2008 64-bit guest. It was previously on a Windows 2003 32-bit guest.
The guests were all updated with the new VMware Tools, part of them were upgraded with the newer VM Hardware. Old or new VM Hardware doesn't seem to make a difference in the backup speed.
Testing the SAN and Samba server via IOmeter get us performance that we'd expect, much higher than we're seeing via Veeam backups.
We have tested backing up via a Samba share (\\server\share) and doing "direct" to the Linux Samba server. Both perform equally as bad.
Have checked the Veeam forums for possible problems, but came up empty. Have watched these forums for the past few days to see if others were seeing this, but didn't see anything there either.
Sort of at the end of my rope as to what is causing this drastic slowdown. A 330GB guest is now taking about 4.5 hours to backup and that's REALLY ridiculous... Sorry for the long post, but wanted to provide the hardware and install steps along with the troubleshooting we've attempted so far.
Would love to hear any suggestions for how to narrows this down and resolve it!
