VMware Cloud Community
DXK5314
Contributor
Contributor

Windows Server 2012 R2 Bad Performance

Hello,

I've a vSphere cluster running with 7x IBM HS22V blades / 2x Intel Xeon X3650 / 256 GB memory in every host.

Running at vSphere 5.5 23202651, with around 150 VM's.

Since a few months we are implementing Windows Server 2012 R2, and since a few weeks we've a new Citrix environment running at 2012 R2 to.

Currently the performance of the Windows Server 2012 machines is really really bad.

The respond slow and it doesn't feel right, the older VM's (W2K8 R2) are running great.

I've searched the internet, and I found that a lot of people had/have problems running 2012 on vSphere.

But I didn't found any solutions or things I could try to solve our problems.

Hopefully you guys have some tips for me, because I'm stuck...

Thanks in advance!

Best regards,

DXK

47 Replies
CoolRam
Expert
Expert

please do get the esxtop logs for this virtual machine and analyze those with standard value.

If you find any answer useful. please mark the answer as correct or helpful.
Reply
0 Kudos
Srinivasu13
Enthusiast
Enthusiast

Hi,

I agree with you even our 2012 are running slow in our environment.

Recommend you to verify below things on your 2012 VMs.

  • do you have VMware tools installed in your guest? do you use paravirtualized SCSI adapter with your guest?
  • host CPU usage - host -> performance tab -> advanced -> cpu - what is average CPU load on the host? is it over 75%? are there peaks over 90%? it may signal that host CPU is saturated.
  • guest CPU usage - host -> select the vm -> performance tab -> advanced -> cpu - the same like above plus check the CPU READY TIME, is it over 2000ms? again, the host CPU may be saturated.
  • guest memory usage - host -> vm -> performance tab -> advanced -> memory - if memory average is over 80% or peaks over 90% the guest may have higher memory demand and so it is another performance bottleneck.
  • guest memory swapping/ballooning - host -> performance tab -> advanced -> memory - check swap-in/swap-out and balloon rates. if it is over zero the host is swapping or ballooning vm memory which may make the vm sluggish as it requires access to disks.
  • storage overloaded - host -> performance tab -> advanced -> disk - check STOP DISK metric, if it is over 0 zero disks/LUNs may be overloaded. additionally, check read/write latencies, if it is over 10ms in average or over 20ms in peaks disks/LUNs are slow or overloaded.
  • Microsoft patches are up to date?
------------------------------------------------------------------------------- If you found this or any other answer helpful, please consider to award points. (use Correct or Helpful buttons) Regards, Srini
Reply
0 Kudos
DXK5314
Contributor
Contributor

"Hi,

I agree with you even our 2012 are running slow in our environment.

Recommend you to verify below things on your 2012 VMs.

 

  • do you have VMware tools installed in your guest? do you use paravirtualized SCSI adapter with your guest?
  • host CPU usage - host -> performance tab -> advanced -> cpu - what is average CPU load on the host? is it over 75%? are there peaks over 90%? it may signal that host CPU is saturated.
  • guest CPU usage - host -> select the vm -> performance tab -> advanced -> cpu - the same like above plus check the CPU READY TIME, is it over 2000ms? again, the host CPU may be saturated.
  • guest memory usage - host -> vm -> performance tab -> advanced -> memory - if memory average is over 80% or peaks over 90% the guest may have higher memory demand and so it is another performance bottleneck.
  • guest memory swapping/ballooning - host -> performance tab -> advanced -> memory - check swap-in/swap-out and balloon rates. if it is over zero the host is swapping or ballooning vm memory which may make the vm sluggish as it requires access to disks.
  • storage overloaded - host -> performance tab -> advanced -> disk - check STOP DISK metric, if it is over 0 zero disks/LUNs may be overloaded. additionally, check read/write latencies, if it is over 10ms in average or over 20ms in peaks disks/LUNs are slow or overloaded.
  • Microsoft patches are up to date?"

- Yes, up to date version.

- No, we don't use SCSI adapters.

- CPU load = 8369 avg --> Max = 12195, Min 4995, so I guess this is good. No peaks over 90%. --> imgbox - fast, simple image host

- I see multiple lines, the green one goes over 2000 Mhz. --> imgbox - fast, simple image host

- Guest memory --> http://imgbox.com/UTg9bz5v

- Disk --> imgbox - fast, simple image host

- Storagepath --> imgbox - fast, simple image host

- All Microsoft Windows updates are installed on the W2K12 VM's.

Hopefully you can take a look at my printscreens, I'm stuck and don't see where it's going wrong.

Reply
0 Kudos
Srinivasu13
Enthusiast
Enthusiast

Sorry I can't access your screen shots.

Can you test one VM with paravirtualized SCSI adapter & VMXNET3 NIC.

------------------------------------------------------------------------------- If you found this or any other answer helpful, please consider to award points. (use Correct or Helpful buttons) Regards, Srini
Srinivasu13
Enthusiast
Enthusiast

Also verify ESXi hosts are running with latest patches.

------------------------------------------------------------------------------- If you found this or any other answer helpful, please consider to award points. (use Correct or Helpful buttons) Regards, Srini
Reply
0 Kudos
williambishop
Expert
Expert

Is it ALL 2012 boxes or just the ones running citrix?

--"Non Temetis Messor."
Reply
0 Kudos
Wh33ly
Hot Shot
Hot Shot

Not sure if your problem is VM performance related or maybe guest related, here is something I encountered:

We have a lot of Win2012 machines which run great, one of the first things I noticed when we started deploying  was the slow and sluggy mous which was caused by the mouse pointer shadow.

Go to the “control panel” and open “System”

Open “Advanced system settings and then click the “Settings” button at the performance field. In the tab “Visual effects” you see an option “Show shadows under mouse pointer”. After disabling this, the performance of the mouse was back to normal again.And the server looked much more responding

I also decided to set the settings to “Adjust for best performance” because this will decrease any other almost not noticable lag :)

Could be as simple as this if applicable.

Reply
0 Kudos
MCOV
Contributor
Contributor

Please check this:

- change network adapter to vmxnet3 (instead of E1000E)

- disable tcp chimney offload (checking status with cmd: netsh int tcp show global)

- in adapter properties in adapter configuration on Advanced tab disable: Large Send Offload V2, Recv Segment Coalescing and TCP Checksum Offload

Reply
0 Kudos
skelpie
Contributor
Contributor

Hi guys !

I think i've solved the problem, but i don' know how can this happened.

Here what i've done.

I have ESXI 6.5 Dell version on Dell R900 Server. There is a Windows 2012 R2 on it. After installation the performance was very poor. When i started to upgrade the Hypervisor (because of hardware incompatibility of R900 'vve installed first ESXI 6.0 then updated to 6.5)

After upgrading i've reregister th VM W2012R2 and the performance was brilliant. Then i upgrade VM's version from 9 to 13 - (esxi 6.5 and later) - performance was sh..t.

My conclusion

When use for ESXI 6.5. VM W2012R2 with hw. compatibility level 9 - everything is superb

My best config on 4 Xeon 7350 is 2 VCPU's 4 Cores each

Now is blasting!!!!

Maddas68
Contributor
Contributor

We are experiencing the same problem with some Windows 2012 R2 servers, especially RDP/Citrix servers. Downgrading from Virtual Machine Version 13 to version 9 solves the problem.

The question is, why is this happening? Why do Windows Server 2012 R2 have problems with VMWare Version 13?

Reply
0 Kudos
peterriederer
Enthusiast
Enthusiast

Hi Folks,

you also have performance issues with Windows 2016 Servers? Our 2012R2 Servers are running quite well on ESXi 6.5 latest patch and VMware Version 13.

But the 2016 Servers are really poor, even if they only have 1 vCPU and only 4GB RAM. With more vCPU they are a bit faster.

Regards

Peter

Reply
0 Kudos
peterriederer
Enthusiast
Enthusiast

Since a few days we have the same Problem here.

We are using vSphere 6.5 and HW-Level 13 several weeks now, and it worked quite well, with our RDP and Fileservers.

But since a few days It all started with Veeam Backup stopped / wasnt able using VSS anymore while backing up our main fileserver.

From this time our W2K12R2 File-Server went slower and slower. 100% Disk-Activity, Disk-Queues running high and Disk-Latency in Windows showed more then 700 - 1000 ms. We tried a lot... uninstalled Windows Updates, uninstalled Anti Virus, rebooted the SAN, rebooted ESX-Hosts, nothing helped.  First we used LSI SAS Controller (same Problem) and now we are using Paravirtuell controller and VMNEXT NIC. If there is just a very little load on the machine, it is going to get unusable, and just small word and excel files take forever to open and to save.

Now we tried to go back to HW-Level 10 for both Fileservers and it is working again, without any Problems. For testing purpose we tried to upgrade to HW-Level 11. One Fileserver is ok with that the other one we got the same problems again, hence we went back again to HW-Level 10.

But it seems more and more Machines are getting sluggish now, Medical-Application with an oracle Database (W2K8R2 Server) now also gets slower and slower, and the application takes a very long time to load.

Does anybody have any news or and helpful Tipps and Tricks?

Regards

Peter

Reply
0 Kudos
Moonduza
Contributor
Contributor

I Would do the following:

- Add a VMXNet3 NIC
- Use Paravirtual Storage Controller
- Update VMware tools
- Check the host power management configuration and, if possible use high performance.

Reply
0 Kudos
peterriederer
Enthusiast
Enthusiast

thanks but this is always our standard setup/config, and was already mentioned above.

- we only use VMXnet3 NIC,

- we only use Paravirtual Storage Controller (except these 2 filservers havnt been migrated so far, due to missing maintenance windows) but now they are and didnt change so much

- Tools are always up2date

- Host Power is high performance

we had no issues in the past, and didnt change anything in our infrastructure.

the only thing we changed was the disk size of the file server -> 1.8 TB to 2.5 TB but that could not be an issue in my opinion.

Reply
0 Kudos
controlsforall
Contributor
Contributor

We are having the exactly same issue.

I had extended the driver from 4TB -> 6TB.

Windows 2012 memory utilization keeps going up. That VM locks and also it locks the whole vsphere server. I end up having to do hard reset on the whole ESXI.

Within the vspehere memory usage show 6GB and keeps increasing but on the Windows server it is stable around 2GB.

MemoryUsage.png

We have upgraded to the latest build. I was on tech support with dell. We upgraded perc and bios. I think we are still having the same issue.

Reply
0 Kudos
peterriederer
Enthusiast
Enthusiast

you tried to go back to hardware compatiblity 10?

with more and more testing it seems other VMs and OSes are also affected, like WSUS Servers etc.

e.g. just start installing windows updates on Machines with HW13, and watch disk queue and disk latency in performance monitor (task-manager)

Reply
0 Kudos
adamsavagex
Contributor
Contributor

I've witnessed the same thing and going back to hardware version 9 or 10 seems to fix the problem..

I was seeing high disk queue length on VMs  that are Windows Server 2012 R2 x64.

I'm going to bring up 2 VMs off the same template and downgrade one of them to hardware version 9/10 and do a comparison of the hardware loaded in the system and see if we can find the culprit.

Reply
0 Kudos
mvduijn
Contributor
Contributor

We are experiencing the same issues with our file servers after upgrading to hardware level 13.

We have high disk queue length on VMs  that are Windows Server 2012 R2

We created a ticket with VMware but are still investigating the issue because support told me there are no known issues about hardware level 13.

The most problematic servers we already downgrade to hardware level 9 and i can't show VMware support the issue because other servers don't have the issue all the time.

Currently i am cloning a downgraded fileserver with hardware level 9 and upgrade to 13 hoping to reproduce the issue.

Does anyone have a vmware support call about this problem I can refer to?

Reply
0 Kudos
peterriederer
Enthusiast
Enthusiast

We opened a ticket too, but after talking to them they dont know anything about problems with HW Level 13. So it just was an informational SR.

we both agreed to close the ticket, because i would like / should to do some more testing in our lab with a cloned fileserver for example etc. opening a new one then.

But at the moment i have not much time to investigate further in this problem 😞

But since i am back at HW10 with our 2012R2 Fileservers, everything is working fine. Other machines with HW13 are still lagging after there is some small load, but i dont want to downgrade them all at the moment.

Reply
0 Kudos