VMware Communities
0WayneH0
Hot Shot
Hot Shot

Workstation 11.0 Performance not as good as Workstation 9.0?

I have been using Workstation 9.0.3 happily for some time now and have recently been considering upgrading to Workstation 11.0, so I did a test today as follows.

Using my work laptop (HP Elitebook Workstation 8560w, Core i7-2820QM CPU, 8GB RAM, nVidia Quadro 2000M video), I ran the Windows Experience Index on a 64-bit Windows 7 Ultimate Guest OS, after freshly installing that guest OS using the easy install feature of Workstation 9.0.3 and 11.0 respectively.

The host OS for both tests (same laptop) is Windows 7 Ultimate x64.

Host WEI:

Processor: 7.5

RAM: 7.6

Graphics: 6.9

Gaming Graphics: 6.9

Primary hard disk: 5.9 (mechanical drive)

First, the Workstation 9.0.3 guest WEI result:

Processor: 6.9

RAM: 5.5

Graphics: 6.0

Gaming Graphics: 6.0

Primary hard disk: 6.6

WS903.jpg

Now the Workstation 11.0 guest WEI result:

Processor: 6.9

RAM: 5.5

Graphics: 4.1

Gaming Graphics: 4.7

Primary hard disk: 6.4

WS11.jpg

Notably, the graphics performance is way down compared to Workstation 9.0.3 and the disk metric is also a little down.

I used the recommended 1GB of video RAM for my WS11 VM.

Is there any reason I would see such a performance degradation? If anything I was expecting to see better performance from WS11, not worse performance.

All of a sudden I am not as in a big of a rush to upgrade.

Regards.

23 Replies
0WayneH0
Hot Shot
Hot Shot

I did some more tests last night, this time just running Workstation 11.0 and the Windows 7 guest that I newly created in Workstation 9.0.3.

If all I do is power it on, and not upgrade the virtual hardware (and VMware tools) then the performance seems to be the same as in Workstation 9.0. If, however, I upgrade the virtual hardware (and VMware tools) the performance degradation kicks in.

Running in WS11

WS9 Hardware:

Processor: 6.9

RAM: 5.5

Graphics: 6.0

Gaming Graphics: 6.0

Primary hard disk: 6.6*

WS10 Hardware:

Processor: 6.8 (-0.1)

RAM: 5.5

Graphics: 4.3 (-1.7)

Gaming Graphics: 4.7 (-1.3)

Primary hard disk: 5.9 (-0.7)

WS11 Hardware (256Mb video RAM)

Processor: 6.8

RAM: 5.5

Graphics: 4.1 (-0.2)

Gaming Graphics: 4.7

Primary hard disk: 5.9

* I often wondered how WS9 would report better primary disk access than the host; I assumed some kind of internal I/O caching or similar. Whatever the case, that benefit seemed to be lost in WS10 and stayed that way in WS11.

Most of the reported performance hit is when changing to WS10 hardware (notably the primary hard disk too), but the graphics component of the WEI takes a further hit on "upgrading" to WS11 hardware.

My excitement and anticipation of WS11 has been somewhat dulled by this result since I mainly run Windows 7 guests, although eventually I'll want to run Windows 10 (and couple Linux) guests.

I'll do some testing with Win8/10 and add my findings later.

Reply
0 Kudos
0WayneH0
Hot Shot
Hot Shot

Additional testing:

Windows 7 x64 Guest

WS9 Hardware (installed using WS11):

Processor: 6.8 (-0.1)

RAM: 5.5

Graphics: 4.2 (-1.8)

Gaming Graphics: 4.7 (-1.3)

Primary hard disk: 6.4 (-0.2)

Windows 8 x64 Guest

WS9 Hardware (installed using WS9):

Processor: 6.9

RAM: 5.5

Graphics: 6.3 (+0.3 vs Win7)

Gaming Graphics: 6.3 (+0.3 vs Win7)

Primary hard disk: 6.4 (-0.2 vs Win7)

WS10 Hardware (upgraded in WS11):

Processor: 6.9

RAM: 5.5

Graphics: 4.1 (-2.2)

Gaming Graphics: 5.5 (-0.8)

Primary hard disk: 6.4

WS11 Hardware (upgraded in WS11):

Processor: 6.9

RAM: 5.5

Graphics: 4.2  (+0.1)

Gaming Graphics: 5.5

Primary hard disk: 6.6 (+0.2)

I think what I am seeing is that the most likely the key is the version of VMware tools that is the difference and primarily with respect to the graphics performance.

So for both Windows 7 x64 guests and Windows 8 x64 guests, the Workstation 9.0.3 tools/drivers are looking far better than Workstation 11.0. I thus cannot consider Workstation 11.0 (in it's current form) an upgrade.

I'll wonder, most likely, if things would be different with a Windows 8 host, but I am unlikely to want to go near that abomination for my host OS, without any guarantees that it would help. If anyone has done any such tests please let me know.

Cheers.

Reply
0 Kudos
newbie93
Hot Shot
Hot Shot

I share your pain, but as an FYI, there were/are substantial problems with vmtools from Workstation 9.x WRT file corruption on Shared Folders if they are being used with Linux guests. Just copying files or opening files things are seriously broken. These problems weren't fixed in VMTools until the ones included in the last release of Workstation 10 (10.0.4 IIRC). If you aren't using Linux guests or Shared Folders in general, you might okay with VMTools from the earlier Workstation versions. Just do a seach of this forum for details, if you're interested.

VMWare seems to have taken the "Add more features" product path and doesn't seem to be concerned with performance at all. I wouldn't complain as much if they had added, ohhhhh, something like DX10 or DX11 support (or the upcoming DX12 in Win 10), but they haven't, we're stuck on the now very, very old DX9 for graphics - soon to be 3 major generations old. - The Great GrandPa of DX versions.

Unfortunate for those of us that use Workstation on a daily basis that we have to suffer with slower USB, slower ethernet and lower WEI numbers with each new release of Workstation and even buying a faster host doesn't seem to help too much either. If it weren't for Windows 8.1/10, I would be on Workstation 9 as well, but even then as new host hardware drops USB2 sockets in favor of USB3, USB3 support will be needed which means a newer version of Workstation. But even Virtual Box doesn't have USB3 support yet..

Thanks for providing the reality of "hard" numbers. Perhaps if people stop buying/upgrading, maybe peformance will take a front seat for a release, but I am not holding my breath for that.

Reply
0 Kudos
0WayneH0
Hot Shot
Hot Shot

Thanks for the info on the Linux/Shared Folders issues in 9.x. You are right though that I don't use Shared Folders; in fact I have not in a while since I used to run into lots of issues with them. I can't recall which version of Workstation it was, but now I'm simply in the habit of running all VMs with bridged network connections and use network file sharing. I do run Linux guests, but usually only to check out different Linux distros as a matter of curiosity and keeping abreast of what is going on in that arena. Yep, so I've been blissfully unaware of that problem. Smiley Happy

Re" and doesn't seem to be concerned with performance at all ", I find that to be a bit of a concern because I really do use workstation all the time. Everything I do is on workstation; if it works slower I am slower. Only when I am forced to use my host OS will I leave the virtualized world and with WS 9.x that is very rarely (basically anything where I really do need direct access to my host's hardware, which at this point just Lync (still cannot get a headset to work with Lync in a VM properly) and if I had the time, which I don't, game playing!)

For S&G I also took a look at the latest Virtual Box to see how it compares performance wise and whilst the WEI values were better than WS 11.0 (not better than 9.x), the fact the 3D support is still experimental, and the whole Virtual Box environment in general, just reminds me why I haven't been using Virtual Box. Smiley Happy

Of course now M$ have removed the WEI from Windows (since 8.1 if I recall correctly) so I cannot compare Windows 10 in WS 9.x/11.0, but I have tried running it in both and it seems pretty sluggish under WS11. Under WS9.x it seems pretty good to be honest, but if I am being honest I cannot make any big claims either way as yet, because I have not compared these on the same Host OS as yet.

I haven't encountered any issues running Windows 10 in WS9.x, but then I don't really care about USB3/touch etc, so it seems like it will work for my current purposes.

>> Thanks for providing the reality of "hard" numbers. <<

You are welcome. I thought someone should. And I personally cannot overlook them. Furthermore, given that I appear to be able to run Windows 10 in 9.x I've just convinced myself not to upgrade to WS11 (just like I did before that with WS10).

Reply
0 Kudos
0WayneH0
Hot Shot
Hot Shot

I did a test in WS10 using a Win7x64 guest.

WS10 Hardware (created in WS9, upgraded in WS10):

Processor: 6.9

RAM: 5.5

Graphics: 5.9 (-0.1)

Gaming Graphics: 5.9 (-0.1)

Primary hard disk: 6.2 (-0.4)

So most of the performance degradation comes with WS11/VMware tools 11.

I have no compelling reason to go to WS10 and definitely not WS11 at this point. Here's hoping the next dot release or two can bring things back to where they were. :smileymischief:

Reply
0 Kudos
Foxxen2
Contributor
Contributor

"Of course now M$ have removed the WEI from Windows (since 8.1 if I recall correctly) so I cannot compare Windows 10 in WS 9.x/11.0".

On a technical point, a WEI stand-alone executable is still accessible from numerous sites (I think the front end was removed but not the engine). Since testing would be in a VM only, it's almost irrelevant if it's not part of "usual" current Windows, for your limited purpose.  But be aware they might rescale it depending on detected OS and platform, I would run 3 tests not 2:  WS9 using WS9 vm model, WS11 using WS9 vm model (not upgraqded) or checking same drivers, and then WS11 after upgrade to latest hw model. Just in case there are other factors at play.

Find your Windows Experience Index scores in Windows 8.1 - CNET

WEI Tool Brings the Windows Experience Index Back to WIndows 8.1

Unearth the Windows Experience Index in Windows 8.1 with PowerShell - TechRepublic

Having said that ..... the better answer is probably, don't use WEI at all. I would find an assessment based on WEI very dubious - I'm not even sure its comparable by much, there's no clarity what went into each number exactly, and no clarity how much real-world difference or scaling difference is signified by any changed results.  WEI just gives you arbitrary numbers and little clarity how much any differences mean or even what they signify.  It's also very hard to know what to target for improvement or what functionality to criticise.  Use a proper benchmarking program (SiSoft Sandra, Passmark, Furmark for 3D, etc) - there are several reputable free ones around.  Programs like these will actually test repeatably on different hardware, and give you hard "before/after" data component by component, so you can see the exact components of your system and its performance that changed, and by how much.

Hope this helps.

0WayneH0
Hot Shot
Hot Shot

OK I took your benchmarking suggestion on board and used two different, albeit old, benchmarks since that's all I have, it was late, and I have not done proper benchmarking in some time. These results are quite interesting though.

My notation: WSX/HX/DX is Workstation version (WSX), Hardware version (HX), Driver/Tools version (DX).

All tests are Windows 7 x64 guests on a Windows 7 x64 host (same test rig as with the WEI tests).

Cinebench 11.5

CPU

WS9/H9/D9

2.52

WS11/H11/D11

2.43

Graphics

WS9/H9/D9

Could not get a test score; OpenGL reference match too low (92.5%)

WS11/H11/D11

Could not get a test score; OpenGL reference match too low (92.2%)

PCMark05

CPU

WS9/H9/D9

9447

WS11/H9/D9

8967

WS11/H11/D11

8314

Graphics

WS9/H9/D9

7916

WS11/H9/D9

5484

WS11/H11/D11

7130

So both additional benchmarks would suggest a slight drop in CPU performance (and WEI also indicated this possibility). It's interesting that some drop in CPU performance is observed merely by running under WS11.

In terms of graphics, PCMark05 does 8 tests. 4x 2D tests and 4x 3D tests. It indicates that graphics performance has also dropped in WS11, but interestingly a much worse value is obtained when running WS9 hardware and drivers under WS11. I can't see my WEI numbers when writing this reply, but I recall that the graphics WEI numbers were basically the same when running WS9 hardware/drivers under WS11 so I assume that PCMark05 does additional tests that expose some issue there. The good thing, I suppose, is that PCMark05 indicates that most of the performance is restored under WS11 when using WS11 hardware and drivers. By the raw numbers alone, about a 10% performance drop overall, although I am not sure the net result can be interpreted that way as the overall score is some combination of those 8 tests.

What I will say about the graphics tests though, is this. When running two of the 3D tests under WS9/H9/D9, despite the high frame-rates reported, both were visually poor. I expected very low frame-rate results for the tests, but somehow high values were reported. Then, when running those same two tests under WS11/H9/D9 (so no hardware or driver changes, just using WS11) the visuals of those tests improved markedly (they now appeared to run smoothly), yet the score under that combination was the lowest (and reported frame rates for those tests were lower). Finally, when running the tests under WS11/H11/D11 the reported frame rates were higher again (in proportion with the overall score), and the visuals stayed clean and smooth (versus the obvious issues under WS9, with contradictory results).

So after all this I am not sure what to believe.

Perhaps there's a slight degradation in CPU performance, but it's slight, and this is just for one host / guest OS combination on one test rig, so not much to be gleaned I would suggest.

As for graphics, the test numbers still indicate some drop in performance, but it seems to me that it's not nearly as bad as the WEI tests would suggest (and as you say the WEI tests are not transparent about what is going on under the hood). Also the fact that despite the lower graphics test result under PCMark05 in WS11, two of the 8 tests (both 3D tests) are much better in terms of visual appearance than when run under WS9 so for those two tests I would tend to ignore the scores and go with what I see on screen.

So I just put in a request with IT to get me Workstation 11.0 after all.

Thanks for pushing me on WEI even though I didn't really have the time to do any of this. Smiley Happy

~~~~~~~~~~~~~~~~~~~~~~~~~

EDIT:

I just noticed also that one of the other 3D tests doesn't even render under WS9 (shows a black screen), but under WS11 it renders properly.

So for completeness I am putting in the test breakdowns for WS9/H9/D9 vs WS11/H11/D11 using a run under the latter configuration that scored 7415 for graphics (higher than previous tests).

WS9/H9/D9 - 7759 (One of the lower results observed)

WS11/H11/D11 - 7415 (The highest and most recent result observed)

Test
WS9/H9/D9
WS11/H11/D11
2D Transparent Windows5372 windows/sec5042 windows/sec
2D Graphics Memory - 64 lines193 fps218 fps
2D Graphics Memory -128 lines186 fps207 fps
2D Video Playback (WMV)57 fps58 fps
3D Fill Rate Multitexturing14705 MTexels/sec *13568 MTexels/sec
3D Polygon Throughput169 MTriangles/sec **
130 MTriangles/sec
3D Pixel Shader545 fps ***463 fps
3D Vertex Shader127 MVertices/sec121 MVertices/sec

* Test appears to render a lot slower (stutters) under WS9/H9/D9.

** Test does not render (blank screen) under WS9/H9/D9.

*** Test appears to render a lot slower (stutters) under WS9/H9/D9.

These numbers are biased towards WS11 since I took the best (and latest) test result from that configuration and compared it to one of the lesser results from the WS9 configuration, however, the fact that 3 of the 3D tests appear not to render correctly under WS9 has me leaning towards WS11 despite the lower figures.

Time for me to go get a life again I think.

Cheers.

Reply
0 Kudos
Neil1T
Enthusiast
Enthusiast

I went looking for articles of this type because I'm testing with Workstation 11 and trying to work out whether or not to upgrade.

I have an Alienware M18x R1 with dual AMD 6790M graphics cards in crossfire, Core-7 2820qm and 32 gig ddr3 1600 RAM.  I boot from SSD but my VM drive is a hybrid drive.  The base OS is W8.1 enterprise and my graphics in the database on the core OS are over 8.

I noticed that after upgrade and hardware upgrade my graphics WEI on my VM had dropped to 2.0.  I found the VMWare tools had not correctly installed and I managed to get it back up to 3.4.  Before the upgrade it was over 6 as my score was 5.9 which was the HDD.  My CPU was 7.5 and my RAM 7.9. The only thing which has changed is the graphics at 3.4 business and 4.9 gaming.

However after reading this whole threat I decided to do some other real life testing.  I had already noticed that the screen display is crisper and better than the previous versions so I had a look at playing a Full screen full 1080p movie in the VM.  It works fine without a stutter in all modes, windowed, full screen and exclusive mode.  I can only assume that the previous versions of the driver sacrificed quality for speed and that the W11 one does not.  Which then leads to better quality overall but lower WEI scores.

With a current score of 3.4 I can live with that until I renew my Laptop and get the latest R290 graphics which are more than twice as fast as the 6970M.  In general the VM performs fine although I currently don't have another monitor to play with.  My goal for the last 3 years has been to work in VM's full screen, multi monitor with max resolution and high performance.  VMWare workstation has been the only solution which has been capable of giving me that.

More importantly for me I'm not seeing the multi cpu blue screen on startup for 64 bit VM's.  That, for me, is a huge win.

Neil1T
Enthusiast
Enthusiast

Just to add to this.  I connected one of my external USB drives on sharing then ran up UT2004.  I maxed all the settings and set the resolution to0 1920x1200.

Apart from a little audio crackling, it played perfectly with no video stutter even in the middle of a 4 way fragfest.  I wasn't on exclusive either as I was doing some other work in the background on the OS coping files.

Seems to be acceptable to me.  My physical laptop with a WEI of 3 would not have even touched that game at those resolutions or settings.  I'm guessing the low WEI numbers are due to something in the response of the driver to Microsoft tests because the VM is certainly not responding according to the results.

0WayneH0
Hot Shot
Hot Shot

I just came back to this thread to say that I installed WS11 on the machine that I built almost 3 years ago specifically to run VMs and the guest WEI values, after upgrading to WS11 hardware and tools are essentially the same as they were on WS9.

The host (Win7 x64) WEI is:

Processor: 7.8

Memory: 7.9

Graphics: 7.9

Gaming: 7.9

Disk: 7.9

*7.9 is the max score under Win7 WEI.

The host specs are:

Core i7-3930K

32Gb 1600 DDR3

Samsung EVO 840 SSD (primary) + 5x various enterprise HDD for VMs.

ASUS HD7950

For the two VMs I have upgraded so far:

Win7 x86 Guest

Processor: 7.1

Memory: 7.9

Graphics: 6.0

Gaming: 6.0

Disk: 6.5

Win 7 x64 Guest

Processor: 7.0

Memory: 5.9   (not sure why this is so much lower than the x86 guest, but it was the same under WS9)

Graphics: 6.0

Gaming: 6.0

Disk: 6.1

With the exception of one value (Disk, for Win 7 x64) dropping by 0.1, all the values are the same as with WS9, which is in direct contrast to the results I obtained with my HP Elitebook Workstation. The video card here is a nVidia Quadro 2000M.

Had I tested on my own host the first time, instead of my company laptop, I would never have made this post. Smiley Happy

Reply
0 Kudos
BillE
Contributor
Contributor

I downgraded the hardware to version 9 and that made it better but not as good as it was.  At least now it does not feel like I am dragging around windows with a rubber band.

Reply
0 Kudos
Neil1T
Enthusiast
Enthusiast

For anyone who is interested, I have two stats using the wei tool.

In the process of my moving to W10, I took the same VM I was working with, which had dropped from ~6 mainly on video graphics performance and I re-ran the test on Workstation 12 with the following.

VM upgraded from W7 to W10

VMware workstation upgraded to 12

VM hardware upgraded to version 12

VM moved from an internal Seagate 1tb 2.5 hybrid drive to a USB3.0 connected identical drive.

So my WEI at the start point was as it was from the tests I did with W7 and Workstation 11. Which gave me 3.8 (I had tweaked it up from 3.4).

My WEI after Workstation 12 and W10?

Base Score 5.9

Processor Calculations per second 7.7

Memory (RAM) Memory operations per second 8.1

Graphics Desktop graphics performance 8.1

Desktop graphics 3D business and gaming graphics performance 9.9

Primary hard drive Disk data transfer rate 5.9

I'd give that a +++++

Note what VMWare and W10 did with the gaming performance...

Although, as I was writing this, I re-tested the base OS as I have downgraded from Enterprise to Pro and had to clean just about all drivers etc and put in the latest AMD graphics drives.  My base WEI climbed from 7.3 to 7.9 and the graphics climbed from 7.3 (both), to 8.1 (both)....

However, in reality, the change in workstation between 11 and 12 is nothing short of dramatic. Sadly the original W7 VM is gone so I can't quite compare apples with apples.

Reply
0 Kudos
Neil1T
Enthusiast
Enthusiast

Whoops, forgot to put in the starting wei before the Workstation 12 upgrade and W10 upgrade.

Base Score 3.8

Processor Calculations per second 7.5

Memory (RAM) Memory operations per second 7.9

Graphics Desktop graphics performance 3.8

Desktop graphics 3D business and gaming graphics performance 6.0

Primary hard drive Disk data transfer rate 5.9

Reply
0 Kudos
henkus123
Contributor
Contributor

Same problem here. I dod not check the scores from Winows, but video perfomance is sluggish. I first had workstation 11 and I had to put my vm's on hardware version 10. NOw I have workstation 12 and though this issue was fixed (even with 4k monitor support) and it is still there. I can also see that vmware can allocate the set amount of video ram dedicated in the vm. When on a different hardware version then 10 the video memory becomes shared video memory. Why is that? What I do know is that no matter how high the shared video memory is it is still sluggish. Set it to hardware version 10 and it's kind of fixed.

Reply
0 Kudos
Neil1T
Enthusiast
Enthusiast

For my latest tests I used HW version 12.

Reply
0 Kudos
henkus123
Contributor
Contributor

Ok, I still have very sluggish perfomance if I'm not using HW vesion 10

Reply
0 Kudos
henkus123
Contributor
Contributor

Capture.PNG

This is on HW version 10. If I'm on any version above that the dedicated video memory drops to 4mb and you will only see change in the shared memory..

Reply
0 Kudos
Neil1T
Enthusiast
Enthusiast

Somehow I don't think the video graphics memory is the biggest issue.

As you can see I have 4mb dedicated, 536mb total and 532mb shared.  I know my graphics are pretty quick, but you don't get 8.1 and 9.9 (windows10, it's less on 8), if the system is not performing well.

Have you completely removed and re-installed the VMWare Tools with each hardware change???

worstation12.png

I've attached the VMX that goes with it.

just to add.  The WEI was done in full screen 1920x1080 exclusive mode...

Reply
0 Kudos
henkus123
Contributor
Contributor

I'm going to try that and remove/reinstall vmware tools

Reply
0 Kudos