"Failed to detect host display topology, using one display as default" error message?

[rileyb76]

Guys,

We have a RHEL 7 machine (3.10.0-514.10.2.el7.x86_64) running Workstation 12.5.5 that is getting this message when powering on any VM.

We have reinstalled Workstation but still get the same error.  We thought it was a VMWare Tools issue (out of date), but started some VM's that didn't have the Tools installed and they did the same thing.

When you power on any VM, you can't view anything inside the VM - its just black and won't display correctly.

Can you all please advise?

[rileyb76]

We determined that this ONLY happens when we use Windows Remote Desktop, to RDP into the RHEL 7 host machine to launch any VM.  The RHEL host machines have XRDP installed, and that is how we connect from Windows. 

This behavior does not happen with RHEL 6, only RHEL 7.  Has anyone else seen this or had an issue?

Heres another screen shot when launching a VM on a RHEL 7 host, while connect thru RDP (Windows 7):

[rileyb76]

Here is a chart that might help explain whats happening a little better:

[gimmely]

I'm not an expert in this, so please take my comments with suspicion.

I think what you're experiencing is hardly an issue with VMware, because, as you said, it has to involve RDP/XRDP.  So, I guess "mapping(s)" in display drivers, from VMware to RDP/XRDP and eventually your Linux is(are) not in sync.  Then, you can try to adjust display settings in RDP and/or XRDP.

Another thing to consider is if this issue just started recently, e.g. after some updates on Linux or XRDP or RDP.  Or, one of these needs an update.

[wila]

Hi,

You don't have direct access to the host with the problem?

Eg. you cannot see if the problem still occurs at the host without first going through remote desktop?

What I am thinking that is happening is an incompatibility between what XRDP sees that is changing at the display and what the console is actually drawing.

In other words, the console screen might not invalidate all the screen regions needed for xrdp to detect and send over.

With multiple levels of indirection, there's always risk you miss out on some details.

You might be able to tweak that with xrdp settings (I don't use it so cannot suggest anything to change) or alternatively you could try to use the built-in VNC sharing.

Under settings of your VM you can setup VNC sharing and instead of looking at the guest via RDP, connect directly via VNC.

Yes, not ideal, but a possible workaround.

--

Wil

[rileyb76]

THanks for the input. 

My issue is these RHEL 7 hosts are in different locations but on the same LAN.  When I login locally to them, VMWare displays everything perfectly.

Again, this worked for me when the host was RHEL 6, so I'm not sure what exactly changed.... if its vmware, rhel, or xrdp.

Installing vmware tools on the guest produced the same effect.

[wila]

Hi,

When I login locally to them, VMWare displays everything perfectly

That statement suggests it is not VMware itself causing the issue otherwise it would have the same issues locally.

Of course there is always the chance of something in VMware that just works different under RHEL7 in comparison to RHEL6, but so far from what you mentioned, the most likely issue is with xrdp.

--

Wil

[rileyb76 · Accepted solution]

This ended up being a Windows issue.  Under remote desktop settings, if we set the Colors to "True Color (24bit)" we could view the VM's being powered on.

[wila]

Hi,

Apparently you can configure xrdp to support 32 bits.

Eg. see this quote:

The following line (in xrdp.ini) is responsible for the color depth of the session:

xserverbpp=?

where ? is 15, 16, 24 or 32.

from page: XRDP Devel - [Xrdp-devel] Does not Xrdp support 32bit color depth?

--

Wil