Re: Problem with vRealize Suite Lifecycle Manager ...

Joffer · ‎02-26-2018

First of all, I did not find a separate forum for vRealize Suite Lifecycle Manager, so I opted to post in what I thought was the best alternative; vRSLCM is some kind of automation tool.

I'm having problem getting my (newly installed) vRSLCM 1.1.0.7-7359844 to register with MyVMware. The server needs to use a proxy to get online and it has of course been configured ("proxy.mydomain.local:80") but it won't register (I've obfuscated the real proxy address/IP). Anyone experienced this, and fixed it? It's a bit of a long shot, since it's a relatively new product...

The MyVMware token fetch seems to throw an error with username problems. I see this in /var/log/vlcm/vrlcm-xserver.log:

2018-02-26 15:04:06.568 INFO [http://10.10.10.95:35995/core/document-index/queries/1519657254959000] c.v.v.l.d.m.h.MyVmwareDownloadRestClient - -- Get myvmware access token - Started

2018-02-26 15:04:06.568 INFO [http://10.10.10.95:35995/core/document-index/queries/1519657254959000] c.v.v.l.d.m.h.MyVmwareDownloadRestClient - -- Get myvmware access token. Current try number : '1'

2018-02-26 15:04:06.570 INFO [http://10.10.10.95:35995/core/document-index/queries/1519657254959000] c.v.v.l.u.p.ProxySetupHelper - -- Loading Default Proxy File : /etc/environment

2018-02-26 15:04:06.570 INFO [http://10.10.10.95:35995/core/document-index/queries/1519657254959000] c.v.v.l.u.p.ProxySetupHelper - -- Setting system proxy: host: proxy.mydomain.local, port: 80

2018-02-26 15:04:06.577 INFO [http://10.10.10.95:35995/core/document-index/queries/1519657254959000] c.v.v.l.u.p.ProxySetupHelper - -- Loading Default Proxy File : /etc/environment

2018-02-26 15:04:06.577 INFO [http://10.10.10.95:35995/core/document-index/queries/1519657254959000] c.v.v.l.u.p.ProxySetupHelper - -- Setting system proxy: host: proxy.mydomain.local, port: 80

2018-02-26 15:04:06.581 INFO [http://10.10.10.95:35995/core/document-index/queries/1519657254959000] c.v.v.l.u.RestHelperWithProxy - -- Connecting with Proxy host : proxy.mydomain.local and port 80

2018-02-26 15:04:06.583 ERROR [http://10.10.10.95:35995/core/document-index/queries/1519657254959000] c.v.v.l.d.m.h.MyVmwareDownloadRestClient - -- Error occurred while fetching access token from https://my.vmware.com.

java.lang.IllegalArgumentException: Username may not be null

at org.apache.http.util.Args.notNull(Args.java:54) ~[httpcore-4.4.4.jar!/:4.4.4]

at org.apache.http.auth.UsernamePasswordCredentials.<init>(UsernamePasswordCredentials.java:80) ~[httpclient-4.5.2.jar!/:4.5.2]

at com.vmware.vrealize.lcm.util.RestHelperWithProxy.readRemoteDataWithProxy(RestHelperWithProxy.java:128) ~[lcm-util-1.1.0.jar!/:?]

at com.vmware.vrealize.lcm.util.RestHelperWithProxy.readRemoteDataWithProxy(RestHelperWithProxy.java:65) ~[lcm-util-1.1.0.jar!/:?]

at com.vmware.vrealize.lcm.util.RestHelper.readRemoteData(RestHelper.java:95) ~[lcm-util-1.1.0.jar!/:?]

at com.vmware.vrealize.lcm.util.RestHelper.readRemoteData(RestHelper.java:87) ~[lcm-util-1.1.0.jar!/:?]

at com.vmware.vrealize.lcm.drivers.myvmware.helper.MyVmwareDownloadRestClient.getAccessToken(MyVmwareDownloadRestClient.java:96) ~[lcm-myvmwareplugin-driver-1.1.0.jar!/:?]

...

After 4 tries, it gives up with:

2018-02-26 15:04:36.607 ERROR [http://10.10.10.95:35995/core/document-index/queries/1519657254959000] c.v.v.l.d.m.h.MyVmwareDownloadRestClient - -- Error occurred while fetching access token from https://my.vmware.com. Status code : ''. Status message :

The proxy doesn't need a username, and of course I have added my username and password for MyVMware.

Both /etc/environment and /etc/sysconfig/proxy contains the proxy address and port. Testing manually, connecting to https://my.vmware.com with curl, works fine (giving a redirect address):

# curl -v -x proxy.mydomain.local:80 https://my.vmware.com

* Rebuilt URL to: https://my.vmware.com/

* Trying PR.OX.Y.IP...

* TCP_NODELAY set

* Connected to proxy.mydomain.local (PR.OX.Y.IP) port 80 (#0)

* Establish HTTP proxy tunnel to my.vmware.com:443

> CONNECT my.vmware.com:443 HTTP/1.1

> Host: my.vmware.com:443

> User-Agent: curl/7.54.0

> Proxy-Connection: Keep-Alive

>

< HTTP/1.1 200 Connection established

< Date: Mon, 26 Feb 2018 15:21:13 GMT

< Proxy-Connection: Keep-Alive

< Via: 1.1 wsproxy.mydomain.local

<

* Proxy replied OK to CONNECT request

* ALPN, offering http/1.1

* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH

* successfully set certificate verify locations:

* CAfile: /etc/pki/tls/certs/ca-bundle.crt

CApath: none

* TLSv1.2 (OUT), TLS header, Certificate Status (22):

* TLSv1.2 (OUT), TLS handshake, Client hello (1):

* TLSv1.2 (IN), TLS handshake, Server hello (2):

* NPN, negotiated HTTP1.1

* TLSv1.2 (IN), TLS handshake, Certificate (11):

* TLSv1.2 (IN), TLS handshake, Server key exchange (12):

* TLSv1.2 (IN), TLS handshake, Server finished (14):

* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):

* TLSv1.2 (OUT), TLS change cipher, Client hello (1):

* TLSv1.2 (OUT), TLS handshake, Unknown (67):

* TLSv1.2 (OUT), TLS handshake, Finished (20):

* TLSv1.2 (IN), TLS change cipher, Client hello (1):

* TLSv1.2 (IN), TLS handshake, Finished (20):

* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256

* ALPN, server did not agree to a protocol

* Server certificate:

* subject: C=US; ST=California; L=Palo Alto; O=VMware, Inc; CN=my.vmware.com

* start date: Aug 3 18:44:48 2016 GMT

* expire date: Aug 3 19:14:47 2019 GMT

* subjectAltName: host "my.vmware.com" matched cert's "my.vmware.com"

* SSL certificate verify ok.

> GET / HTTP/1.1

> Host: my.vmware.com

> User-Agent: curl/7.54.0

> Accept: */*

>

< HTTP/1.1 302 Found

< Date: Mon, 26 Feb 2018 15:21:16 GMT

< Server: Apache

< Location: http://my.vmware.com/group/vmware/home

< Content-Length: 222

< Content-Type: text/html; charset=iso-8859-1

< Set-Cookie: myvmware-www=!fDp+QDYmfSp61nOv8zFWbHIofySrMeWtbEn9Ssj2/t2bQpNjq2YDuvpxZc+sppwGO3sAym/NLcfw3w==; path=/

< Set-Cookie: nlbi_894469=IHwBfRXQ2Ca57LOrJ5dTDgAAAADzgdVlOxSVQefTu6ztz4Ur; path=/

< Set-Cookie: incap_ses_721_894469=uGiZSj4YUwoiLsHs+oEBCuwllFoAAAAAq9lvU423fOD9VAl54PAUmA==; path=/

< X-Iinfo: 11-14324803-14324812 NNNN CT(0 0 0) RT(1519658476182 164) q(0 0 0 0) r(2 2) U5

< X-CDN: Incapsula

<

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">

<title>302 Found</title>

</head><body>

<h1>Found</h1>

<p>The document has moved <a href="http://my.vmware.com/group/vmware/home">here</a>.</p>

</body></html>

* Connection #0 to host proxy.mydomain.local left intact

I've also tried manually setting HTTPS_PROXY and FTP_PROXY to the same proxy address/port in /etc/sysconfig/proxy, but to no help.

SeanKohler · ‎02-26-2018

I would say this is the best place to post vRSLCM questions for now...

I haven't run the product since beta and unfortunately didn't need to use a proxy. Is there any way you can attempt without proxy? (just see if everything otherwise works)

I only have a couple of thoughts here, that you may have already considered.

-- Error occurred while fetching access token from https://my.vmware.com. Status code : ''. Status message :

Interesting that the error is no error. Almost like it didn't get a response from the site through the proxy.

If you can't share your proxy config (even obscured), do you have both http and https configured to use the proxy? If you made that clear already, I missed it. (Why do I think HTTP? Because they might have a bug... pointing to my.vmware.com which redirects 301 to https. Longshot... just a thought.)

Joffer · ‎02-27-2018

Yes proxy set for both HTTP and HTTPS in /etc/sysconfig/proxy. Here is the config (http_proxy set by appliance webui)

root@vrslcm01 [ ~ ]# cat /etc/environment

#

# This file is parsed by pam_env module

#

# Syntax: simple "KEY=VAL" pairs on separate lines

#

PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/vmware/bin"

http_proxy=http://proxy.mydomain.local:80

and

root@vrslcm01 [ ~ ]# cat /etc/sysconfig/proxy

# Enable a generation of the proxy settings to the profile.

# This setting allows to turn the proxy on and off while

# preserving the particular proxy setup.

#

PROXY_ENABLED="yes"

# Some programs (e.g. wget) support proxies, if set in

# the environment.

# Example: HTTP_PROXY="http://proxy.provider.de:3128/"

HTTP_PROXY="http://proxy.mydomain.local:80"

# Example: HTTPS_PROXY="https://proxy.provider.de:3128/"

HTTPS_PROXY="http://proxy.mydomain.local:80"

# Example: FTP_PROXY="http://proxy.provider.de:3128/"

FTP_PROXY="http://proxy.mydomain.local:80"

# Example: GOPHER_PROXY="http://proxy.provider.de:3128/"

GOPHER_PROXY=""

# Example: SOCKS_PROXY="socks://proxy.example.com:8080"

SOCKS_PROXY=""

# Example: SOCKS5_SERVER="office-proxy.example.com:8881"

SOCKS5_SERVER=""

# Example: NO_PROXY="www.me.de, do.main, localhost"

NO_PROXY="localhost, 127.0.0.1, vcenter01.mydomain.local, vrops01.mydomain.local, vrli01.mydomain.local, vsa01.mydomain.local"

Here I've tried with HTTPS_PROXY and FTP_PROXY as well, since all has to go through it. The traffic to *.vmware.com should be transported/tunneled through without any inspection also.

(and why does VMware keep forgetting HTTPS_PROXY in their settings? Had to do the same manual config on vcenter etc to get it to check updates online via proxy as it was HTTPS traffic.

As you mentioned, it's a bit strange that it redirects the https://my.vmware.com to http://my.vmware.com/group/vmware/home . But the link will eventually be redirected back to HTTPS again:

root@vrslcm01 [ /var/log/vlcm ]# curl -vx proxy.mydomain.local:80 http://my.vmware.com/group/vmware/home
*   Trying PR.OX.Y.IP...
* TCP_NODELAY set
* Connected to proxy.mydomain.local (PR.OX.Y.IP) port 80 (#0)
> GET http://my.vmware.com/group/vmware/home HTTP/1.1
> Host: my.vmware.com
> User-Agent: curl/7.54.0
> Accept: */*
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 301 Moved Permanently
< Location: https://my.vmware.com/group/vmware/home
< Content-Length: 0
< Date: Tue, 27 Feb 2018 12:58:34 GMT
< Age: 0
< Via: 1.1 wsproxy.mydomain.local
<
* Connection #0 to host proxy.mydomain.local left intact

SeanKohler · ‎02-27-2018

So you don't have https_proxy configured in /etc/environment?

The log appears to be saying that it is looking for proxy configuration there. I don't know if /etc/sysconfig/proxy is providing proxy for vrslcm even if it is for curl.

Couple things to try, add these to /etc/environment and reboot. (obviously the first one exists)

http_proxy=http://proxy.mydomain.local:80

HTTP_PROXY=http://proxy.mydomain.local:80

https_proxy=http://proxy.mydomain.local:80

HTTPS_PROXY=http://proxy.mydomain.local:80

Remove the CAPS ones and reboot... try again.

This is a complete guess.... unfortunately I cannot test this at the moment.

Joffer · ‎03-01-2018

So both tests (with and without CAPS in environment file) failed the same way (I think I tried it before, but tested it again to be sure.):

2018-03-01 10:03:03.343 INFO [ForkJoinPool-1-worker-0] c.v.v.l.u.p.ProxySetupHelper - -- Loading Default Proxy File : /etc/environment

2018-03-01 10:03:03.351 INFO [ForkJoinPool-1-worker-0] c.v.v.l.u.p.ProxySetupHelper - -- Setting system proxy: host: proxy.mydomain.local, port: 80

...

2018-03-01 09:55:23.152 INFO [http://10.10.10.95:42246/core/document-index/queries/1519897921363000] c.v.v.l.u.RestHelperWithProxy - -- Connecting with Proxy host : proxy.mydomain.local and port 80

2018-03-01 09:55:23.154 ERROR [http://10.10.10.95:42246/core/document-index/queries/1519897921363000] c.v.v.l.d.m.h.MyVmwareDownloadRestClient - -- Error occurred while fetching access token from https://my.vmware.com.

java.lang.IllegalArgumentException: Username may not be null

Also, the /opt/vmware/share/vami/vami_set_proxy script that sets the proxy updates /etc/sysconfig/proxy, but only http_proxy/HTTP_PROXY.

WSFowler · ‎04-10-2018

Did you ever solve this? We are seeing the same error and I think it has to do with our proxy not requiring a username/password combo for services like this. Typically if we are supplying a username/password it's from a browser and a pac file is involved. For services like this, we whitelist the server/site combo.

I tried just putting a random username/password and that seemed to get me past getting a token from my.vmware.com. Little less than ideal though.

cwade · ‎09-10-2018

I realize that this is an older thread, but I too would like to know if this issue was resolved. We are also having the same issues with a proxy in vRSLCM.

All

Problem with vRealize Suite Lifecycle Manager proxy settings