vCO 5.5.3 & vRO 6.0.4: SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure

[jauling]

Hello.

Our production system is vCO 5.5.1, and I'm looking to do incremental upgrade to 5.5.3, and at the same time testing 6.0.4 and seeing how the workflows react to the major version change. All are appliance installs.

So far, testing has been good. But we have some legacy REST code that runs on older Perl 5.12.1, which uses LWP routines, and these are failing hard on these versions. For 5.5.3, we cloned our production 5.5.1 instance, then upgraded it to 5.5.3. For 6.0.4, we stood up a new instance, and imported configuration and packages. We also have some newer REST code written in Ruby, which seems to work fine with these newer versions.

But, for whatever reason, the Perl code that uses LWP is failing hard on these upgraded versions. I know that newer releases of Java 7 and Java 8 disable SSL3, so I took the effort to re-enable these protocols. I've gotten so far as to confirm that SSLv3 and TLS1 works fine, by comparing output using openssl s_client -connect <VCOHOSTNAME>:8281, then using -tls1 as well as -ssl3 to confirm. The output is the same between our production 5.5.1 instance and both by test 5.5.3 and 6.0.4 versions now. But, our Perl LWP code still barfs on connecting to these newer vCO/vRO releases.

Error is:

LWP::Protocol::https::Socket: SSL connect attempt failed with unknown errorerror:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure at /CUSTOM/perl/lib/site_perl/5.12.1/LWP/Protocol/http.pm line 51

Does anyone have any thoughts on what else I can do from the vRO/vCO side? I've touched both /etc/vco/app-server/server.xml and /usr/java/jre-vmware/lib/security/java.security files to enable SSLv3 and TLS1. Yes, I know that enabling these protocols isn't exactly best practice, but we can't get rid of this legacy Perl code yet.

Thanks!

[jauling · Accepted solution]

After battling this for multiple days I found a solution. The Perl REST module I use depends on an older LWP module, which looks to use SSLv2Hello 'pseudo-protocol'. I found this out after trial and error. The Java used in both vCO 5.5.3 as well as vRO 6.0.4 not only explicitly disable SSLv3, but it also disables by default the SSLv2Hello protocol.

See this Java article, particularly the very bottom:

CVE-2014-3566 - Instructions to Mitigate the SSL v3.0 Vulnerability (aka "Poodle Attack") in Java SE

- "SSLv2Hello" is a special pseudo-protocol that controls whether the initial SSL/TLS client message is packaged using the older SSLv2 format or the standard newer (v3+) format.  It is not a real protocol, it is strictly a compatibility mechanism from the early days of SSL/TLS.

Enabling "SSLv2Hello" on the server side allows the inbound connection to use either format.  If "SSLv2Hello" is specified as an enabled Protocol on the client side, the server MUST be prepared to accept it (i.e. "SSLv2Hello" or equivalent) or else the SSL/TLS negotiation WILL FAIL.  This applies for servers of any type, including those based on other SSL/TLS implementations.  See the appropriate documentation for the details on how to enable/disable the SSL 2.0-compatible ClientHello format on the client and/or server if necessary.

    JDK 5/6 enables SSLv2Hello by default on both the client and server sides.  (Will send and receive SSLv2Hellos)

    JDK 7-9 enables SSLv2Hello on the server side only.  (Will not send, but will accept SSLv2Hellos)

I added SSLv2Hello to the list of sslEnabledProtocols in /etc/vco/app-server/server.xml, and my error went away! I hope this can be helpful to others.