SRM 5.5 broken (SSL Exception) after vCenter 5.5 Update 3b

[Bleeder]

After installation of vCenter 5.5 Update 3b, SRM 5.5 no longer functions.  The vmware-dr logs show several SSL Exceptions "error: class Vmacore::Ssl::SSLException(SSL Exception: error:140000DB:SSL routines:SSL routines:short read)".

The VMware product interoperability matrixes for SRM have never shown any of the a/b/c/d/etc releases of vCenter, so I don't know if this is supposed to be working or not.

[admin]

There have been some changes to SSLv3 support in this vCenter Server release - it's now disabled by default. Could be related.

http://pubs.vmware.com/Release_Notes/en/vsphere/55/vsphere-vcenter-server-55u3b-release-notes.html

VMware KB: VMware vCenter Site Recovery Manager Server service fails to start after changing securit...

What exact build of Site Recovery Manager is this?

[Bleeder]

This is the latest public SRM 5.5 build (5.5.1.5 build 2653439).

[msjagadish]

Hello,

I believe the problem here is the vcenter server U3b. SSLv3 is disabled by default in vcenter server Update 3b. SRM 5.5.x requires sslv3 to operate. This is the reason why SRM stops working after the vcenter server is upgraded to 5.5 U3b.

Workaround : You have to enable sslv3 in vcenter server to make your SRM work normally again. The procedure to enable sslv3 in vcenter is outlined in the following link :

VMware KB: Enabling SSLv3 protocol on vSphere 5.5

Resolution : Upgraded both vcenter server and SRM to 6.0. Version 6.0 is not affected by this issue.

[admin]

I had a customer with this problem and they had to patch to 5.8.1 to resolve it. I don't think this is likely to be fixed in SRM 5.5.x

[cyberfed2727]

Just a follow on up the response of p_hall. Our team is being asked to upgrade vCenter to 5.5u3b (from 3a) along with our ESXi hosts afterwards.

My concern is with SRM, we are running 5.8.0.8607 currently. I understand that SRM will cease to work since 3b disables SSLv3.

If we upgrade to SRM 5.8.1 will it work with 5.5u3b vCenter/ESXi?

We cannot upgrade to 6.x because unfortunately our hardware is no longer on VMware's HCL (older IBM blades) for versions 6.x.

[admin]

I haven't found anything official that confirms this yet but from what I've seen, SRM 5.8.1 is compatible with the latest version of vCenter 5.5. It appears that SRM 5.8.1 does not require SSLv3 to be enabled in order to function.

It looks like 5.5.1.x or 5.8.0.x SRM will not work with this version of vCenter Server (without re-enabling SSLv3) but if you upgrade your SRM to 5.8.1 it should work ok.

[cyberfed2727]

Thanks, I called VMware and explained our scenario. They recommended we hold off on upgrading to 3B until VMware releases updated packages for both SRM and Horizon View.

[jordanovi]

Hi

The official doc is the compatibility matrix (and choose vCenter Server Requirements from the dropdown)

For SRM 5.8 - Compatibility Matrixes for vCenter Site Recovery Manager 5.8:

• Site Recovery Manager 5.8.1 is compatible with vCenter Server 5.5u3 and 5.5u2
• Site Recovery Manager 5.8.0 is compatible with vCenter Server 5.5u2

For SRM 5.5 - Compatibility Matrixes for vCenter Site Recovery Manager 5.5

Site Recovery Manager 5.5 and its updates are compatible with specific versions of vCenter Server.

• Site Recovery Manager 5.5.0 is compatible with vCenter Server 5.5.0
• Site Recovery Manager 5.5.1.x is compatible with vCenter Server 5.5 and 5.5u1.
• There is no Site Recovery Manager 5.5.2 release to correspond with the vCenter Server 5.5u2 release. Site Recovery Manager 5.5.1.x has been fully tested with and fully supports vCenter Server 5.5u1 and 5.5u2.

Hope this helps

Ivan

[Bleeder]

Yes, Site Recovery Manager 5.5.1 is listed as compatible with vCenter Server 5.5 U3 in the interoperability matrix.  The Site Recovery Manager 5.5.1 release notes also state "SRM 5.5.1.x has been fully tested with and fully supports vCenter Server 5.5u1, 5.5u2, and 5.5u3."

However, the fact remains that SRM 5.5.1 is broken with vCenter Server 5.5 U3b.

[cyberfed2727]

I looked at that as well prior to calling VMware. I have never put too much faith into those matrices. Regardless of what that shows I'm going with what VMware support said. They have seen lots of "issues" with 3B and other VMware component and said to wait.

[jordanovi]

We're working to clarify this in the release notes/interop matrix.. will update here when the updated docs are publicly available.

Thank you for the feedback

Ivan

[jordanovi]

Hi guys

The compatibility matrices are updated:

• SRM 5.8: https://www.vmware.com/support/srm/srm-compat-matrix-5-8.html
• SRM 5.5: https://www.vmware.com/support/srm/srm-compat-matrix-5-5.html

and dedicated KB created: http://kb.vmware.com/kb/2142487

Hope this helps

Ivan

[DigitalKiller]

I too was bitten by this issues where nothing in the release notes or the compatibility guides called this out as an issue. I have a ticket open, but have been told different things. re-enable on vCenter and the Web Client, then just vCenter. No mention of the hosts. I brought the hosts up, and the guy had no clue. So I escalated. Escalation engineer says I need to re-enable for every single service, which seems excessive. If you go by the ports that need to be open (see chart below from KB 1009562) I would think vCenter and the hosts that have SRM protected workloads are what is needed. I just want and official answer and is it too much to ask someone at VMware to test to be 100% sure.

80	HTTP	SRM	Remote vCenter Server	All management traffic to SRM Server goes to port 80 on the vCenter Server proxy system.
443	HTTPS	SRM	vCenter Server	Default SSL web port
902	TCP	SRM	Remote ESXi host	Traffic from the SRM Server on the recovery site to ESX hosts when recovering or testing virtual machines with IP customization, with configured callout commands on recovered virtual machines, or that use raw disk mapping (RDM). All NFC traffic for updating or patching the VMX files of virtual machines that are replicated using vSphere Replication use this port.

KB 2142487 was a mess yesterday where it read:

The issue is resolved in VMware Site Recovery Manager 5.8.1 available at VMware Downloads

To work around this issue when you do not want to upgrade,  re-enable SSLv3 on Port 902 on your VMware ESXi 5.5 Update 3b/ 6.0 Update 1 hosts. After upgrading VMware View Connection Server to 6.2, disable SSLv3 on the VMware ESXi 5.5 Update 3b/ 6.0 Update 1 hosts.

It looks to have been cleaned up, but it makes no mention of vCenter, just hosts, which I think is need as well. Can VMware clarify this? An unplanned update to 5.8.1 is not something I want to just go and do.

[cyberfed2727]

Supposedly the matrix was updated 2 days ago and now includes this very specific nugget of information:

• Site Recovery Manager 5.8.1 is compatible with vCenter Server 5.5u3 and 5.5u2. Site Recovery Manager 5.8.1 does not require SSLv3 to be enabled to work with vCenter Server 5.5u3.

So it sounds like SRM 5.8.1 should work with 5.5u3b with no issues or fussing with SSLv3. We just recently upgraded to SRM 5.8.1 on both our test/dev and production clusters. We are going to plan to upgrade our test/dev cluster from ESXi/vCenter 5.5u3a to 3b and see if this is indeed true and see if it works with SRM 5.8.1. I'm skeptical but hopeful. I'll post back our results.

[jordanovi]

I believe this is addressed in:

• kb.vmware.com/kb/2139396 - Enabling SSLv3 protocol on vSphere 5.5 (2139396)
• kb.vmware.com/kb/2057795 - Update sequence for VMware vSphere 5.5 and its compatible VMware products (2057795)
• and in What's New section in VMware vCenter Server 5.5 Update 3b Release Notes

Hope this helps

Ivan

[DigitalKiller]

Not really. The release notes make no mention of SRM. The update sequence lists 5.5, not 5.8.1. The :Enabling SSLv3 protocol on vSphere 5.5" document has since been updated to say "Site Recovery Manager (SRM 5.5 or SRM 5.8.0) might fail to start the SRM service after upgrading vCenter Server to 5.5 Update 3b"

The root issue is no where was it called out that 3B would break 5.5.x of SRM, so people to proceeded to update. That is bad of VMware, but what is done is done. What I am trying to get a handle on is where exactly does SSLv3 need to be re-enabled. KB 2142487 states after an update to 5.5 U3B SRM may not start and to re-enable SSLv3 on port 902 on the hosts. For me SRM was no working until I re-enabled SSLv3 on vCenter. I probably need to do the hosts as well to be covered, but sine support has given me different answers at different times, my confidence is low.

Really hoping VMware can provide concise guidance on getting SRM 5.5.1 fully functioning after updating to 5.5 U3B so we are protected while we evaluate the process of updating to 5.8.1.

[AndrewFSimpson]

Blanket response since I didn't see it in the thread.

Same issue as KB represented earlier.  Upgraded to vCenter 5.5u3, broke SRM 5.5.1.2.

I was able to to upgrade to SRM 5.8.1 from SRM 5.5.1.2 with no issues.  Part of the compatibility checking was to ensure you could log in and verify each PG/RP was working correct - otherwise it would remove them during upgrade.  I had 2 RP's that were our of whack and I couldn't log in at all.  Everything upgraded fine and came back in the same state.