I have exactly the same problem with vCenter Server Appliance 5.1.0.5300 Build 947940 after update.

I can log perfectly if i type my windows credentials, but error appear if I check "use Windows session credentials"