I ended up rebooting the server and it just came right. Added a new port group and it's all good. Why it needed a reboot unsure.
I am facing the same issue with fresh installation of ESXi 6.5. I have tried changing the browsers but no use.
The only workaround I have found out is to logon and then login again and it starts working.