Are there any special requirements for receiving multicast messages (for example, enabling promiscuous mode on the port group) or should this just work out of the box?
I'm using standard switches and trunk ports with VLAN tagging.
Since multicast is UDP based it will just work. The real trick is configuring your external switches to transport multicast correctly. It's not broken on your ESXi setup it's 100 switch setup that munges it. We have used multicast with Weblogic and oracle RACK and even VXLAN for a long time. All our problems came from the physical switches not being configured to carry multicast traffic.
I hope it helps let me know if not.
Thanks for the information. Our network team have confirmed that multicast is enabled on the switch ports, so perhaps I need them to check again? Do you have a sample cisco configuration that I could reference to compare our settings? It's also not out of the question that the application team have mis-configured their app so I will get them to check that as well.
I am sorry to say I don't we run a juniper shop. When we were testing it was really helpful to remove the application from the mix. We used a pretty simple tool to verify multicast worked:
This allowed us to ensure the networking changes were correct without the application people getting into the mix. We then had problems with the application people but could always fall back to the fact that multicast works on our tests.
Cisco seems to have a lot of documentation on multicast but I am not a network guy so I cannot translate them into usable options.
I hope it helps..