Rob,
thanks.
I am probably personally one of the least paranoid about network security and segmentation (probably because I am not a network expert

). I continue to stress on that point because talking to partners and customers that's the concern I usually hear.
There are 3 scenarios that you could usually bump into as far as I have seen so far:
1- no paranoia: collapse all network zones (COS, VMOTION, VMs) on a single cable and leverage VLANs / PortGroups for everything
2- low paranoia: use dedicated NICs for COS, VMOTION, and VMs. VMs that are on different subnets/networks would use VLANs / PortGroups
3- medium paranoia: use dedicated NICs for COS, VMOTION, and also dedicated NICs for VMs that are on different subnets/networks (NO VLANs / PortGroups).
4- high paranoia: use dedicated ESX servers for hosting VMs that are on different subnets/networks.
The bigger the organization is, the more paranoid they tend to be. So #1 would be for the SMB shops etc etc.
As far as I can say the vast majority of customers with a relatively medium/big VMware deployment are using #2 and #3.
You might argue that, if you are so paranoid than you shouldn't even have, on the same host a mix of COS, VMotion and VMs if you really really need to separate security zones (and I agree that the server itself in this case is a potential security threat as you are bundling into a single physical servers network connections that are supposed to be either completely physically separated or separated by firewalls - I think). However this is not technically possible since each ESX host has to have VMs dedicated NICs, COS NICs, VMotion NICs etc etc.
What it is technically possible and what it is not plays a role as well. The problem of the Converged Network at the moment is that it's technically possible to maintain a certain and higher level of separation (as per #3 and #4) above. There is no doubt that it would be better to have only a couple of wires coming out the server and that's it and we would all be doing that if that was the only choice (I think).
It's a trade-off at this point between the level of paranoid you need to keep and what you could achieve if you relax it a little bit. As Ken stated the fact that this is a brand new technology doesn't help relaxing the requirements.
Is this a good summary?
Massimo.