Is it possible to run opensource technologies (kvm, open vswitch, ceph) beside vmware products (ESXi, NSX, vSAN) on the same VIO deployment in separated clusters?
No, it's not possible to run KVM, Open vSwitch or Ceph when using VIO.
VIO supports any datastore supported by ESXi, it doesn't have to be vSAN. You can also run VIO without NSX (VLAN backed dvSwitch)
ESXi is a must for VIO.