We have vCenter with ELM as well and unique vDS which are used from serveral (stretched) Clusters. I cant remember if we have move hosts between those clusters after the was initialize add, but i cant see a no go or short stopper.
So try and report!
There is no dependency or direct relation between Cluster and VDS objects in your virtual infrastructure. They have also two different view in the vCenter. So I want to say you can have many hosts from different clusters that are connected to a single VDS and on the other hand hosts inside a single cluster can be connected to the different VDS.Please mark my comment as the Correct Answer if this solution resolved your problem
Thank you everyone!