If you want to connect 3 esxi-hosts, then outside-work (maybe some NAS or whatever), than I definitely think it is worth to get switch. That's what switch is made for. I do not think that argument about price is so serious, you can get some entry-level 8-port 10gbit switch for less than 1k €/$. FYI, I'm using d-link dxs-1210-10ts (new for ~700€, 8x 10Gbase, 2x SFP+) and I think it is money well spent.
Solution without switch is acceptable really only for direct connection between 2 hosts (as your original question was). With every new connection it is getting more and more complicated, and at the end you might pay more for all those NICs than for 10gbit switch...