Its not possible. The host kernel would need to mount the VMFS to access whatever file in it. so It cant boot from VMFS.
If you are building a nested ESXi than yes, you create a ESXi VM where the iso is on VMFS, where on this case the real ESXi host kernel is already running mounting the VMFS first (judging from your question, it seems you are not seeking this type of architecture).
The simplest solution to your question is via USB thumb drive, with ESXi image done with rufus.
Or the more elaborate, via auto deploy with stateless ESXi would do too, but you need to put the ISO on the auto deploy server as image profile then setup host profiles to deploy with.
