There could be a number of reasons for the error. First check to see if there are enough resources available on the esxi host. Run the command nvidia-smi from ESXi.
You may have a VM consolidation issue in which the VMs are spanning the different gpu's and reserving chunks that don't align properly. See this official doc. There is a setting for vGPU.consolidation = "true"
Best Practices for Configuring 3D Rendering
Also, check to see if xorg is running properly on the esxi host. The was a recent bug that prevented it from starting.
VMware Knowledge Base