vSphere ESXi 6 UEFI PXE Deployment via WDS

[Jungo83]

Hello

I'm trying to create a PXE deployment for vSphere ESXi 6 through WDS.

My environment is this:

- WDS on Server 2012 R2

- WDS is the DHCP server and WDS server

- WDS is able to properly deploy UEFI-based windows servers

I would like to also deploy ESXi via the same WDS server, while maintaining the existing capabilities.

I read multiple documentation stating that PXE booting UEFI ESX is possible, but none talks about how to implement it in WDS

I already tried to manipulate my WDS with the mboot.efi file and such, but nothing work

Has anyone dealt with that scenario and had success?

Please don't tell me to build another PXE server or eliminate WDS or move to alternate solutions (such as auto-deploy). The need is that a single WDS server will deploy both ESXi and Windows based on UEFI

Any help is much appreciated

Thanks,

Omer

[dariusd]

Hi Omer, and welcome to the VMware Communities!

PXE booting ESXi from WDS is definitely not something we officially support.  I've never tried it myself.  As long as there aren't any limitations with the WDS DHCP and TFTP services, there might not be any technical reason why it couldn't be made to work given a big enough hammer. 

I already tried to manipulate my WDS with the mboot.efi file and such, but nothing work

How far is it getting when it tries to boot?  Is the PXE client at least fetching mboot.efi and then giving an error message from mboot?  If you can provide any error messages that appear, or screenshots, or a description of what happens during the attempt to boot, that could help us figure out where it's failing.

Cheers,

--

Darius

[Jungo83]

Hi Darius

Thanks for assisting. I will give a bit more background -

In WDS the folder structure is RemoteInstall/Boot/%arch% (x64, x86, etc.)

I looked at a few documents (such as this) and to the best of my understand, I extracted the ESXi ISO to a sub-folder - Boot/x64/ESXi6U2

I took the /EFI/Boot/BOOTX64.EFI and copied it to Boot/x64/ESXi6U2 and renamed it to mboot.efi

Once I try to boot the ESXi I get the following error:

Configuration error while parsing Boot/x64/ESXi6U2/boot.cfg
Fatal error: 24 (TFTP error)

I tried editing the boot.cfg file and enter the relative path (I noticed that in some documentation) but that didn't work also - I tried to change / to /ESXi6U2/

I'm unsure how to go about verifying that my TFTP is configured as expected. I removed all firewall so this won't be a factor and I'm in a single LAN segment

Both the WDS and ESXi are VMs running on VMWare workstation on my laptop. I hope this doesn't present any issues

Thanks,

Omer

[dariusd]

On the server, do you have boot.cfg in the same folder as the copy of mboot.efi being used for booting?  It looks like it should be at RemoteInstall/Boot/x64/ESXi6U2/boot.cfg .

Usually the best way to troubleshoot PXE boot issues is by using a packet capture tool (i.e. Wireshark) and a copy of the relevant protocol specifications for DHCP‌ and TFTP to fill in the details.  After DHCP completes, you will see a TFTP Read Request (RRQ) for mboot.efi followed by an exchange of DATA and ACK packets, then you should see another RRQ for a boot configuration filename which includes the MAC address (and that should receive an ERROR response because the file doesn't exist), followed by an RRQ for RemoteInstall/Boot/x64/ESXi6U2/boot.cfg .  Wireshark should show how the outcome of that request, which will hopefully give a clue about where to look in WDS to remedy the problem.

Cheers,

--

Darius

[Jungo83]

Thanks for the info Darius.

I will install Wireshark and do some testing (I'm not a network expert but I will manage I believe)

In the meantime, I added screenshots with the relevant info. Let me know if this helps in any way

Thanks again for all your help!

ESXi boot error

WDS File locations:

Boot.cfg configuration:

bootstate=0

title=Loading ESXi installer

timeout=5

kernel=/tboot.b00

kernelopt=runweasel

modules=/b.b00 --- /jumpstrt.gz --- /useropts.gz --- /k.b00 --- /chardevs.b00 --- /a.b00 --- /user.b00 --- /uc_intel.b00 --- /uc_amd.b00 --- /sb.v00 --- /s.v00 --- /mtip32xx.v00 --- /ata_pata.v00 --- /ata_pata.v01 --- /ata_pata.v02 --- /ata_pata.v03 --- /ata_pata.v04 --- /ata_pata.v05 --- /ata_pata.v06 --- /ata_pata.v07 --- /block_cc.v00 --- /ehci_ehc.v00 --- /elxnet.v00 --- /emulex_e.v00 --- /weaselin.t00 --- /esx_dvfi.v00 --- /esx_ui.v00 --- /ima_qla4.v00 --- /ipmi_ipm.v00 --- /ipmi_ipm.v01 --- /ipmi_ipm.v02 --- /lpfc.v00 --- /lsi_mr3.v00 --- /lsi_msgp.v00 --- /lsu_hp_h.v00 --- /lsu_lsi_.v00 --- /lsu_lsi_.v01 --- /lsu_lsi_.v02 --- /lsu_lsi_.v03 --- /lsu_lsi_.v04 --- /misc_cni.v00 --- /misc_dri.v00 --- /net_bnx2.v00 --- /net_bnx2.v01 --- /net_cnic.v00 --- /net_e100.v00 --- /net_e100.v01 --- /net_enic.v00 --- /net_forc.v00 --- /net_igb.v00 --- /net_ixgb.v00 --- /net_mlx4.v00 --- /net_mlx4.v01 --- /net_nx_n.v00 --- /net_tg3.v00 --- /net_vmxn.v00 --- /nmlx4_co.v00 --- /nmlx4_en.v00 --- /nmlx4_rd.v00 --- /nvme.v00 --- /ohci_usb.v00 --- /qlnative.v00 --- /rste.v00 --- /sata_ahc.v00 --- /sata_ata.v00 --- /sata_sat.v00 --- /sata_sat.v01 --- /sata_sat.v02 --- /sata_sat.v03 --- /sata_sat.v04 --- /scsi_aac.v00 --- /scsi_adp.v00 --- /scsi_aic.v00 --- /scsi_bnx.v00 --- /scsi_bnx.v01 --- /scsi_fni.v00 --- /scsi_hps.v00 --- /scsi_ips.v00 --- /scsi_meg.v00 --- /scsi_meg.v01 --- /scsi_meg.v02 --- /scsi_mpt.v00 --- /scsi_mpt.v01 --- /scsi_mpt.v02 --- /scsi_qla.v00 --- /uhci_usb.v00 --- /vsan.v00 --- /vsanheal.v00 --- /vsanmgmt.v00 --- /xhci_xhc.v00 --- /tools.t00 --- /xorg.v00 --- /imgdb.tgz --- /imgpayld.tgz

build=

updated=0

[Jungo83]

Hi Darius

Wireshark gave me a direction. When looking at the capture I see that the ESXi is getting "access violation" when trying to download data from the WDS.

I used this link and modified the registry and it looks like this error is now cleared!

However, now I'm getting a new error 🙂

Loading /tboot.b00
Error loading /tboot.b00
Compressed MD5: 000000000.... (a lot of 000)
Decompressed MD5: 000000000.... (a lot of 000)
Fatal Error: 24 (TFTP error)

According to the wireshark capture it looks like both files boot.cfg and mboot.efi are found and accessible, but I'm getting access violation on /tboot.b00

I was thinking I might need to change the path in the boot.cfg to include my subfolder (similar to what's described here) -

We need to append our boot path to all the files listed in this file. This path needs to match the path of our ESXi files.

but even after I added /ESXi6U2/ before all the files, I'm still getting the same error...

I feel like I'm close but missing some key point... any idea on how to properly configure the boot.cfg file? Or maybe it's configured correctly and the error is different?

Thanks,

Omer

[dariusd · Accepted solution]

Nice work!  (And Wireshark is awesome!)  Try putting this into your boot.cfg:

bootstate=0

title=Loading ESXi installer

timeout=5

kernel=tboot.b00

kernelopt=runweasel

prefix=/Boot/x64/ESXi6U2

modules=b.b00 --- jumpstrt.gz --- useropts.gz --- k.b00 --- chardevs.b00 --- a.b00 --- user.b00 --- uc_intel.b00 --- uc_amd.b00 --- sb.v00 --- s.v00 --- mtip32xx.v00 --- ata_pata.v00 --- ata_pata.v01 --- ata_pata.v02 --- ata_pata.v03 --- ata_pata.v04 --- ata_pata.v05 --- ata_pata.v06 --- ata_pata.v07 --- block_cc.v00 --- ehci_ehc.v00 --- elxnet.v00 --- emulex_e.v00 --- weaselin.t00 --- esx_dvfi.v00 --- esx_ui.v00 --- ima_qla4.v00 --- ipmi_ipm.v00 --- ipmi_ipm.v01 --- ipmi_ipm.v02 --- lpfc.v00 --- lsi_mr3.v00 --- lsi_msgp.v00 --- lsu_hp_h.v00 --- lsu_lsi_.v00 --- lsu_lsi_.v01 --- lsu_lsi_.v02 --- lsu_lsi_.v03 --- lsu_lsi_.v04 --- misc_cni.v00 --- misc_dri.v00 --- net_bnx2.v00 --- net_bnx2.v01 --- net_cnic.v00 --- net_e100.v00 --- net_e100.v01 --- net_enic.v00 --- net_forc.v00 --- net_igb.v00 --- net_ixgb.v00 --- net_mlx4.v00 --- net_mlx4.v01 --- net_nx_n.v00 --- net_tg3.v00 --- net_vmxn.v00 --- nmlx4_co.v00 --- nmlx4_en.v00 --- nmlx4_rd.v00 --- nvme.v00 --- ohci_usb.v00 --- qlnative.v00 --- rste.v00 --- sata_ahc.v00 --- sata_ata.v00 --- sata_sat.v00 --- sata_sat.v01 --- sata_sat.v02 --- sata_sat.v03 --- sata_sat.v04 --- scsi_aac.v00 --- scsi_adp.v00 --- scsi_aic.v00 --- scsi_bnx.v00 --- scsi_bnx.v01 --- scsi_fni.v00 --- scsi_hps.v00 --- scsi_ips.v00 --- scsi_meg.v00 --- scsi_meg.v01 --- scsi_meg.v02 --- scsi_mpt.v00 --- scsi_mpt.v01 --- scsi_mpt.v02 --- scsi_qla.v00 --- uhci_usb.v00 --- vsan.v00 --- vsanheal.v00 --- vsanmgmt.v00 --- xhci_xhc.v00 --- tools.t00 --- xorg.v00 --- imgdb.tgz --- imgpayld.tgz

build=

updated=0

This is following the instructions in https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/techpaper/vsphere-esxi-vcenter-ser... on page 10, step 6... adding the "prefix=" line pointing to the path within your TFTP root directory, and removing the leading slashes from each of the filenames.

Let us know if that gets you any further!

Cheers,

--

Darius

[Jungo83]

Success!

This did the trick. I was able to successfully deploy the EFI-Based ESXi from WDS.

For some reason it took the installer a long time to load. Not sure why yet, but I will do some more tests to see what's causing that.

My next step is to embed a ks.cfg file on a web-server installed on the WDS so the installation can be fully unattended.

I saw a few articles regarding that so I will give that a try and if it's not working I will post a new question

Darius, thank you very much for all the help during this.

I've been beating myself up for a week around that and finally nailed it with your great assistance and direction.

Omer

[dariusd]

AWESOME!  Glad I could be of assistance.

If you're PXE booting ESXi on [U]EFI on Workstation, it should be possible to substantially improve the performance of PXE by manually editing the PXE client VM's configuration (.vmx) file and adding the following setting:

   vnet.recvClusterSize = "1"

There's a theoretical performance impact associated with that option if your ESXi VM is going to see absolutely massive amounts of network traffic once it's up and running -- Workstation will use a bit more of your host's CPU time while handling network packets -- but you should see a massive improvement in UEFI PXE performance in the guest, and the performance impact of using the custom vnet.recvClusterSize option really should be near-negligible unless you need to use the full "bandwidth" of the VM's virtual network adapters.

Have fun there!

--

Darius

[Jungo83]

Hi Darius

The addition to the .vmx did the trick and now the deployment works much faster

The full process to make it happen is documented here on my blog

In addition, I added a step to host the ks.cfg on IIS on the same WDS server to make the installation fully automatic

Feel free to spread the word if you see others are in need

Thanks again for all your help

Omer

[Jungo83]

Hi dariusd‌

I managed to fully deploy the ESXi through WDS in my lab. It's working great!

I'm using a simple ks.cfg file for some basic settings and there's one option I'm missing

How can I disable the IPV6 during the installation? I tried various commands and couldn't find the right way

I know it's a one-liner and I'm hoping you can nail it for me

Thanks,

Omer

[VINEETHMTEST]

Hi Jungo83,

Request your guidance for Fully deployment of ESXi using WDS and later kickstart to complete the action.

Let me know if you can help me here.

While configuration of WDS i am facing issue.

Thanks,

Vineeth

[VINEETHMTEST]

Any updated on this part?