What happened to adding a work around to a future release?
While I understand that this probably isn't as high as fixing runtime issues in priority but for some of us, this is still important in allowing us to run a scripted install that is consistently the same across hundreds of systems which is made more difficult with this flawed implementation that was released...
Again, the reason that chainloading from EFI GRUB to VMware's bootloader does not work is because of bug(s) in GRUB. It's not because of a "flawed implementation" by VMware.
I did look into this further after my last post. I put a workaround into the VMware bootloader (mboot64.efi) that compensates for the GRUB bug in device path formation. With the workaround, it is possible to chain from GRUB to mboot and boot from local media. The workaround will be included in the upcoming major ESXi release that is in beta now. mboot64 is backward compatible, so it will also be possible to boot older versions of ESXi using the new mboot version.
Unfortunately, though, even with the workaround, chaining from GRUB to mboot64.efi still fails to network boot ESXi successfully. It appears that when chainloading, GRUB calls Stop on the EFI firmware's PXE base code protocol before handing off, so the protocol cannot be used to load the rest of ESXi over the network. I tested this only with GRUB2, so it's possible that it would work with GRUB Legacy, but I doubt it. I tried adding a workaround in mboot that calls Start again on the PXE base code protocol, but that doesn't work -- UEFI firmware apparently can't restart the protocol once it has been stopped.
If you only need to boot ESXi off of local media after chaining from GRUB to mboot, that will work with the new mboot version.
> mboot64 is backward compatible, so it will also be possible to boot older versions of ESXi using the new mboot version.
Thats very good news.
You seem to have the missing piece that I needed for a nice but not important project of mine.
Do you think that using the new mboot64 will allow to create a bootmenu for a LiveDVD that has the following boot options:
- boot ESXi 3.5 LiveCD
- boot ESXi 4.0 LiveCD
- boot ESXi 4.1 LiveCD
- boot ESXi 5.0 LiveCD
- boot ESXi 5.1 LiveCD
- boot ESXi 5.5 LiveCD
- boot ESXi 6.0 LiveCD
- boot Ubuntu 64 LiveCD
- boot CentOS 64 LiveCD
I had to give up as I had problems with incomptible mboot versions.
If you ever feel bored - the most part of the work is already done - a few hours together and we could create a very cool tool for ESXi-administrators
Unfortunately, PXE/Network install is certainly our preferred method given our current provisioning system. We have over a thousand hypers spread across 10 physical datacenters, relying on physical media would not be feasible while utilizing alternate methods would require retooling of our provisioning systems...
I can certainly see your point of view regarding it being a "flawed implementation" but perhaps it should be termed a "non-functional implementation", vmware has their published doc regarding PXE implementation for vSphere 6.0 which includes a section for UEFI where I'm not seeing anything stating it doesn't work(I certainly could be missing it but if I am if should be more obvious)...
Perhaps this doc should be edited indicating that that install method doesn't work or perhaps pulling out the UEFI sections altogether...
Otherwise VMware should engineer a functional solution...
As far as I'm aware, PXE-booting VMware's mboot.efi directly should work just fine, as documented in the PDF you reference. Could you explain why that is not an adequate solution for your environment?
If you choose to use a 3rd-party bootloader such as GRUB, and you encounter difficulties due to a defect in that 3rd-party bootloader, it would not seem to be reasonable cause for us to no longer document the supported configuration of EFI PXE-booting with VMware's own mboot.efi.
Feel free to provide further explanation if I am missing your point.
I personally wrote the document that you linked above and personally developed and tested every scenario. We network boot in UEFI mode daily within VMware.
The one thing that does not work is chaining to our bootloader from GRUB, due to an issue with GRUB as previously discussed. Note that the document above does not describe any scenarios that involve chaining from GRUB, only booting mboot64 directly or chaining from ipxe to mboot64.
It would be good for us to document the incompatibility with chaining from GRUB. I wasn't aware of it the time I was working on the document. Perhaps a KB article and/or a note in the next rev of the document.
ESXi's mboot.c32 and mboot64.efi have similar names to a multiboot plugin that comes with syslinux/pxelinux, but they aren't meant to be general-purpose multiboot standard bootloaders that could boot a Linux kernel. ESXi started off using the multiboot standard, but we've been slowly making additions (which should be compatible, but aren't tested with anything other than ESXi). So although the newest ESXi mboot should be able to boot old ESXi versions going back quite far (possibly all the way to 3.5, though I haven't personally tested back that far), I would be more surprised if it does boot Linux than if it doesn't.
Just like many others in the thread I am switching to UEFI based system ESXi install. The UEFI/mboot.efi worked fine for me.
I do have a request for feature though. Currently the default is that it starts installing after 5 seconds of countdown. I was wondering if a menu item can be added where the default is local-disk boot instead - just like all other boot loaders have out there.
The way it is right now with mboot.efi/boot.cfg is that there is no protection against someone accidentally hitting F12... and will result in system getting re-installed.
Any word on this? Did this make it into 6.7?
The workaround for the GRUB bug where it does not null-terminate device paths is in the 6.7 ESXi bootloader, so I think chaining from GRUB to our bootloader will work for disk and ISO image boots now.
However, as explained earlier in this thread, the reason that chaining from GRUB into VMware's bootloader during PXE boot does not work is a limitation in GRUB. GRUB calls Stop on the UEFI PXE prototol before handing off to our bootloader, so our bootloader cannot load any more files over the network. It's not possible to restart the protocol (I tried). I don't know of a way to work around this on our side. You could perhaps try modifying GRUB to not do that.
More simply, I suggest using something other than GRUB to create menus, also as discussed earlier in this thread.
1 person found this helpful
I am actually inserting kickstart files within the iso image,. I've had this working with legacy mode for quite a while, with multiple entries in isolinux.cfg, but have been struggling to get this to work with UEFI.
Can you point me to an example of what the efi/boot/boot.cfg would look like? Would I be putting the menu items into bootx64.conf, or maybe grub.conf? If so, can you tell me what they would look like? Any help you can provide would be greatly appreciated.
1 person found this helpful
> I am actually inserting kickstart files within the iso image,. I've had this working with legacy mode for quite a while, with multiple entries in isolinux.cfg, but have been struggling to get this to work with UEFI.
> Can you point me to an example of what the efi/boot/boot.cfg would look like? Would I be putting the menu items into bootx64.conf, or maybe grub.conf? If so, can you tell me what they would look like? Any help you can provide would be greatly appreciated.
Hmm, I think you're saying you want to have multiple different kickstart files on an ISO image that you can select from using a menu. I see where the missing piece is here. An ESXi ISO image uses isolinux as the first stage for booting in legacy BIOS mode, and that has a menu feature. I gather you're using that. But for booting in UEFI mode, there's nothing on the ISO image that has a menu feature. The first stage is isobounce (which is installed as bootx64 in the ISO's second El Torito image), which loads an ISO9660 filesystem driver from the El Torito image and then chains to mboot (which is installed as bootx64 on the iso9660 filesystem). Neither isobounce nor mboot has a menu feature.
If you want to create something on the ISO that shows a menu when booted in UEFI mode, there are a few alternatives you could try. I haven't done this myself, though, so I'm not aware if there are any pitfalls or which is best.
Personally I'd probably try VMware menu.efi, but mostly that's because I wrote it and we use it a lot internally at VMware. We haven't advertised it for customer use, but it's available under the GPL as part of our bootloader (esx-boot) source disclosure. There's documentation at the top of the source code at esx-boot/menu.c at master · vmware/esx-boot · GitHub . It's not very easy to compile the esx-boot package that it lives in, though.
You could also try grub, rEFInd, or rEFIt.
Whatever you try as a menu program, you will have to get the ISO image to execute it instead of booting right into ESXi. I suppose you could modify the ISO9660 filesystem to rename its \efi\boot\bootx64.efi to something else (say mboot.efi), and put in the menu program you choose as \efi\boot\bootx64.efi. Alternatively, you could modify the second El Torito image to invoke your menu program -- that seems like it would be more work, though.
I was in the same boat and have been using the isolinux Menu with ks files for ESXi installs on legacy BIOS. But we have installations also with UEFI for which this didn't work.
For me solution for hosts using UEFI was to keep the same setup with KS files in iso, but skip the menu. Instead I adjusted \EFI\BOOT\BOOT.CFG to have a long timeout and added CFG to kernelops line (so we don't have to type it all every time). When ESXI install boots we press Shift-O and change ks filename, rest is the same as having a menu.
Thank you very much TimMann for a description of the boot process.