Running Buster/Bullseye VM on ESXi-Arm on a raspberry pi

[jannegpriv]

Hi!

I'm trying to follow instructions on YT for running Buster or Bullseye VM on ESXi-Arm on a raspberry pi:

I've downloaded and converted the arm64 ISO img to vmdk using qemu-img on my Mac, I've also downloaded the Debian netinst ISO and uploaded them to my datastore.

I then created a thin provisioning version of the vmdk using vmkfstools on my ESXi raspberry host.

I then created a VM using the converted vmdk HDD and pointing CD drive to the netinst ISO and then booted up into netinst rescue mode and then mounted boot partition on /dev/sda1 and raspios on /dev/sda2 and then executed a shell on /dev/sda2.

Then I did an apt update and then installed linux-image-arm64 and grub-efi-arm64 packages and then run:

grub-install --efi-directory=/boot

but then I got the following warning:

EFI Variables are not supported on this system

There is an /boot/EFI folder on /dev/sda1 (that contains a lot of subfolders).

I've tried using the exact same versions as in the instructions but I probably then use newer versions of grub that causes this error.

What causes this to happen? Is there any easy fix?

Are there instructions else where that goes through how to run RaspiOS (Buster/Bullseye) on VM on ESXi on arm64/raspberry pi?

BR,

/Janne

[cyprienlaplace]

Hello Janne,

What are these instructions on YT?

Are you not able to just use the debian installer ISO with a "New Disk"? There should be no need of converting anything.

Cyprien

[jannegpriv]

Hi! Cyprien!

I've tried to follow instructions in this video:
https://www.youtube.com/watch?v=JN1h2ztUHLo&t=205s

I'm not trying to install Debian using netinst, I'm trying to install a downloaded RaspiOS arm64 img file but using the netinst to be able to add grub EFI boot.
I've tried to use the exact same versions of OS img and netinst ISO as in the video, but since I use latest repos I will install a later linux-image.arm64 and grun-efi-arm64 package than in the video, I guess that is what causing my error?

BR,

/Janne

[cyprienlaplace]

Alright, I will try the latest Raspberry Pi OS images and see what I can find.

[cyprienlaplace]

Just to confirm, is this the Raspi OS you're trying to install?
https://downloads.raspberrypi.org/raspios_arm64/images/raspios_arm64-2021-11-08/

[jannegpriv]

Hi!

Yes, I've tried both that one and also the 2020-08 Buster version:

https://downloads.raspberrypi.org/raspios_arm64/images/raspios_arm64-2020-08-24/

I've also tried latest version of Debian netinst and this version:

https://linuxtracker.org/index.php?page=downloadcheck&id=28fb59c0d4cac9af7d9de3074c6d440af57bb398

but I got the same error.

When I used the Bullseye version I had to resize the HDD from 4G to 10G before being able to install linux-image and grub since the Bullseye version occupies more disc space than Buster.

Big thanks for trying to find a solution! 🙂

BR,

/Janne

[cyprienlaplace · Accepted solution]

Here is what I did, using debian 11 netinst ISO:

1/ Expand the size of the virtual disk. Default size is not big enough to install the additional linux kernel. This can be done from the web interface.

2/ First edit /etc/fstab to put /dev/sda1 into /boot/efi. Debian needs an ext4 partition for /boot.. Also add efivarfs to solve the "EFI Variables are not supported on this system" issue:
PARTUUID=ae083906-01 /boot/efi vfat defaults 0 2
none /sys/firmware/efi/efivars efivarfs defaults

3/ umount /boot && mkdir /boot/efi && mount /boot/efi && mount /sys/firmware/efi/efivars

4/ Increase root partition size: raspi-config --expand-rootfs ; resize2fs /dev/sda2

5/ install the kernel: apt update && apt install linux-image-arm64 grub-efi-arm64

6/ install grub: grub-install ; update-grub2

7/ exit, eject the cdrom and reboot 😀

[jannegpriv]

Works like a charm! 🙂

Thanks for helping out!

BR,

/Janne

[Milad_M]

I followed your instruction, but  still get the error "EFI Variables are not supported on this system". any thoughts ?

[cyprienlaplace]

Hi Milad_M, what ISO have you been using (full name please)?

[Milad_M]

Hi,

I used the latest versions

pi image : 2022-04-04-raspios-bullseye-arm64.img.xz 

debian net :  debian-11.3.0-arm64-netinst.iso

[cyprienlaplace]

Hi, so I went on using the same files than you, and was able to install the linux kernel and grub. I added a few more information below, so please try again and check that the output matches (especially the one about the EFI variables).

1/ Expand the size of the virtual disk. Default size is not big enough to install the additional linux kernel. This can be done from the web interface. I used 8 GB.

2/ First edit /etc/fstab to put the first partition into /boot/efi. Debian needs an ext4 partition for /boot.. Also add efivarfs to solve the "EFI Variables are not supported on this system" issue:

proc /proc proc defaults 0 0
PARTUUID=50a67c3e-01 /boot/efi vfat defaults,flush 0 2
PARTUUID=50a67c3e-02 / ext4 defaults,noatime 0 1
none /sys/firmware/efi/efivars efivarfs defaults

3/ umount /boot && mkdir /boot/efi && mount /boot/efi && mount /sys/firmware/efi/efivars

You should now be able to see the EFI variables:

root@debian:~# ls /sys/firmware/efi/efivars/
Boot0000-8be4df61-93ca-11d2-aa0d-00e098032b8c
Boot0001-8be4df61-93ca-11d2-aa0d-00e098032b8c
Boot0002-8be4df61-93ca-11d2-aa0d-00e098032b8c
Boot0003-8be4df61-93ca-11d2-aa0d-00e098032b8c
Boot0004-8be4df61-93ca-11d2-aa0d-00e098032b8c
Boot0005-8be4df61-93ca-11d2-aa0d-00e098032b8c
BootCurrent-8be4df61-93ca-11d2-aa0d-00e098032b8c
BootOptionSupport-8be4df61-93ca-11d2-aa0d-00e098032b8c
BootOrder-8be4df61-93ca-11d2-aa0d-00e098032b8c
ConIn-8be4df61-93ca-11d2-aa0d-00e098032b8c
ConInDev-8be4df61-93ca-11d2-aa0d-00e098032b8c
ConOut-8be4df61-93ca-11d2-aa0d-00e098032b8c
ConOutDev-8be4df61-93ca-11d2-aa0d-00e098032b8c
db-d719b2cb-3d3a-4596-a3bc-dad00e67656f
dbx-d719b2cb-3d3a-4596-a3bc-dad00e67656f
ErrOut-8be4df61-93ca-11d2-aa0d-00e098032b8c
ErrOutDev-8be4df61-93ca-11d2-aa0d-00e098032b8c
KEK-8be4df61-93ca-11d2-aa0d-00e098032b8c
Lang-8be4df61-93ca-11d2-aa0d-00e098032b8c
LangCodes-8be4df61-93ca-11d2-aa0d-00e098032b8c
OsIndicationsSupported-8be4df61-93ca-11d2-aa0d-00e098032b8c
PerfDataMemAddr-76b6bdfa-2acd-4462-9e3f-cb58c969d937
PK-8be4df61-93ca-11d2-aa0d-00e098032b8c
PL031RtcDaylight-b336f62d-4135-4a55-ae4e-4971bbf0885d
PL031RtcTimeZone-b336f62d-4135-4a55-ae4e-4971bbf0885d
PlatformLang-8be4df61-93ca-11d2-aa0d-00e098032b8c
PlatformLangCodes-8be4df61-93ca-11d2-aa0d-00e098032b8c
SbConfigState-793d9786-44dc-4709-b57f-85b8e8fdbfd2
Timeout-8be4df61-93ca-11d2-aa0d-00e098032b8c

4/ Increase root partition size: raspi-config --expand-rootfs ; resize2fs /dev/sda2

5/ install the kernel: apt update && apt install linux-image-arm64 grub-efi-arm64

6/ install grub: grub-install ; update-grub2

root@debian:~# grub-install
Installing for arm64-efi platform.
Installation finished. No error reported.
root@debian:~# update-grub2
Generating grub configuration file ...
Found background image: /usr/share/images/desktop-base/desktop-grub.png
Found linux image: /boot/vmlinuz-5.10.0-14-arm64
Found initrd image: /boot/initrd.img-5.10.0-14-arm64
Adding boot menu entry for EFI firmware configuration
done

7/ exit, eject the cdrom and reboot 😀

[Milad_M]

Hi, 

it worked this time 👍. I still don't get why it didn't work the first time!!.

Thank you;) 

by any chance, do you know if it's possible to get home assistant OS running on ESXi-arm?

[cyprienlaplace]

I'm happy it worked!

For HA, I haven't tried myself, but it looks like google had some positive feedback: 1st one is https://www.ivobeerens.nl/2020/10/26/installing-home-assistant-on-esxi-arm/

[k5map]

Thanks for the info on how to configure a larger drive in order to complete the process.  I've successfully built a VM using the 2023-02-23 image and v11.6 netinst iso.

I do have one issue which I hope you can provide some guidance.  Once I have a working VM, what is the process to clone or copy the vmdk so I can use it to create a 2nd VM?

[cyprienlaplace]

Hi, the easiest way to clone a VM is to do it from vCenter. But if you don have one, there are alternatives.

If you have a backup/restore solution (and a few free options work with ESXi Arm fling) you can backup your VM and restore it with a different name.

Or if you only have ssh access, then you can copy most of the files with "cp", but the vmdk. For those you need to use "vmkfstools". see https://kb.vmware.com/s/article/900 or https://serverok.in/clone-vm-on-vmware-esxi-using-vmkfstools for detailed explanations.

[primetechguides]

Bookworm is a bit different I've done a guide here.

https://www.youtube.com/watch?v=qd-dvskk1_c

[k5map]

MUCH thanks for the new set of instructions... When I follow your steps, everything goes great until I boot up the final time to do the initial RPi OS configuration.  In my console, I get a black screen with the cursor flashing in the upper left corner.  Am I using the desktop version whereas I believe you used the lite version.  Any ideas how to get past this so I can complete the OS config?

[cyprienlaplace]

the instructions only work for the Raspberry Pi OS Lite version. The desktop version is now using Wayland, and that is not supported at the moment.

[k5map]

I haven't tried this but if it possible to run raspi-config and set the option to boot to command line somewhere in your instructions, where would you recommend?