VMware Cloud Community
DCasota
Expert
Expert
Jump to solution

Fresh created Photon VM with attached ISO stucks in VM BIOS

Hi,

I wanted to deploy some Photon 3 and 4 VMs on ESXi buildnr 17839012, RPi4B, local USB3 datastore with Photon isos, eg. photon-3.0-1ca1d7be-aarch64.iso. ESXi runs on apipa network with connectivity to a client, on apipa too.

The configuration of the VM with attached iso from local datastore successfully finished. On VM boot however right after expected Start PXE over IPv4 and PXE over IPv6 network fail, it enters Bios but vmx setting isn't configured for forced entry into Bios setup screen because of standard settings. In addition, VM Console shows Bios screen with 0.00 GHz and keyboard does not work in Bios (?). Any ideas how to configure the VM "offline"? vmware.log + dmesg extract is attached.

Daniel

Reply
0 Kudos
1 Solution

Accepted Solutions
DCasota
Expert
Expert
Jump to solution

The expected behavior in virtual hardware fallback combinations sata/scsi/ide + network + bios/efi first led me to the conclusion of issues, mostly because of error messages. 

The real major issue however was usb flash drive related. Actually there is no newer revision for my specific usb device id: vid = 090C pid = 1000.

Imho esxi on arm limitations eg. usb 2.0 performance (20.9MB/s) on usb3 ports does not hurt in tech preview, but robustness is a must.
Options like disabling usb passthrough for a specific device are helpful.

I have to solve the thermal aspects of the poweredon Pi now as the usb stick gets very hot (> 60 degrees) only after a minute. 

View solution in original post

Reply
0 Kudos
7 Replies
DCasota
Expert
Expert
Jump to solution

In webclient UI starting with default settings (Create new vm > Name {enter a name} > Guest OS family Linux > Guest OS version VMware Photon OS 74bit > Next > Next> CD/DVD Drive 1 : Configure Datastore iso file > Finish) USUALLY works. Nevertheless here the findings so far.

#A I didn't find out how to reproduce the issue described which happened on a default settings vm setup.
The similar effect of 'auto-entering Bios + no keyboard' can actively be forced with the following modification on a with default settings created vm:
1) After a stored default configuration, in vm settings delete the following: Hard disk 1 (save&reopen settings), CD/DVD Drive 1 (save&reopen settings), USB Controller 1, SCSI Controller 0, SATA Controller 0
2) Start vm.
Using IDE-controller-only is known as not supported.

#B The attached files are from a newly created vm with default settings. On start it auto-entered Bios, and there keyboard worked.
This constellation happened so far more than once.

#C Once, in vmx settings the entry on scsi0.pciSlotNumber was "-1" and I've changed it to "160" and boot from hard disk started successfully again.

Hope this helps. Daniel

 

 

Reply
0 Kudos
DCasota
Expert
Expert
Jump to solution

fyi - Photon 4 (minimal installation, esxi-optimized) works, Photon 3 (minimal installation, esxi-optimized) stucks with "error: not a regular file." like here.

Tags (1)
Reply
0 Kudos
Rooker1
Contributor
Contributor
Jump to solution

Starting a fresh OSTree repo · Creating summary metadata  After you have downloaded the latest Photon OS ISO image into a folder of your choice,  Important : Before you finish creating the Photon OS Virtual Machine, we strongly  settings, select Options, choose Advanced, and select Boot with EFI instead of BIOS.

DGCustomerFirst.com

Reply
0 Kudos
DCasota
Expert
Expert
Jump to solution

@Rooker1 my intention was/is to help identify workflow issues. Some issues are easily reproducible, some not. The description above describes an issue of 2nd sort which doesn't occur causally after strictly have executed a reproduction recipe. I've tried to stripe it down to causal reproducible recipes with similar effect and well-known 'not supported'-scenarios.

Thanks to your input, here some further findings:

Photon 3 rev 2 full iso arm64, EFI or BIOS, Default RPM-OSTree Server: boot ok

Photon 3 rev 2 full iso arm64, EFI or BIOS, minimal | developer | edge: esxi-optimized -> 'error: not a regular file'

Photon 3 rev 2 full iso arm64, EFI or BIOS, minimal | developer | edge : generic -> boot ok

Photon 4 GA full iso arm64, BIOS, minimal | developer | Default RPM-OSTree Server : -> boot ok, but no boot output 'till login, and boot options afterwards has autochanged firmware to EFI.

Reply
0 Kudos
cyprienlaplace
VMware Employee
VMware Employee
Jump to solution

Hi Daniel,

in the vmware.log I see this:

2021-05-25T14:04:31.135Z| vmx| I125: OBJLIB-FILEBE : FileBEOpen: can't open 'Ph3Arm64Test0001.nvram' : Could not find the file (393218).
2021-05-25T14:04:31.135Z| vmx| I125: OBJLIB-FILEBE : FileBEOpen: can't open 'Ph3Arm64Test0001.nvram' : Could not find the file (393218).
2021-05-25T14:04:31.136Z| vmx| W115: NVRAMMGR: No valid NVRAM file found, will create default NVRAM.

Do you know why the nvram file is missing? Unfortunately it is required for UEFI to find the Linux distribution boot loader (and without, it will try the next boot option.. until going into the UEFI screen).

Reply
0 Kudos
DCasota
Expert
Expert
Jump to solution

Hi @cyprienlaplace

No, not. I didn't investigate too much time about the 'why'. The keyboard freeze might have had a correlation to the phase correlating to the "VUsb powered on, but no USB controllers" vmware.log entry, too. I've attached the lsusb and esxcli output.


Edited (13.06.2021): The same effect of 'timeout + auto-entering Bios + no keyboard' is reproducible, indeed. Here's a recipe: Create / Register VM > Create a new virtual machine > Name: Ph3Arm64T0010 > Compatibility (default ESXi 7.0), Guest OS family Linux, Guest OS version VMware Photon OS 64-bit > storage: 'local' VMFS6 (usb3 Pi port) > Memory Reservation checkboxed 'reserve all guest memory (all locked)'; Hard disk 1: Controller location: SCSI controller0, SCSI(0:0) ; CD/DVD Drive 1: Controller location IDE controller 0 master + checkboxed connect datastore ISO file from 'local' datastore photon-3.0-01ca1d7be-aarch64.iso with checkboxed connect at power on; remove SATA Controller 0 ; remove USB Controller 1 > Next > Finish.
As said, it does not generate the same vmware.log entries.

Other issues on this tech preview esxi on Pi4b with Pi Bios secureboot=disabled mostly were already mentioned.
#1 A photonos3 vm vmx entry sata0:1.startConnected = "FALSE" leads to auto-entering Bios after pxe timeout.
#2 vm bios autochanges to efi, but has explicitly been configured as mbr.
#3 Boot after having configured ide(master/slave) instead of sata for the drive controller and/or for the cdrom with the attached iso ends with a dmesg coredump.
     2021-05-25T12:19:13.427Z cpu0:135152)MemSched: vm 135152: 5817: extended swap to 20480 pgs
     2021-05-25T12:19:13.579Z cpu3:135152)World: vm 135153: 7347: Starting world vmm0:ph3t1 of type 8
     2021-05-25T12:19:13.580Z cpu3:135152)Sched: vm 135153: 6770: Adding world 'vmm0:ph3t1', group 'host/user', cpu: shares=-3 min=0 minLimit=-1 max=-1, mem: shares=-3 min=0 minLimit=-1 max=-1
     2021-05-25T12:19:13.580Z cpu3:135152)Sched: vm 135153: 6785: renamed group 26987 to vm.135152
     2021-05-25T12:19:13.580Z cpu3:135152)Sched: vm 135153: 6802: group 26987 is located under group 4
     2021-05-25T12:19:13.805Z cpu0:135153)WARNING: VSCSI: 965: Couldn't find handle 0
     2021-05-25T12:19:13.806Z cpu0:135153)WARNING: World: vm 135153: 8824: vmm0:ph3t1:vcpu-0:VERIFY devices/vide/iovmk/videVMK-vmkio.c:1904
     2021-05-25T12:19:13.806Z cpu0:135153)Backtrace for current CPU #0, worldID=135153, fp=0xfffffc407f90
     2021-05-25T12:19:13.806Z cpu0:135153)0x451a0e29bf40:[0x420034961d80]WorldPanicWork@vmkernel#nover+0x94 stack: 0x451a16f21100, 0x1, 0x41fff48bb000, 0x451a0e29bffc, 0xfffffc407d98
     2021-05-25T12:19:13.806Z cpu0:135153)0x451a0e29bf90:[0x420034962500]World_VMMPanic@vmkernel#nover+0x1c stack: 0xfffffcc28000, 0xfffffc407cc0, 0xfffffccf60e8, 0x450980003000, 0xc
     2021-05-25T12:19:13.806Z cpu0:135153)0x451a0e29bfa0:[0x42003495a218]VMMVMKCall_Call@vmkernel#nover+0xcc stack: 0xfffffccf60e8, 0x450980003000, 0xc, 0x451a0e29c000, 0x420034972c7c
     2021-05-25T12:19:13.806Z cpu0:135153)0x451a0e29bfe0:[0x420034972c84]VMKVMM_ArchEnterVMKernel@vmkernel#nover+0xc stack: 0x0, 0x34bea488, 0x0, 0x0, 0x0
     2021-05-25T12:19:15.429Z cpu0:135160)UserDump: 2630: vmx-vcpu-0:ph3t1: Dumping cartel 135152 (from world 135160) to file /vmfs/volumes/60ace1c5-fcb04c45-c92a-dca632c8a3b6/ph3t1/vmx-zdump.000 ...
     2021-05-25T12:19:16.531Z cpu1:135160)UserDump: 2818: vmx-vcpu-0:ph3t1: Userworld(vmx-vcpu-0:ph3t1) coredump complete.
     2021-05-25T12:19:22.692Z cpu3:135151)J6: 2792: 'local': Exiting async journal replay manager world
#4 At least my low-cost but useful usb storage drives used as esxi install media and as datastore do not work work smoothly yet. Typically usb3 Pi ports issue situations are {delete datastore or clear partition table, device does not support unmap aka temporarily plugout the usb drive -> apd situation}.

#5 An already setuped Pi4B, physically removed sd card + reinitialized with same or newer firmware (eg. v1.27) + physically replugged-in, before start from boot the bios settings have to be changed back to 3GB RAM limit = disabled. -> Not an esxi issue.

-Daniel

Reply
0 Kudos
DCasota
Expert
Expert
Jump to solution

The expected behavior in virtual hardware fallback combinations sata/scsi/ide + network + bios/efi first led me to the conclusion of issues, mostly because of error messages. 

The real major issue however was usb flash drive related. Actually there is no newer revision for my specific usb device id: vid = 090C pid = 1000.

Imho esxi on arm limitations eg. usb 2.0 performance (20.9MB/s) on usb3 ports does not hurt in tech preview, but robustness is a must.
Options like disabling usb passthrough for a specific device are helpful.

I have to solve the thermal aspects of the poweredon Pi now as the usb stick gets very hot (> 60 degrees) only after a minute. 

Reply
0 Kudos