Headless:: ESXi 7.0 Upgrade does not stay after reboot:: 6.7 Returns

[DennisTheMenac1]

Dear Forum,

First, we upgrade our headless (no video card possible) box as described below,

esxcli software profile update -p ESXi-7.0.0-16324942-standard -d /vmfs/volumes/gg0_ssd/images/VMware-ESXi-7.0.0-16324942-depot.zip

Update Result

   Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.

...

reboot

Everything is Great and we get have 7.0.0 running successfully! 🙂

However, after another reboot, Version 6.7 returns....

What is going on? Can we resolve this with our headless machine,

Thanks so kindly,

Dennis The Menace.

[sudeshnas]

Hi DennisTheMenace8,

Kindly provide the following output from the respective esxi host:

#esxcli storage core device list

Regards,

Sudeshna Sarkar

Install-Upgrade Specialist

[DennisTheMenac1]

Hi Sudeshna,

Thanks for your answer. However, we could not wait and decided to do a new fresh 7.0 install. That seemed to work except for one booting error:

cr0=0x8001003d cr2=0x0 cr3=0x7f33b000 cr4=0x10012c

FMS=06/5f/1 uCode=0x2e

*PCPU0:262144/bootstrap

PCPU  0: SIII

Code start: 0x420034400000 VMK uptime: 0:00:00:05.325

0x450184398ca0:[0x4200344fb88a]PanicvPanicInt@vmkernel#nover+0x2c6 stack: 0x4200344fb88a

~0x450184398d50:[0x4200344fbd68]Panic_NoSave@vmkernel#nover+0x4d stack: 0x450184398db0

~0x450184398db0:[0x4200344e01af]IOMMU_ValidateRMRR@vmkernel#nover+0xd4 stack: 0xff000

~0x450184398df0:[0x4200348e09f5]VTDInitDrhdAndRmrrList@vmkernel#nover+0x1a6 stack: 0x70348e5c1a

~0x450184398e60:[0x4200348e0196]VTDInit@vmkernel#nover+0x203 stack: 0x1112e

0x450184398eb0:[0x42003446df5d]IOV_Init@vmkernel#nover+0x8a stack: 0x420034a3c530

~0x450184398ec0:[0x42003452392f]SysInitTable_Start@vmkernel#nover+0x7c stack: 0x400

~0x450184398f10:[0x4200344dbccf]InitVMKernel@vmkernel#nover+0x1c stack: 0xffff8100000000ca

~0x450184398f20:[0x4200345cb329]LegacyBootVMKernel@vmkernel#nover+0x486 stack: 0x4501843a21b0

~0x450184398fc0:[0x4200344144f8]_start@vmkernel#nover+0x195 stack: 0x4adf40

base fs=0x0 gs=0x420040000000 Kgs=0x0

0:00:00:05.303 cpu0:262144)IOMMU: 1690: RMRR at [0x3e2e0000:0x3e2ff000] overlaps a range marked in the firmware memory map as usable by the operating system.

No place on disk to dump data.

No file configured to dump data.

No port for remote debugger.

By disabling VT-d in the BIOS, this error is resolved. However, we do need VT-d. Is there anything that can be done or should I post to another Forum Section?

Thanks

Dennis The Menace

[Lalegre]

Hey,

Could you please provide the Hardware vendor, model and CPU where you are trying to install ESXi?

Regards!

[DennisTheMenac1]

ESXi 7.0

[root@gg0:~] smbiosDump

Dumping live SMBIOS data!

  BIOS Info (Type 0): #0

    Size: 0x00018

    Vendor: "American Megatrends Inc."

    Version: "5.13"

    Date: "08/28/2019"

    Start Address: 0xf0000

    ROM Size: 8192 kB

    Features: 0x0d0300000001378b9880

      PCI supported

      BIOS flashable

      BIOS shadowing allowed

      CD boot supported

      Selectable boot supported

      BIOS ROM socketed

      EDD spec supported

      1.2MB Floppy supported

      720kB Floppy supported

      2.88MB Floppy supported

      Print Screen supported

      Serial Services supported

      Printer Services supported

      ACPI supported

      USB Legacy supported

      BIOS Boot Spec supported

      Enable targeted content distribution

      UEFI supported

    System BIOS release: 5.13

  System Info (Type 1): #1

    Manufacturer: "Default string"

    Product: "Default string"

    Version: "Default string"

    Serial: "Default string"

    SKU: "Default string"

    Family: "Default string"

    UUID: 09000800070006000500040003000200

    Wake-up: 0x06 (Power Switch)

  Baseboard Info (Type 2): #2

    Manufacturer: "Default string"

    Product: "Default string"

    Version: "Default string"

    Serial: "Default string"

    Asset Tag: "Default string"

    Type: 0x0a (Motherboard)

    Features: 0x09

      Hosting Board

      Replaceable

    Location: "Default string"

    Chassis: #3

  Chassis Info (Type 3): #3

    Manufacturer: "Default string"

    Version: "Default string"

    Serial: "Default string"

    Asset Tag: "Default string"

    Type: 0x03 (Desktop)

    Bootup State: 0x03 (Safe)

    Power Supply State: 0x03 (Safe)

    Thermal State: 0x03 (Safe)

    Security Status: 0x03 (None)

    Power Cords: 1

    SKU: "Default string"

  Port Connector (Type 8): #4

    Type: 0x0e (Mouse Port)

    Internal Designator: "J1A1"

    External Designator: "PS2Mouse"

    External Connector: 0x0f (PS/2)

  Port Connector (Type 8): #5

    Type: 0x0d (Keyboard Port)

    Internal Designator: "J1A1"

    External Designator: "Keyboard"

    External Connector: 0x0f (PS/2)

  Port Connector (Type 8): #6

    Type: 0xff (Other)

    Internal Designator: "J2A1"

    External Designator: "TV Out"

    External Connector: 0x1d (Mini-Centronics Type-14)

  Port Connector (Type 8): #7

    Type: 0x09 (Serial Port 16550A Compatible)

    Internal Designator: "J2A2A"

    External Designator: "COM A"

    External Connector: 0x08 (DB-9 pin male)

  Port Connector (Type 8): #8

    Type: 0x1c (Video Port)

    Internal Designator: "J2A2B"

    External Designator: "Video"

    External Connector: 0x07 (DB-15 pin female)

  Port Connector (Type 8): #9

    Type: 0x10 (USB)

    Internal Designator: "J3A1"

    External Designator: "USB1"

    External Connector: 0x12 (Access Bus [USB])

  Port Connector (Type 8): #10

    Type: 0x10 (USB)

    Internal Designator: "J3A1"

    External Designator: "USB2"

    External Connector: 0x12 (Access Bus [USB])

  Port Connector (Type 8): #11

    Type: 0x10 (USB)

    Internal Designator: "J3A1"

    External Designator: "USB3"

    External Connector: 0x12 (Access Bus [USB])

  Port Connector (Type 8): #12

    Type: 0xff (Other)

    Internal Designator: "J9A1 - TPM HDR"

    Internal Connector: 0xff (Other)

  Port Connector (Type 8): #13

    Type: 0xff (Other)

    Internal Designator: "J9C1 - PCIE DOCKING CONN"

    Internal Connector: 0xff (Other)

  Port Connector (Type 8): #14

    Type: 0xff (Other)

    Internal Designator: "J2B3 - CPU FAN"

    Internal Connector: 0xff (Other)

  Port Connector (Type 8): #15

    Type: 0xff (Other)

    Internal Designator: "J6C2 - EXT HDMI"

    Internal Connector: 0xff (Other)

  Port Connector (Type 8): #16

    Type: 0xff (Other)

    Internal Designator: "J3C1 - GMCH FAN"

    Internal Connector: 0xff (Other)

  Port Connector (Type 8): #17

    Type: 0xff (Other)

    Internal Designator: "J1D1 - ITP"

    Internal Connector: 0xff (Other)

  Port Connector (Type 8): #18

    Type: 0xff (Other)

    Internal Designator: "J9E2 - MDC INTPSR"

    Internal Connector: 0xff (Other)

  Port Connector (Type 8): #19

    Type: 0xff (Other)

    Internal Designator: "J9E4 - MDC INTPSR"

    Internal Connector: 0xff (Other)

  Port Connector (Type 8): #20

    Type: 0xff (Other)

    Internal Designator: "J9E3 - LPC HOT DOCKING"

    Internal Connector: 0xff (Other)

  Port Connector (Type 8): #21

    Type: 0xff (Other)

    Internal Designator: "J9E1 - SCAN MATRIX"

    Internal Connector: 0xff (Other)

  Port Connector (Type 8): #22

    Type: 0xff (Other)

    Internal Designator: "J9G1 - LPC SIDE BAND"

    Internal Connector: 0xff (Other)

  Port Connector (Type 8): #23

    Type: 0xff (Other)

    Internal Designator: "J8F1 - UNIFIED"

    Internal Connector: 0xff (Other)

  Port Connector (Type 8): #24

    Type: 0xff (Other)

    Internal Designator: "J6F1 - LVDS"

    Internal Connector: 0xff (Other)

  Port Connector (Type 8): #25

    Type: 0xff (Other)

    Internal Designator: "J2F1 - LAI FAN"

    Internal Connector: 0xff (Other)

  Port Connector (Type 8): #26

    Type: 0xff (Other)

    Internal Designator: "J2G1 - GFX VID"

    Internal Connector: 0xff (Other)

  Port Connector (Type 8): #27

    Type: 0xff (Other)

    Internal Designator: "J1G6 - AC JACK"

    Internal Connector: 0xff (Other)

  System Slot (Type 9): #28

    Designation: "J6B2"

    Type: 0xa5 (PCI Express)

    Bus Width: 0x0d (x16)

    Status: 0x04 (In Use)

    Length: 0x04 (Long)

    Slot ID: 0

    Characteristics: 0x010c (3.3 V, Shared, PME#)

    PCI Address: 0000:00:01.0

  System Slot (Type 9): #29

    Designation: "J6B1"

    Type: 0xa5 (PCI Express)

    Bus Width: 0x08 (x1)

    Status: 0x04 (In Use)

    Length: 0x03 (Short)

    Slot ID: 1

    Characteristics: 0x010c (3.3 V, Shared, PME#)

    PCI Address: 0000:00:1c.3

  System Slot (Type 9): #30

    Designation: "J6D1"

    Type: 0xa5 (PCI Express)

    Bus Width: 0x08 (x1)

    Status: 0x04 (In Use)

    Length: 0x03 (Short)

    Slot ID: 2

    Characteristics: 0x010c (3.3 V, Shared, PME#)

    PCI Address: 0000:00:1c.4

  System Slot (Type 9): #31

    Designation: "J7B1"

    Type: 0xa5 (PCI Express)

    Bus Width: 0x08 (x1)

    Status: 0x04 (In Use)

    Length: 0x03 (Short)

    Slot ID: 3

    Characteristics: 0x010c (3.3 V, Shared, PME#)

    PCI Address: 0000:00:1c.5

  System Slot (Type 9): #32

    Designation: "J8B4"

    Type: 0xa5 (PCI Express)

    Bus Width: 0x08 (x1)

    Status: 0x04 (In Use)

    Length: 0x03 (Short)

    Slot ID: 4

    Characteristics: 0x010c (3.3 V, Shared, PME#)

    PCI Address: 0000:00:1c.6

  Onboard Devices (Type 10): #33

    Video: "To Be Filled By O.E.M."

  OEM Strings (Type 11): #34

    Default string

  System Config Options (Type 12) #35:

    Default string

  Type 32 Record: #36

    Data 00: 20 14 24 00 00 00 00 00 00 00 00 00 00 00 00 00

    Data 10: 00 00 00 00

  Onboard Device (Type 41): #37

    Designation: "Onboard IGD"

    Status: Enabled

    Type: 0x03 (Video)

    Instance: 0x01

    PCI Address: 0000:00:02.0

  Onboard Device (Type 41): #38

    Designation: "Onboard LAN"

    Status: Enabled

    Type: 0x05 (Ethernet)

    Instance: 0x01

    PCI Address: 0000:00:19.0

  Onboard Device (Type 41): #39

    Designation: "Onboard 1394"

    Status: Enabled

    Type: 0x01 (Other)

    Instance: 0x01

    PCI Address: 0000:03:1c.2

  Type 43 Record: #40

    Data 00: 2b 1f 28 00 00 43 54 4e 02 00 03 00 01 00 01 00

    Data 10: 00 00 02 10 00 00 00 00 00 00 00 00 00 00 00

    String 1: "TPM 2.0"

    String 2: "NUVOTON"

  Physical Memory Array (Type 16): #41

    Use: 0x03 (System memory)

    Location: 0x03 (Motherboard)

    Slots: 4

    Max. Size: 128 GB

    ECC: 0x06 (Multi-bit)

  Memory Array Mapping (Type 19): #42

    Memory Array: #41

    Partition Width: 1

    Start Address: 0x0000000000000000

    End Address: 0x0000000400000000

  Memory Device (Type 17): #43

    Location: "DIMM0"

    Bank: "BANK 0"

    Manufacturer: "Toshiba"

    Serial: "2762EE38"

    Asset Tag: "BANK 0 DIMM0 AssetTag"

    Part Number: "9965657-011.A00G"

    Memory Array: #41

    Form Factor: 0x09 (DIMM)

    Type: 0x1a (DDR4)

    Type Detail: 0x4080 (Synchronous, Unregistered)

    Data Width: 8 bits

    Size: 16 GB

    Max. Speed: 2400 MT/s

    Rank: 2

    Configured Speed: 2133 MT/s

    Min. Voltage: 1200 mV

    Max. Voltage: 1200 mV

    Configured Voltage: 1200 mV

  Memory Device Mapping (Type 20): #44

    Memory Device: #43

    Array Mapping: #42

    Interleave Pos: 0

    Interleaved Depth: 0

    Start Address: 0x0000000000000000

    End Address: 0x0000000400000000

  Memory Device (Type 17): #45

    Location: "DIMM0"

    Bank: "BANK 1"

    Manufacturer: "NO DIMM"

    Serial: "NO DIMM"

    Asset Tag: "NO DIMM"

    Part Number: "NO DIMM"

    Memory Array: #41

    Form Factor: 0x09 (DIMM)

    Type: 0x02 (Unknown)

    Type Detail: 0x0004 (Unknown)

    Data Width: 0 bits

    Size: No Memory Installed

  Memory Device (Type 17): #46

    Location: "DIMM1"

    Bank: "BANK 0"

    Manufacturer: "NO DIMM"

    Serial: "NO DIMM"

    Asset Tag: "NO DIMM"

    Part Number: "NO DIMM"

    Memory Array: #41

    Form Factor: 0x09 (DIMM)

    Type: 0x02 (Unknown)

    Type Detail: 0x0004 (Unknown)

    Data Width: 0 bits

    Size: No Memory Installed

  Memory Device (Type 17): #47

    Location: "DIMM1"

    Bank: "BANK 1"

    Manufacturer: "NO DIMM"

    Serial: "NO DIMM"

    Asset Tag: "NO DIMM"

    Part Number: "NO DIMM"

    Memory Array: #41

    Form Factor: 0x09 (DIMM)

    Type: 0x02 (Unknown)

    Type Detail: 0x0004 (Unknown)

    Data Width: 0 bits

    Size: No Memory Installed

  Cache Info (Type 7): #48

    Designation: "L1-Cache"

    Level: L1

    State: Enabled

    Mode: 0x01 (Write Back)

    Location: 0x00 (Internal, Not Socketed)

    ECC: 0x05 (Single-bit)

    Type: 0x03 (Instruction)

    Associativity: 0x07 (8-way Set-Associative)

    Max. Size: 56 kB

    Current Size: 56 kB

    Supported SRAM Types: 0x0020 (Synchronous)

    Current SRAM Type: 0x0020 (Synchronous)

  Cache Info (Type 7): #49

    Designation: "L2-Cache"

    Level: L2

    State: Enabled

    Mode: 0x01 (Write Back)

    Location: 0x00 (Internal, Not Socketed)

    ECC: 0x05 (Single-bit)

    Type: 0x05 (Unified)

    Associativity: 0x08 (16-way Set-Associative)

    Max. Size: 2048 kB

    Current Size: 2048 kB

    Supported SRAM Types: 0x0020 (Synchronous)

    Current SRAM Type: 0x0020 (Synchronous)

  Processor Info (Type 4): #50

    Payload length: 0x30

    Socket: "CPU0"

    Socket Type: 0x15 (Socket LGA775)

    Socket Status: Populated

    Type: 0x03 (CPU)

    Family: 0xb2 (Pentium 4)

    Manufacturer: "Intel(R) Corporation"

    Version: "Intel(R) Atom(TM) CPU C3558 @ 2.20GHz"

    Asset Tag: "UNKNOWN"

    Processor ID: 0xbfebfbff000506f1

    Status: 0x01 (Enabled)

    Voltage: 1.6 V

    External Clock: 100 MHz

    Max. Speed: 3800 MHz

    Current Speed: 2200 MHz

    L1 Cache: #48

    L2 Cache: #49

    Core Count: 4

    Core Enabled Count: 4

    Thread Count: 4

    Characteristics: 0x00fc (64-bit Capable, Multi-Core, Hardware Thread, Execute Protection, Enhanced Virtualization, Power/Performance Control)

[Lalegre]

I could not get from there your Hardware model. Basically I am asking this because your hardware must be on the HCL to be supported and vSphere 7 is a quite new version.

Have you checked that?

[DennisTheMenac1]

Hi There,

The hardware is about a year old or so. It is described at:

http://www.lannerinc.com/products/network-appliances/x86-desktop-network-appliances/nca-1510

[Lalegre]

Hey,

Is this some sort of Security Appliance? Because I do not think this even have all the CPU instructions needed for installing the ESXi. Also this Atom processor is supported in not many servers and this is for sure not of them.

I think that if you solve how to do it it will be best effort.

[scott28tt]

Moderator: Please use the Attach function in the bottom-right of the post creator/editor to post long text output or logs, rather than doing a copy/paste of raw text (such as your smbiosDump output)

[DennisTheMenac1]

Hi Lalegre,

Yes it is a security appliance. Both ESXi 6.7U3 and 7.0 run very well on it. It ran for more than a year on several production systems. The CPU is an ATOM C3558 and, as such, is fully supported. The other components as well, as Lanner purposely chooses very conservative hardware: all standard Intel.  So we don't think it is an HCL topic.

The installation, however, is a real pain, and took like 40-50 hours of trail & error research to get completed. But once when it runs, it runs very well.

We just would like to have the VT-d Support running for extra flexibility.

Enabling this causes above  shown crash at boot-time. We suspect the BIOS and hope VMware could provide a solution.

Thanks,

Dennis

[barnabyjones]

Hi There, 

I came across this post while trying to install ESXi on a Lanner 1515. Would you be able to share what you had to do to get ESXi working on a Lanner NCA? I'd appreciate any advice!

Cheers