VMware Cloud Community
nfaf1
Contributor
Contributor

Hugepages

Hello,

I am running ESXi 6.7 and I have enabled hugapages support as per Huge Pages

but this still doesn't do anything

cat /proc/meminfo | grep -i huge

AnonHugePages:    329728 kB

ShmemHugePages:        0 kB

HugePages_Total:       4

HugePages_Free:        4

HugePages_Rsvd:        0

HugePages_Surp:        0

Hugepagesize:       2048 kB

This should be to 1G

The other parameters seem to go well:

cat /proc/sys/vm/nr_hugepages

4

Can someone help ? as my container doesn't start because of this ....

[  496.326645] hugetlbfs: Unsupported page size 1024 MB

[  497.030697] hugetlbfs: Unsupported page size 1024 MB

[  497.632851] hugetlbfs: Unsupported page size 1024 MB

[  498.237018] hugetlbfs: Unsupported page size 1024 MB

.....

Thank you

Nuno

Reply
0 Kudos
9 Replies
ThompsG
Virtuoso
Virtuoso

Hi nfaf1 and welcome to the community!

Can you download your VMX file for the VM and upload to this thread? Remove anything that might be sensitive before doing so Smiley Happy

Kind regards.

Reply
0 Kudos
nfaf1
Contributor
Contributor

Hello ThompsG

Sure ... please find it below

Thanks for your help Smiley Happy Really appreciate it

.encoding = "UTF-8"

config.version = "8"

virtualHW.version = "14"

vmci0.present = "TRUE"

floppy0.present = "FALSE"

numvcpus = "10"

memSize = "8192"

bios.bootRetry.delay = "10"

powerType.suspend = "soft"

tools.upgrade.policy = "manual"

sched.cpu.units = "mhz"

sched.cpu.affinity = "all"

vm.createDate = "1551546548460994"

scsi0.virtualDev = "lsilogic"

scsi0.present = "TRUE"

usb.present = "TRUE"

ehci.present = "TRUE"

scsi0:0.deviceType = "scsi-hardDisk"

scsi0:0.fileName = "ESXi2.vmdk"

sched.scsi0:0.shares = "normal"

sched.scsi0:0.throughputCap = "off"

scsi0:0.present = "TRUE"

ethernet0.virtualDev = "e1000"

ethernet0.networkName = "VM Network"

ethernet0.addressType = "generated"

ethernet0.wakeOnPcktRcv = "FALSE"

ethernet0.present = "TRUE"

ide0:0.deviceType = "atapi-cdrom"

ide0:0.fileName = "auto detect"

ide0:0.present = "TRUE"

displayName = "VER-01"

guestOS = "otherlinux-64"

toolScripts.afterPowerOn = "TRUE"

toolScripts.afterResume = "TRUE"

toolScripts.beforeSuspend = "TRUE"

toolScripts.beforePowerOff = "TRUE"

tools.syncTime = "FALSE"

uuid.bios = "56 4d bf f9 a7 bd 82 95-68 4c 2f 0c ac 6d 5f 32"

uuid.location = "56 4d bf f9 a7 bd 82 95-68 4c 2f 0c ac 6d 5f 32"

vc.uuid = "52 70 5e 3f 76 63 08 e9-af 83 1a f4 c5 c8 08 37"

sched.cpu.min = "0"

sched.cpu.shares = "normal"

sched.mem.min = "8192"

sched.mem.minSize = "8192"

sched.mem.shares = "normal"

vhv.enable = "TRUE"

vpmc.enable = "TRUE"

ethernet0.generatedAddress = "00:0c:29:00:00:01"

vmci0.id = "-1402118350"

cleanShutdown = "FALSE"

ide0:0.startConnected = "FALSE"

ide0:0.autodetect = "TRUE"

tools.guest.desktop.autolock = "FALSE"

nvram = "ESXi2.nvram"

pciBridge0.present = "TRUE"

svga.present = "TRUE"

pciBridge4.present = "TRUE"

pciBridge4.virtualDev = "pcieRootPort"

pciBridge4.functions = "8"

pciBridge5.present = "TRUE"

pciBridge5.virtualDev = "pcieRootPort"

pciBridge5.functions = "8"

pciBridge6.present = "TRUE"

pciBridge6.virtualDev = "pcieRootPort"

pciBridge6.functions = "8"

pciBridge7.present = "TRUE"

pciBridge7.virtualDev = "pcieRootPort"

pciBridge7.functions = "8"

hpet0.present = "TRUE"

RemoteDisplay.maxConnections = "-1"

sched.cpu.latencySensitivity = "normal"

svga.autodetect = "TRUE"

numa.autosize.cookie = "100001"

numa.autosize.vcpu.maxPerVirtualNode = "10"

sched.swap.derivedName = "/vmfs/volumes/5c7ab815-9b21a8d0-821a-7446a0924210/ESXi2/ESXi2-67669089.vswp"

pciBridge0.pciSlotNumber = "17"

pciBridge4.pciSlotNumber = "21"

pciBridge5.pciSlotNumber = "22"

pciBridge6.pciSlotNumber = "23"

pciBridge7.pciSlotNumber = "24"

scsi0.pciSlotNumber = "16"

usb.pciSlotNumber = "32"

ethernet0.pciSlotNumber = "33"

ehci.pciSlotNumber = "34"

vmci0.pciSlotNumber = "35"

ethernet0.generatedAddressOffset = "0"

monitor.phys_bits_used = "43"

vmotion.checkpointFBSize = "16777216"

vmotion.checkpointSVGAPrimarySize = "16777216"

softPowerOff = "FALSE"

usb:1.speed = "2"

usb:1.present = "TRUE"

usb:1.deviceType = "hub"

usb:1.port = "1"

usb:1.parent = "-1"

sched.mem.lpage.enable1GPage = "TRUE"

migrate.hostLog = "./ESXi2-67669089.hlog"

scsi0:0.redo = ""

usb:0.present = "TRUE"

usb:0.deviceType = "hid"

usb:0.port = "0"

usb:0.parent = "-1"

Reply
0 Kudos
nfaf1
Contributor
Contributor

Hello ThompsG

I have created a new VM with some different Settings but still not working

The vmx file below:

.encoding = "UTF-8"

config.version = "8"

virtualHW.version = "14"

vmci0.present = "TRUE"

floppy0.present = "FALSE"

numvcpus = "10"

memSize = "8192"

bios.bootRetry.delay = "10"

powerType.suspend = "soft"

tools.upgrade.policy = "manual"

sched.cpu.units = "mhz"

sched.cpu.affinity = "all"

vm.createDate = "1552062831551412"

scsi0.virtualDev = "pvscsi"

scsi0.present = "TRUE"

sata0.present = "TRUE"

usb.present = "TRUE"

ehci.present = "TRUE"

scsi0:0.deviceType = "scsi-hardDisk"

scsi0:0.fileName = "VER-CE-09.vmdk"

sched.scsi0:0.shares = "normal"

sched.scsi0:0.throughputCap = "off"

scsi0:0.present = "TRUE"

ethernet0.virtualDev = "vmxnet3"

ethernet0.networkName = "VM Network"

ethernet0.addressType = "generated"

ethernet0.wakeOnPcktRcv = "FALSE"

ethernet0.uptCompatibility = "TRUE"

ethernet0.present = "TRUE"

sata0:0.deviceType = "cdrom-image"

sata0:0.fileName = ""

sata0:0.present = "TRUE"

displayName = "VER-CE-09"

guestOS = "oraclelinux8-64"

toolScripts.afterPowerOn = "TRUE"

toolScripts.afterResume = "TRUE"

toolScripts.beforeSuspend = "TRUE"

toolScripts.beforePowerOff = "TRUE"

tools.syncTime = "FALSE"

uuid.bios = "56 4d fa 1c bb 6a 5b 35-6e 34 2a 67 f5 2d 4e c3"

uuid.location = "56 4d fa 1c bb 6a 5b 35-6e 34 2a 67 f5 2d 4e c3"

vc.uuid = "52 0b d2 7b f6 5d 2b 24-a4 9d 07 4f ca cc 1a eb"

sched.cpu.min = "0"

sched.cpu.shares = "normal"

sched.mem.min = "8192"

sched.mem.minSize = "8192"

sched.mem.shares = "normal"

ethernet0.generatedAddress = "00:0c:29:2d:4e:c3"

vmci0.id = "-181580093"

cleanShutdown = "TRUE"

vpmc.enable = "TRUE"

vhv.enable = "TRUE"

vvtd.enable = "TRUE"

firmware = "efi"

cpuid.coresPerSocket = "5"

tools.guest.desktop.autolock = "FALSE"

nvram = "VER-CE-09.nvram"

pciBridge0.present = "TRUE"

svga.present = "TRUE"

pciBridge4.present = "TRUE"

pciBridge4.virtualDev = "pcieRootPort"

pciBridge4.functions = "8"

pciBridge5.present = "TRUE"

pciBridge5.virtualDev = "pcieRootPort"

pciBridge5.functions = "8"

pciBridge6.present = "TRUE"

pciBridge6.virtualDev = "pcieRootPort"

pciBridge6.functions = "8"

pciBridge7.present = "TRUE"

pciBridge7.virtualDev = "pcieRootPort"

pciBridge7.functions = "8"

hpet0.present = "TRUE"

RemoteDisplay.maxConnections = "-1"

sched.cpu.latencySensitivity = "normal"

svga.autodetect = "TRUE"

numa.autosize.cookie = "100001"

numa.autosize.vcpu.maxPerVirtualNode = "10"

sched.swap.derivedName = "/vmfs/volumes/5c7ab815-9b21a8d0-821a-7446a0924210/VER-CE-09/VER-CE-09-5a0ed416.vswp"

pciBridge0.pciSlotNumber = "17"

pciBridge4.pciSlotNumber = "21"

pciBridge5.pciSlotNumber = "22"

pciBridge6.pciSlotNumber = "23"

pciBridge7.pciSlotNumber = "24"

scsi0.pciSlotNumber = "160"

usb.pciSlotNumber = "32"

ethernet0.pciSlotNumber = "192"

ehci.pciSlotNumber = "33"

vmci0.pciSlotNumber = "34"

sata0.pciSlotNumber = "35"

scsi0.sasWWID = "50 05 05 6c bb 6a 5b 30"

ethernet0.generatedAddressOffset = "0"

monitor.phys_bits_used = "43"

vmotion.checkpointFBSize = "16777216"

vmotion.checkpointSVGAPrimarySize = "16777216"

softPowerOff = "TRUE"

usb:1.speed = "2"

usb:1.present = "TRUE"

usb:1.deviceType = "hub"

usb:1.port = "1"

usb:1.parent = "-1"

tools.remindInstall = "TRUE"

migrate.hostLog = "./VER-CE-09-5a0ed416.hlog"

sched.mem.lpage.enable1GPage = "TRUE"

scsi0:0.redo = ""

usb:0.present = "TRUE"

usb:0.deviceType = "hid"

usb:0.port = "0"

usb:0.parent = "-1"

HELP !!!! Smiley Happy

Thank you

Reply
0 Kudos
continuum
Immortal
Immortal

Is this a nested ESXi VM ?
Try to set a more appropriate guestOS value like "vmkernel" or "vmkernel6"


________________________________________________
Do you need support with a VMFS recovery problem ? - send a message via skype "sanbarrow"
I do not support Workstation 16 at this time ...

Reply
0 Kudos
nfaf1
Contributor
Contributor

Hi Continuum

It is NOT an Nested ESXi VM ... Although the first name misleads to think it could be

This is a VM that runs Kubernetes (and docker) and DPDK and we are making use of Hugepages.

The results are the same with both VM Configurations (both VM configurations posted)

Please let me know

Thanks

Reply
0 Kudos
nfaf1
Contributor
Contributor

I tried to do what you suggested anyway - As I don't have any clue anymore on what to try - But it didn't work

Exact same problem

Please let me know if you or someone can help ?

Thanks

Nuno

Reply
0 Kudos
nfaf1
Contributor
Contributor

I wanted to add a few notes

We have this working in Azure with their hypervisor:

master-0 ~ # cat /proc/meminfo | grep Huge

AnonHugePages:   1304576 kB

ShmemHugePages:        0 kB

HugePages_Total:       4

HugePages_Free:        2

HugePages_Rsvd:        0

HugePages_Surp:        0

Hugepagesize:    1048576 kB

master-0 ~ # cat /proc/cpuinfo | grep pdpe1gb

flags        : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm rdseed adx smap xsaveopt

This is not available on ESXi ...

Thank you all for any help you can give me

Nuno

Reply
0 Kudos
scottRosenberg
Enthusiast
Enthusiast

Did you ever end up figuring out a solution for this?

Reply
0 Kudos
kerkyra
Contributor
Contributor

Hi,

as i am facing the same issue, it seems it is related to the hardware/cpu family you are using. so the esXi VM of yours is on top of the hardware but the cpu does not support 1gb huge pages, so it cannot allocate them. if however, the cpu supports huge pages, then check whether you change the eVC from your vCenter on the host to a cpu family that supports hugepages, i am using Sandy Bridge and i know it does not support it.

Regards,

Stamatis

Reply
0 Kudos