VMware Cloud Community
onthefly
Contributor
Contributor

The total physical memory size is less than the memory assigned to the vm, it's weird.

I created a virtual machine and assigned 64GB memory for the VM on the vCenter 6.0 server, installed the suse 11 sp3 64bit OS on it.

I found only 62GB memory allocalted for the VM when i looked at the /proc/meminfo file inside the VM.

MemTotal:       65979232 kB

MemFree:        64453488 kB

Buffers:           30264 kB

Cached:           696196 kB

SwapCached:            0 kB

Active:           316680 kB

Inactive:         436768 kB

Active(anon):      27068 kB

Inactive(anon):       36 kB

>>> 65979232/1024

64432

>>> 65979232/1024

64432

>>> 64432/1024

62

it is quite strange, does anyone know what's reason for this?

13 Replies
linotelera
Hot Shot
Hot Shot

Hi

please could you post the content of vmx file?

Regards

Reply
0 Kudos
onthefly
Contributor
Contributor

hi linotelera,

thanks for your reply, the content of the vmx file are the followings:

.encoding = "UTF-8"

config.version = "8"

virtualHW.version = "11"

nvram = "E1_Main_ICPRunner01.nvram"

pciBridge0.present = "TRUE"

svga.present = "TRUE"

pciBridge4.present = "TRUE"

pciBridge4.virtualDev = "pcieRootPort"

pciBridge4.functions = "8"

pciBridge5.present = "TRUE"

pciBridge5.virtualDev = "pcieRootPort"

pciBridge5.functions = "8"

pciBridge6.present = "TRUE"

pciBridge6.virtualDev = "pcieRootPort"

pciBridge6.functions = "8"

pciBridge7.present = "TRUE"

pciBridge7.virtualDev = "pcieRootPort"

pciBridge7.functions = "8"

vmci0.present = "TRUE"

hpet0.present = "TRUE"

numvcpus = "16"

cpuid.coresPerSocket = "16"

memSize = "65536"

scsi0.virtualDev = "lsilogic"

scsi0.present = "TRUE"

ide1:0.startConnected = "FALSE"

ide1:0.deviceType = "cdrom-image"

ide1:0.fileName = "/vmfs/volumes/56822d39-76a94110-b502-a4dcbe1b95de/SLES-11-SP3-DVD-x86_64-GM-DVD1.iso"

ide1:0.present = "TRUE"

floppy0.startConnected = "FALSE"

floppy0.clientDevice = "TRUE"

floppy0.fileName = "vmware-null-remote-floppy"

ethernet0.virtualDev = "vmxnet3"

ethernet0.networkName = "VM Network"

ethernet0.addressType = "vpx"

ethernet0.generatedAddress = "00:50:56:9d:76:fd"

ethernet0.uptCompatibility = "TRUE"

ethernet0.present = "TRUE"

scsi0:0.deviceType = "scsi-hardDisk"

scsi0:0.fileName = "E1_Main_ICPRunner01-000003.vmdk"

scsi0:0.present = "TRUE"

displayName = "E1_Main_ICPRunner01"

guestOS = "sles11-64"

toolScripts.afterPowerOn = "TRUE"

toolScripts.afterResume = "TRUE"

toolScripts.beforeSuspend = "TRUE"

toolScripts.beforePowerOff = "TRUE"

uuid.bios = "42 1d 69 cf bd c8 21 4f-5b b5 a7 a7 89 59 fd d5"

vc.uuid = "50 1d a4 06 ab e1 c7 86-5e 3c 87 bd 41 cf 0d 3e"

virtualHW.productCompatibility = "hosted"

sched.swap.derivedName = "/vmfs/volumes/568c8c73-a75e20b8-87cb-a4dcbe1b95dc/E1_Main_ICPRunner01/E1_Main_ICPRunner01-13938b86.vswp"

uuid.location = "56 4d 22 9d 43 ff 3c 68-ee e9 10 7c aa 1e 4c 1a"

replay.supported = "FALSE"

replay.filename = ""

migrate.hostlog = "./E1_Main_ICPRunner01-13938b86.hlog"

scsi0:0.redo = ""

pciBridge0.pciSlotNumber = "17"

pciBridge4.pciSlotNumber = "21"

pciBridge5.pciSlotNumber = "22"

pciBridge6.pciSlotNumber = "23"

pciBridge7.pciSlotNumber = "24"

scsi0.pciSlotNumber = "16"

ethernet0.pciSlotNumber = "160"

vmci0.pciSlotNumber = "32"

vmci0.id = "-1990591019"

monitor.phys_bits_used = "42"

vmotion.checkpointFBSize = "4194304"

vmotion.checkpointSVGAPrimarySize = "4194304"

softPowerOff = "FALSE"

tools.remindInstall = "FALSE"

bios.forceSetupOnce = "FALSE"

sched.scsi0:0.throughputCap = "off"

toolsInstallManager.lastInstallError = "0"

toolsInstallManager.updateCounter = "8"

extendedConfigFile = "E1_Main_ICPRunner01.vmxf"

tools.syncTime = "FALSE"

sched.scsi0:0.shares = "normal"

ide1:0.allowGuestConnectionControl = "TRUE"

cleanShutdown = "FALSE"

Reply
0 Kudos
onthefly
Contributor
Contributor

i guess the VMware hypervisor may consumes some memory on the VM, but I am not sure.

Reply
0 Kudos
linotelera
Hot Shot
Hot Shot

If you could make test to this vm: try to poweroff and reduce the number of vCPUs (1/2 from the original)...

Reply
0 Kudos
onthefly
Contributor
Contributor

I will try this next week, but I think the CPU configuration is not related to the memory, and what is your consideration?

Reply
0 Kudos
linotelera
Hot Shot
Hot Shot

I think (I'm not sure!!|!) these overhead could be:

- vCPU and vNUMA relation and HotAdd (in vSphere 5.5 from 8 vCPU and hot add enabled, could cause an increased consumption due to hot add feature overhead)

- 3d graphics options (because virtual memory is shared by all virtual hardware architecture, reading the official guide vSphere 6.0 Documentation Center , seems that this feature could cause a memory overhead)

But the first feature is disabled by default (you should enable manually) and the second seems to be too high (2GB!!!)

Reply
0 Kudos
dariusd
VMware Employee
VMware Employee

Does the VM's dmesg or kernel.log contain any complaints about "CPU MTRRs don't cover all of memory"?  If so, you can probably work around the issue by adding the disable_mtrr_trim option to the guest's Linux kernel command-line (as a one-off at your bootloader's prompt, or permanently via your grub/lilo/yaboot configuration).


Cheers,

--

Darius

TomHowarth
Leadership
Leadership

could it be as simple as the difference between decimal and octal notation?

Tom Howarth VCP / VCAP / vExpert
VMware Communities User Moderator
Blog: http://www.planetvm.net
Contributing author on VMware vSphere and Virtual Infrastructure Security: Securing ESX and the Virtual Environment
Contributing author on VCP VMware Certified Professional on VSphere 4 Study Guide: Exam VCP-410
Reply
0 Kudos
onthefly
Contributor
Contributor

today I adjusted the vCPU configuration to 8vCPU and checked the memory again, the result was not changed, still showed 62GB

Reply
0 Kudos
onthefly
Contributor
Contributor

hi dariusd,

The dmesg did't contain the complaints you mentioned, do you have some other ideas?

I found some messages about the memory in the output of the dmesg.

[    0.000000] initial memory mapped : 0 - 20000000

[    0.000000] Base memory trampoline at [ffff88000009a000] 9a000 size 20480

[    0.000000] init_memory_mapping: 0000000000000000-00000000c0000000

[    0.000000] init_memory_mapping: 0000000100000000-0000001040000000

[    0.000000] Reserving 128MB of memory at 752MB for crashkernel (System RAM: 66560MB)

[    0.000000] PM: Registered nosave memory: 000000000009f000 - 00000000000a0000

[    0.000000] PM: Registered nosave memory: 00000000000a0000 - 00000000000dc000

[    0.000000] PM: Registered nosave memory: 00000000000dc000 - 0000000000100000

[    0.000000] PM: Registered nosave memory: 00000000bfef0000 - 00000000bfeff000

[    0.000000] PM: Registered nosave memory: 00000000bfeff000 - 00000000bff00000

[    0.000000] PM: Registered nosave memory: 00000000c0000000 - 00000000f0000000

[    0.000000] PM: Registered nosave memory: 00000000f0000000 - 00000000f8000000

[    0.000000] PM: Registered nosave memory: 00000000f8000000 - 00000000fec00000

[    0.000000] PM: Registered nosave memory: 00000000fec00000 - 00000000fec10000

[    0.000000] PM: Registered nosave memory: 00000000fec10000 - 00000000fee00000

[    0.000000] PM: Registered nosave memory: 00000000fee00000 - 00000000fee01000

[    0.000000] PM: Registered nosave memory: 00000000fee01000 - 00000000fffe0000

[    0.000000] PM: Registered nosave memory: 00000000fffe0000 - 0000000100000000

[    0.000000] please try 'cgroup_disable=memory' option if you don't want memory cgroups

[    0.042301] Initializing cgroup subsys memory

[    0.796974] Freeing initrd memory: 7204k freed

[    1.042239] Non-volatile memory driver v1.3

[    1.053572] Freeing unused kernel memory: 1360k freed

[    1.057189] Freeing unused kernel memory: 1612k freed

[    1.059601] Freeing unused kernel memory: 948k freed

[    1.095428] mptbase: ioc0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total memory = 65980064 kB

Reply
0 Kudos
dariusd
VMware Employee
VMware Employee

OK, the following two lines:

[    0.000000] init_memory_mapping: 0000000000000000-00000000c0000000

[    0.000000] init_memory_mapping: 0000000100000000-0000001040000000

show that we are presenting precisely 65536 MiBytes (67108864 kiBytes, or precisely 68719476736 bytes) of RAM to the guest, as expected.

I don't recall precisely what /proc/meminfo's MemTotal value means, but it doesn't appear to be total system RAM.  Maybe it excludes the page tables (which are shown separately in /proc/meminfo), the crash kernel and its supporting data structures, and/or various other bits of "reserved" memory.  If you grep for "Memory:" in dmesg, it should give a higher-level breakdown of how the physical memory is being used, and its grand total should be much closer to your VM's 67108864 kBytes.  (There will always be a little bit of memory reserved by the firmware and also reserved for various x86 architectural needs.)

I'm seriously rusty on this stuff, but if you attach the guest's whole dmesg as a file (please use the forum's file attachment facility, not copy-and-paste) I can see if there's anything particularly interesting within.

Cheers,

--

Darius

Reply
0 Kudos
onthefly
Contributor
Contributor

thanks for your reply, maybe we cannot find out the reason for this phenomenon easily, it is a little complicated for me.

thank you all.

Reply
0 Kudos
Angerak
Contributor
Contributor

hello, I am presenting the same situation 64gb of ram was allocated from a vmware esxi 6.7, and it only shows 62gb on the server based on a linux centos

 

resolved the situation?

thx

Reply
0 Kudos