12 Replies Latest reply on Jan 19, 2016 5:47 AM by onthefly

    The total physical memory size is less than the memory assigned to the vm, it's weird.

    onthefly Novice

      I created a virtual machine and assigned 64GB memory for the VM on the vCenter 6.0 server, installed the suse 11 sp3 64bit OS on it.

      I found only 62GB memory allocalted for the VM when i looked at the /proc/meminfo file inside the VM.

      MemTotal:       65979232 kB

      MemFree:        64453488 kB

      Buffers:           30264 kB

      Cached:           696196 kB

      SwapCached:            0 kB

      Active:           316680 kB

      Inactive:         436768 kB

      Active(anon):      27068 kB

      Inactive(anon):       36 kB

       

      >>> 65979232/1024

      64432

      >>> 65979232/1024

      64432

      >>> 64432/1024

      62

       

      it is quite strange, does anyone know what's reason for this?

        • 1. Re: The total physical memory size is less than the memory assigned to the vm, it's weird.
          linotelera Hot Shot
          vExpert

          Hi

          please could you post the content of vmx file?

           

          Regards

          • 2. Re: The total physical memory size is less than the memory assigned to the vm, it's weird.
            onthefly Novice

            hi linotelera,

            thanks for your reply, the content of the vmx file are the followings:

             

            .encoding = "UTF-8"

            config.version = "8"

            virtualHW.version = "11"

            nvram = "E1_Main_ICPRunner01.nvram"

            pciBridge0.present = "TRUE"

            svga.present = "TRUE"

            pciBridge4.present = "TRUE"

            pciBridge4.virtualDev = "pcieRootPort"

            pciBridge4.functions = "8"

            pciBridge5.present = "TRUE"

            pciBridge5.virtualDev = "pcieRootPort"

            pciBridge5.functions = "8"

            pciBridge6.present = "TRUE"

            pciBridge6.virtualDev = "pcieRootPort"

            pciBridge6.functions = "8"

            pciBridge7.present = "TRUE"

            pciBridge7.virtualDev = "pcieRootPort"

            pciBridge7.functions = "8"

            vmci0.present = "TRUE"

            hpet0.present = "TRUE"

            numvcpus = "16"

            cpuid.coresPerSocket = "16"

            memSize = "65536"

            scsi0.virtualDev = "lsilogic"

            scsi0.present = "TRUE"

            ide1:0.startConnected = "FALSE"

            ide1:0.deviceType = "cdrom-image"

            ide1:0.fileName = "/vmfs/volumes/56822d39-76a94110-b502-a4dcbe1b95de/SLES-11-SP3-DVD-x86_64-GM-DVD1.iso"

            ide1:0.present = "TRUE"

            floppy0.startConnected = "FALSE"

            floppy0.clientDevice = "TRUE"

            floppy0.fileName = "vmware-null-remote-floppy"

            ethernet0.virtualDev = "vmxnet3"

            ethernet0.networkName = "VM Network"

            ethernet0.addressType = "vpx"

            ethernet0.generatedAddress = "00:50:56:9d:76:fd"

            ethernet0.uptCompatibility = "TRUE"

            ethernet0.present = "TRUE"

            scsi0:0.deviceType = "scsi-hardDisk"

            scsi0:0.fileName = "E1_Main_ICPRunner01-000003.vmdk"

            scsi0:0.present = "TRUE"

            displayName = "E1_Main_ICPRunner01"

            guestOS = "sles11-64"

            toolScripts.afterPowerOn = "TRUE"

            toolScripts.afterResume = "TRUE"

            toolScripts.beforeSuspend = "TRUE"

            toolScripts.beforePowerOff = "TRUE"

            uuid.bios = "42 1d 69 cf bd c8 21 4f-5b b5 a7 a7 89 59 fd d5"

            vc.uuid = "50 1d a4 06 ab e1 c7 86-5e 3c 87 bd 41 cf 0d 3e"

            virtualHW.productCompatibility = "hosted"

            sched.swap.derivedName = "/vmfs/volumes/568c8c73-a75e20b8-87cb-a4dcbe1b95dc/E1_Main_ICPRunner01/E1_Main_ICPRunner01-13938b86.vswp"

            uuid.location = "56 4d 22 9d 43 ff 3c 68-ee e9 10 7c aa 1e 4c 1a"

            replay.supported = "FALSE"

            replay.filename = ""

            migrate.hostlog = "./E1_Main_ICPRunner01-13938b86.hlog"

            scsi0:0.redo = ""

            pciBridge0.pciSlotNumber = "17"

            pciBridge4.pciSlotNumber = "21"

            pciBridge5.pciSlotNumber = "22"

            pciBridge6.pciSlotNumber = "23"

            pciBridge7.pciSlotNumber = "24"

            scsi0.pciSlotNumber = "16"

            ethernet0.pciSlotNumber = "160"

            vmci0.pciSlotNumber = "32"

            vmci0.id = "-1990591019"

            monitor.phys_bits_used = "42"

            vmotion.checkpointFBSize = "4194304"

            vmotion.checkpointSVGAPrimarySize = "4194304"

            softPowerOff = "FALSE"

            tools.remindInstall = "FALSE"

            bios.forceSetupOnce = "FALSE"

            sched.scsi0:0.throughputCap = "off"

            toolsInstallManager.lastInstallError = "0"

            toolsInstallManager.updateCounter = "8"

            extendedConfigFile = "E1_Main_ICPRunner01.vmxf"

            tools.syncTime = "FALSE"

            sched.scsi0:0.shares = "normal"

            ide1:0.allowGuestConnectionControl = "TRUE"

            cleanShutdown = "FALSE"

            • 3. Re: The total physical memory size is less than the memory assigned to the vm, it's weird.
              onthefly Novice

              i guess the VMware hypervisor may consumes some memory on the VM, but I am not sure.

              • 4. Re: The total physical memory size is less than the memory assigned to the vm, it's weird.
                linotelera Hot Shot
                vExpert

                If you could make test to this vm: try to poweroff and reduce the number of vCPUs (1/2 from the original)...

                • 5. Re: The total physical memory size is less than the memory assigned to the vm, it's weird.
                  onthefly Novice

                  I will try this next week, but I think the CPU configuration is not related to the memory, and what is your consideration?

                  • 6. Re: The total physical memory size is less than the memory assigned to the vm, it's weird.
                    linotelera Hot Shot
                    vExpert

                    I think (I'm not sure!!|!) these overhead could be:

                    - vCPU and vNUMA relation and HotAdd (in vSphere 5.5 from 8 vCPU and hot add enabled, could cause an increased consumption due to hot add feature overhead)

                    - 3d graphics options (because virtual memory is shared by all virtual hardware architecture, reading the official guide vSphere 6.0 Documentation Center , seems that this feature could cause a memory overhead)

                     

                    But the first feature is disabled by default (you should enable manually) and the second seems to be too high (2GB!!!)

                    • 7. Re: The total physical memory size is less than the memory assigned to the vm, it's weird.
                      dariusd Virtuoso
                      VMware EmployeesUser Moderators

                      Does the VM's dmesg or kernel.log contain any complaints about "CPU MTRRs don't cover all of memory"?  If so, you can probably work around the issue by adding the disable_mtrr_trim option to the guest's Linux kernel command-line (as a one-off at your bootloader's prompt, or permanently via your grub/lilo/yaboot configuration).


                      Cheers,

                      --

                      Darius

                      • 8. Re: The total physical memory size is less than the memory assigned to the vm, it's weird.
                        TomHowarth Guru
                        User ModeratorsvExpert

                        could it be as simple as the difference between decimal and octal notation?

                        • 9. Re: The total physical memory size is less than the memory assigned to the vm, it's weird.
                          onthefly Novice

                          today I adjusted the vCPU configuration to 8vCPU and checked the memory again, the result was not changed, still showed 62GB

                          • 10. Re: The total physical memory size is less than the memory assigned to the vm, it's weird.
                            onthefly Novice

                            hi dariusd,

                             

                            The dmesg did't contain the complaints you mentioned, do you have some other ideas?

                            I found some messages about the memory in the output of the dmesg.

                             

                            [    0.000000] initial memory mapped : 0 - 20000000

                            [    0.000000] Base memory trampoline at [ffff88000009a000] 9a000 size 20480

                            [    0.000000] init_memory_mapping: 0000000000000000-00000000c0000000

                            [    0.000000] init_memory_mapping: 0000000100000000-0000001040000000

                            [    0.000000] Reserving 128MB of memory at 752MB for crashkernel (System RAM: 66560MB)

                            [    0.000000] PM: Registered nosave memory: 000000000009f000 - 00000000000a0000

                            [    0.000000] PM: Registered nosave memory: 00000000000a0000 - 00000000000dc000

                            [    0.000000] PM: Registered nosave memory: 00000000000dc000 - 0000000000100000

                            [    0.000000] PM: Registered nosave memory: 00000000bfef0000 - 00000000bfeff000

                            [    0.000000] PM: Registered nosave memory: 00000000bfeff000 - 00000000bff00000

                            [    0.000000] PM: Registered nosave memory: 00000000c0000000 - 00000000f0000000

                            [    0.000000] PM: Registered nosave memory: 00000000f0000000 - 00000000f8000000

                            [    0.000000] PM: Registered nosave memory: 00000000f8000000 - 00000000fec00000

                            [    0.000000] PM: Registered nosave memory: 00000000fec00000 - 00000000fec10000

                            [    0.000000] PM: Registered nosave memory: 00000000fec10000 - 00000000fee00000

                            [    0.000000] PM: Registered nosave memory: 00000000fee00000 - 00000000fee01000

                            [    0.000000] PM: Registered nosave memory: 00000000fee01000 - 00000000fffe0000

                            [    0.000000] PM: Registered nosave memory: 00000000fffe0000 - 0000000100000000

                            [    0.000000] please try 'cgroup_disable=memory' option if you don't want memory cgroups

                            [    0.042301] Initializing cgroup subsys memory

                            [    0.796974] Freeing initrd memory: 7204k freed

                            [    1.042239] Non-volatile memory driver v1.3

                            [    1.053572] Freeing unused kernel memory: 1360k freed

                            [    1.057189] Freeing unused kernel memory: 1612k freed

                            [    1.059601] Freeing unused kernel memory: 948k freed

                            [    1.095428] mptbase: ioc0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total memory = 65980064 kB

                            • 11. Re: The total physical memory size is less than the memory assigned to the vm, it's weird.
                              dariusd Virtuoso
                              User ModeratorsVMware Employees

                              OK, the following two lines:

                               

                              [    0.000000] init_memory_mapping: 0000000000000000-00000000c0000000

                              [    0.000000] init_memory_mapping: 0000000100000000-0000001040000000

                               

                              show that we are presenting precisely 65536 MiBytes (67108864 kiBytes, or precisely 68719476736 bytes) of RAM to the guest, as expected.

                               

                              I don't recall precisely what /proc/meminfo's MemTotal value means, but it doesn't appear to be total system RAM.  Maybe it excludes the page tables (which are shown separately in /proc/meminfo), the crash kernel and its supporting data structures, and/or various other bits of "reserved" memory.  If you grep for "Memory:" in dmesg, it should give a higher-level breakdown of how the physical memory is being used, and its grand total should be much closer to your VM's 67108864 kBytes.  (There will always be a little bit of memory reserved by the firmware and also reserved for various x86 architectural needs.)

                               

                              I'm seriously rusty on this stuff, but if you attach the guest's whole dmesg as a file (please use the forum's file attachment facility, not copy-and-paste) I can see if there's anything particularly interesting within.

                               

                              Cheers,

                              --

                              Darius

                              • 12. Re: The total physical memory size is less than the memory assigned to the vm, it's weird.
                                onthefly Novice

                                thanks for your reply, maybe we cannot find out the reason for this phenomenon easily, it is a little complicated for me.

                                thank you all.