3 Replies Latest reply: Jun 26, 2007 8:29 AM by frank.macha RSS

    Server running on Linux Host - Recommended Settings

    frank.macha Novice

      Greetings all...

       

      Can we start a thread which summarizes some of the recommended settings/configs/tidbits with Linux Hosts?  The thought is to capture as many brain dumps as possible so we can reference to when we are deploying vmware server on linux hosts.

      Let me start off with a quick brain dump....

       

      sysctl settings:

        /etc/sysctl.conf         <- some of this might/might not apply to your environment, handle with care

           vm.min_free_kbytes = 16384

           vm.overcommit_memory = 2

           vm.overcommit_ratio = 75

           vm.lower_zone_protection = 80

           vm.swappiness = 60                         

           net.ipv4.tcp_window_scaling = 0

       

      vmware 'global' config settings:    <- is there a list of all of the possible entries which can be in this file?

         /etc/vmware/config

             sched.mem.pshare.enable = "FALSE"    <- does this actually affect Linux hosts?

             mainMem.useNamedFile = "FALSE"

       

      vmware 'vm-specific' vmx settings:

            sched.mem.max =  xxxx   <- this  is VM specific, specially after converting p2v

       

      host general recommendations:

          . Use Vmware Server v1.0.3 or better

          . Use bigmem/hugemem kernels 

          . If possible, use 64bit kernels/distributions

          . XFS or JFS for your VM image filesystem (on top of LVM)   <- this might need to be expanded as filesystem  tuning can be 'interesting'

           . physical RAM/memory rule of thumb:  1.25 * VM allocated memory.  The more physical memory, the better     

       

       

      Of course, any type of tuning will depend heavily on the type of workload the VMs are doing as well as the underlying hardware.  Please add to this topic anything else which might have helped you.

       

      Thanks!

      Frank

        • 1. Re: Server running on Linux Host - Recommended Settings
          boogieshafer Expert

          probably not much point in running a 64bit host if you are only going to run a 32bit app on top of it. vmware workstation 6 is available in 64bit version, but the server product is still 32bit

           

          likewise unless you have 4GB+ of RAM on the host its not even worth considering 64bit

           

          IMO, vmware server is a *SERVER* type of applcation where stability is as important (if not more) than raw performance and your document might want to get into some kernel level specifics about what revisions of kernel work natively, are tested by vmware and which are "unsupported" and must be patched to make work

           

          i would suggest you probably want to add some configuration around controlling the clock features of the HOST os and the underlying hardware (e.g. cripple hardware coolnquiet or speedstep, and related os level services...cripple apic lapic etc in the host kernel)

           

          linux users would probably benefit from installing the mui as well as other related tools for tracking performance (ala snmp, vmktree,etc) are very useful

           

           

          you will probably want to take a peek at these documents if you havent already

          http://www.vmware.com/pdf/WS6_Performance_Tuning_and_Benchmarking.pdf

          http://www.vmware.com/pdf/vi_performance_tuning.pdf

          • 2. Re: Server running on Linux Host - Recommended Settings
            nick.couchman Champion

            Actually, depending on which host O/S you're using, it does make sense to run a 64-bit O/S - especially if you have 4GB or more of memory.  The Linux "bigsmp" kernels will allow you to use more than 4GB of memory on a host, but not quite as well as the true 64-bit kernels.

             

            I agree that sticking to a supported kernel is a good idea - I don't recommend anything higher than 2.6.18 right now for VMware Server - the later ones seem to have a lot of trouble with compiling kernel modules, etc.

            • 3. Re: Server running on Linux Host - Recommended Settings
              frank.macha Novice

              Greetings once again... What to thank you for some feedback and to close this topic out.

              So far, the following seems to be good starting points as reference for folks wanting to use Linux as their Host OS:

               

              . Hardware

                - Ensure that you have decent hardware with supported devices.

               

              . Linux OS

                 - Stick to Stable distributions.  People have had success running

                    RedHat 4.0

                    Fedora Core 5.0

                    Debian 4.0

                    Ubuntu 6.0

               

                    For the most part, stick to Linux Kernels 2.6.18 or less, and the distributions above seem to have done that. 

                 - Your Host OS should be stable, stable stable. Do not populate with custom

                    apps/programs/tgz.

                  - Stick to your distributions rpms/debs/etc with regards to packages you'll need for vmware server

                   - 32bit/64bit?   Your call....

                        . 64bit natively supports 4GB+ memory with no problems/hassles. Will need to install ia32libs in your host to be able to run native 32bit apps (i.e. vmware)

                        . 32bit will need to have a hugemem/bigmem kernel installed to address 4GB+ memory.

               

               

              Linux Additional Tidbits

               

                  . Clock/Timekeeping

                      - You'll have to manipulate your BIOS/motherboard so that none of the fancy powerstep/frequency deceleration is enabled.  

                       -  (hardware dependent) Modify your grub/lilo entries so that your kernel boot parameters include

                          hpet=disable

                          By disabling hpet, you will have linux return back proper RTC values.

                       - Modify your grub/lilo entries so that your kernel boot parameters include

                          clocksource=pit

                            There are other options for clocksource, pit seems to be the more secure/stable.

                   . Good reliable timekeeping vs performance

                       -   By disabling hpet, you'll notice vmware-rtc ( and the counters to the timer in /proc/interrupts) being heavily used. This can be good thing, or bad thing. Your host basically has additional load in handling RTC.

                       -  If you don't want this additional load on your host, then reenable hpet, and in your /etc/vmware/config file, put the entry

                           host.usefastclock = "FALSE"

               

                        - Enable the vmware-tools setting on your guests to sync up time with your host

                        - Enable NTP at your host, have it sync up with ntp.org servers and/or an in-house ntp server.

                        - Enable your guest NTP to sync up with the host

                              Windows guest:  net time /setsntp:HOSTIP

                               Linux guests:  update /etc/ntp.conf with HOSTIP

               

                  . Sysctl / VMM settings

                        - Use the following as starting points to get the most out of your host:

                       vm.min_free_kbytes = 16384

                       vm.overcommit_memory = 2

                       vm.overcommit_ratio = 75

                       vm.lower_zone_protection = 80

                       vm.swappiness = 60

                        

                           Needless to say, review each of these as to how it will apply in your environment. These settings can help (or hurt) you with regards to OOM (out of memory conditions).  Handle with care.

               

                        - Consider modifying the following settings as well:

                       net.ipv4.tcp_window_scaling = 0

                       net.core.rmem_max =

                       net.core.wmem_max =

                       net.ipv4.tcp_rmem =

                       net.ipv4.tcp_wmem =

                       net.core.netdev_max_backlog =

               

                        - Consider updating your NICs behavior:

                        ethtool -K ethX tx off

                        ethtool -K ethX sg off

                        ethtool -K ethX tso off

               

                         - Consider syncing your filesystems via sync (crontab entry)

                        5,20,35,50 * * * *  /bin/sync > /dev/null &

               

               

              VMware considerations

                      - Memory management

                              . /etc/vmware/config

                                mainMem.useNamedFile = 'False'   - ensure that your /tmp has enough space to hold your VMs assigned memory

               

                       - Vm vmx entries

                               sched.mem.max = xxx update/remove entry

                               mainmen.trimrate = '0'   update/remove entry

               

                        - Start off with  a 'clean' VMX file when moving/converting VMs around your environment.  Converter is known to populate your vmx file with entries not needed by your VMs.

               

               

              Anyhow, hope this helps some of you.