9 Replies Latest reply on Jun 30, 2007 10:13 AM by SilvioSantoZ

    16Gb Dual Quad Xeon with Debian Etch host - x64, bigmem or x86?

    SilvioSantoZ Novice

      Hello everyone.

       

      We're setting up a new server. It's a dual cpu, quad-core xeon, with 16Gb RAM (actually a Dell 2950, maxed-out). Its host OS is going to be Debian Etch (company policy) and run VMWare Server, hosting about 8 virtual machines (x86, 2gb max each). Running Dell OMSA (OpenManage/RAID monitor) is a requirement, since I need to monitor the machine and RAID arrays constantly.

       

      Given this scenario I have a few options:

      1) Debian Etch x64/amd64.[/b] Googled around and saw that this is possible with VMWare Server. Dunno about performance. Will probably have issues with Dell OMSA.

       

      2) Debian Etch x86 with 'bigmem' kernel.[/b] This would be good since OMSA would run out of the box, but I dunno about VMWare Server. Does it run well with bigmem (CONFIG_HIGHMEM4G) kernels?

       

      3) Debian Etch pure x86.[/b] Great OMSA support, but I dunno if VMWare will be able to handle the 16Gb with this setup.

       

      What would you suggest? Any experience or tips with any of the possible scenarios?

      Thanks a lot!

       

      PS. Windows, Redhat and CentOS are NOT an option - only Debian is allowed.

        • 1. Re: 16Gb Dual Quad Xeon with Debian Etch host - x64, bigmem or x86?
          devzero Master

          x86 with bigmem should do well, but....

           

          >hosting about 8 virtual machines (x86, 2gb max each)

          this is 2x8=16gb for your virtual machines. too much for a host with 16gb - you forgot to include calculation of virtualisation memory overhead. add 15% more ram to your host - or size down your VMs to get good performance.

          also don`t forget setting mainmem.usenamedfile="false" for your VMs

          • 2. Re: 16Gb Dual Quad Xeon with Debian Etch host - x64, bigmem or x86?
            nick.couchman Champion

            with 16Gb RAM (actually a Dell 2950, maxed-out)

             

            We order Dell all the time and I'm pretty sure it has at least 6 slots, maybe even 8.  So I'm thinking you can go to 24GB or 32GB if you really wanted to.  Of course, that's pretty expensive.

             

             

             

            I agree with devzero - x86 with the bigmem kernel should be fine - you will have a lot fewer headaches than the 64 bit distributions sometimes manage to give.  If you don't go with bigmem, you'll only get 4GB RAM - a shame to waste the other 12GB!

             

            If you're spending that much on a server (16GB of RAM and dual quad-core processors), why aren't you looking at ESX??  You could order the dual core processors and ESX would make better use of them than Server will of the quad cores (in my humble opinion/experience).

            • 3. Re: 16Gb Dual Quad Xeon with Debian Etch host - x64, bigmem or x86?
              spectVM Hot Shot

              I faced similar issues on Windows.

               

              I went with Windows 32bit with PAE over Windows 64bit.

              From benchmarks, 64bit performed slightly lower than 32bit with PAE:

              1. because of greater memory overhead on 64bit processes

              2. many applications and drivers are 32bit running in compatibility mode (not optimal)

               

              Add to that a much lower application compatibility, and it makes sense to stick to 32bit for a while.

               

              VMware runs fine with PAE, so I guess it should run fine with BIGMEM.

              You will be limited to less than 4GB with option 3.

              So, go with option 2.

              • 4. Re: 16Gb Dual Quad Xeon with Debian Etch host - x64, bigmem or x86?
                VirtualEntity Hot Shot

                I would go with Option 1 or 2.  Definitely forget about option 3.  Not only is option 3 limited to slightly less than 4 GB, it also maps the memory in such a way that a considerable amount of it is wasted, given your desire to use the machine as a VMware server (ex:  I cannot run 5 512 MB VMs on a 4 GB machine with a stock kernel--I was limited to two 384 GB and three 448 GB), but once I switched to a Bigmem kernel (PAE kernel), all my problems running 5 512 MB VMs went away.

                 

                I am not sure that I agree with the claim that 64-bit system will run slower.  My experience has been that VMware Server 1.03 runs quite nicely on my 64-bit CentOS 5 system at home.  (The system is very similar to your Dell 2950, but with only 4 GB, rather than 16, but it does have the 2 quad core CPUs and it's the fastest thing I've ever seen!)  Though my measurements of its performance are largely subjective, the impression I have is that the 64-bit CentOS 5 with VMware Server 1.03 is significantly faster and far more responsive than the old 32-bit CentOS 4.5/VMware Server 1.03 installation was.  Also, as you are no doubt aware, there will probably be a 64-bit version of VMware Server out sooner or later, and that will be even faster....

                 

                Anyhow, if you don't wish to go 64-bit, definitely, the 32-bit PAE (Bigmem) is the way to go.  Note that with Debian Etch, you will need to apply the "any-to-any" patch to VMware Server, otherwise it won't run.

                • 5. Re: 16Gb Dual Quad Xeon with Debian Etch host - x64, bigmem or x86?
                  nick.couchman Champion

                  I am not sure that I agree with the claim that 64-bit system will run slower. My experience has been that VMware Server 1.03 runs quite nicely on my 64-bit CentOS 5 system at home.

                   

                  Who claimed that 64-bit stuff runs slower??  I don't see that anywhere in this thread.  The user said that he 'didn't know about performance' - indicating (to me, anyway) that he doesn't know whether the 64-bit or 32-bit PAE (bigmem) kernel will run faster.  I certainly didn't say that - I only indicated that 64-bit O/S's have given a lot of people problems (as can be indicated by the number of threads in this forum dealing with 64-bit O/S issues) and that 32-bit might be a bit smoother experience.

                  • 6. Re: 16Gb Dual Quad Xeon with Debian Etch host - x64, bigmem or x86?
                    SilvioSantoZ Novice

                    Thanks for your replies everyone. I think I'll go with bigmem kernel for now, in the hopes of getting Dell OMSA running ok and also being able to see all (or almost all, giving some for the host overhead) physical memory.

                     

                    We're definitely looking into ESX server, the Starter Edition.

                    • 7. Re: 16Gb Dual Quad Xeon with Debian Etch host - x64, bigmem or x86?
                      IdeCable Novice

                      As far as I know, the pure x86 Debian Linux Kernel is \*stable*. I have no  idea for the other arch in Debian.

                       

                      I had to choose between AMD64 and i686. I didn't tested the AMD64 kernel over Debian, only over Gentoo. that was very unstable.

                       

                      I agree that AMD64 Linux Kernel is still in a newborn and still needs to be patched. But the Debian distro takes care of these issues bigtime.

                       

                      I recommend \*trying out* at first a kernel arch that would give you as much power this quad Xeon (these systems are extremely powerful!!). Gve your 64bit OS a good week of benchmarks and \*stress tests*. I've seen many people over the VMTN forums saying they're running VMware Server off Ubuntu (Debian based..) in 64bit without problems.

                       

                      But always trusts what you get confirmed by your own tests.

                       

                      Of course the i686 (or i586?) Debian kernel will be rock solid.. But since you're still at the first stages of installing this server setup, I assume you'll get the chance to test-drive the 64bit version of Debian.

                       

                      I didn't tested the 64 bit versions of Debian yet..

                       

                      PS: Kernel 2.6.18 works with VMware Server. anything over 2.6.18 is not stable over VMware Server (yet). I use Suse 10 (Debian based) over 2.6.13. Never had a single crash ever since.

                      • 8. Re: 16Gb Dual Quad Xeon with Debian Etch host - x64, bigmem or x86?
                        jack-uk Novice

                        Hi,

                         

                        What you are describing is identical to what we are currently in the process of speccing.

                         

                        Have you made any progress with this, did you encounter any 'gotchas' with the system?

                         

                        Any advice is appreciated.

                         

                        TIA,

                         

                        jack

                        • 9. Re: 16Gb Dual Quad Xeon with Debian Etch host - x64, bigmem or x86?
                          SilvioSantoZ Novice

                          Hello! Yes, we're using this setup on 5 Dell Hosts (2950 and 2850, with 2gb up to 16gb RAM). It works wonderfully. I've documented the full procedure at my blog: http://pardini.net/blog/arkivos/2007/06/09/ultimate-debian-etch-vmware-server-setup-16gb-dual-quad-core-dell-2950-with-omsa/

                           

                          There is one important detail on Dells and Debian: set the kernel option hpet=disable in Grub. Else the clock can behave strangely in some situations. Also there's a few more things I'd like to say about OMSA and SNMP. When I have the time I will post here and update the blog entry.

                           

                          All in all, I think this is a really good setup. Thanks to everyone who helped. "bigmem" is excellent. Not having to use patches is a blessing. Thanks VMWare and also the folks at sara.nl for the dellomsa debian package!