5 Replies Latest reply on Nov 6, 2008 7:01 AM by kbk00

    Best solution for up- and downloading movies in a Guest-OS

    JimSmits Lurker

       

      Hi all,

       

       

      For your information:

       

       

      Environment:

      1 * ESX 3.5 Upodate 2

      VC 2.5 Update 3

      2 * Dell PE2950

      1 * Dell MD3000i (with latest firmware); 2 Volumes are created; iSCSI-ports are redundant configured.

      2 * PowerConnect 5*

      All VM's are configured on 1 GB speed "autonigociate) as on the PowerConnect Switches.

      Guest-OS on VM's is installed with Ubuntu (was Gentoo before).

       

       

      I have this customer (International school) that provide students to upload and download movies (student material). A local copy of a movie of 800 GB takes ages and even makes it impossible to do anything else on other Virtual Machines.

       

       

      Does anyone have experience in this area?

       

       

      What do you advice to use for uploading and downloading big files in a virtualized environment?

       

       

      Maybe it is not enough information but a start to get some answers.

       

       

      Hope to get good advice.

       

       

      Cheers,

      Jim Smits

      VCP

       

       

       

       

       

        • 1. Re: Best solution for up- and downloading movies in a Guest-OS
          RParker Guru

           

          Sounds like you have networking issues.  It's not related to a VM, we have VM's attached to Gig interfaces, and they don't suffer any bandwidth issues.  800 Meg is quite a large file, so how would you handle this in a physical machine?

           

           

           

           

           

          Have you tested this thoroughly BEFORE you attempted virtualization?  It seems like you came up with this solution after you started using VM's, so it's really not a VM issue, its your infrastructure. 

           

           

           

           

           

          We need more info, where are the files coming from, Internet?  What kind of networks do you 10/100 10/1000?  What kind of NICS.  Do they share bandwidth with many other VM's ALL downloading the same 800 Meg file at once?  It might be a better solution to centralize the location of the files rather than each VM downloading separately.

           

           

          • 2. Re: Best solution for up- and downloading movies in a Guest-OS
            JimSmits Lurker

             

             

             

             

            Hi there ,

             

             

            Thanks for your reply.

            Question 1:

            Have you tested this thoroughly BEFORE you attempted virtualization?  It seems like you came up with this solution after you started using VM's, so it's really not a VM issue, it's your infrastructure. 

             

             

            Answer 1:

            The customer have set up VMware them selves. The hardware MD3000i is installed and configured together with an 3rd party company.  So unfortunately we were not involved at all from the start.

             

             

             

            Question 2:

            We need more info, where are the files coming from, Internet?  What kind of networks do you 10/100 10/1000?  What kind of NICS.  Do they share bandwidth with many other VM's ALL downloading the same 800 Meg file at once?  It might be a better solution to centralize the location of the files rather than each VM downloading separately.

             

             

            Answer 2:

             

             

            The files are coming from the webserver (stored by the administrator).

             

             

            All NIC's are configured on 1000 Auto negotiate.

             

             

            The files are centralized.

             

             

             

            Question 3:

            We need more info, where are the files coming from, Internet?  What kind of networks do you 10/100 10/1000?  What kind of NICS.  Do they share bandwidth with many other VM's ALL downloading the same 800 Meg file at once?  It might be a better solution to centralize the location of the files rather than each VM downloading separately.

             

             

            Answer 3:

            The customer is using internal 1000 networking (see drawing).

             

             

            Yes, they do share bandwidth with the other (about 20) VM's but they are not downloading the same file!

             

             

            Types of NIC's:

            NIC 0 Broadcom NetXtreme II BCM5708; driver bnx2

             

             

            NIC 1 Broadcom NetXtreme II BCM5708; driver bnx2

             

             

            NIC 2 Intel 82571 EB Gigabit Ethernet Controller; driver e1000

             

             

            NIC 3 Intel 82571 EB Gigabit Ethernet Controller; driver e1000

             

             

            NIC 4 Intel 82571 EB Gigabit Ethernet Controller; driver e1000

             

             

            NIC 5 Intel 82571 EB Gigabit Ethernet Controller; driver e1000

             

             

             

             

             

            Other information

             

             

            Switches:

             

             

            PowerConnect 5324 (no iSCSI optimization as on the PowerConnect 54**)

             

             

            Up- and Download servers:

             

             

            There are 2 Webservers, Guest OS is UBUNTU Server 8.0.4.

             

             

            MD300i:

             

             

            The MD3000i is re-installed and configured following the best practice.

            15 * SAS 300 GB 15K

             

             

            The internal copy is taken place on the same server!!! So I can imagine that on a 1GB connection with a speed of 100MB per second (read and write) can take approximately 4 hours.

             

             

            It hangs up the total environment!

             

             

            The 2 Webservers are having its own set of disks.

            Webserver 1 has a RAIDset of 5 disk in a RAID5.

            Webserver 2 has a RAIDset of 4 disks in a RAID5.

            There is one hotspare and the rest of the disks being used for Virtual Machines.

             

             

            Additional info:

             

            Object                                                 Total

            Shared Datastores                           3

            Datastores                                          4

            Hosts                                                   3

            Virtual Machines                             32

             

             

            Operating System                          Running              Total

            Microsoft W2K3 SE 32-bit            5                             5

            Microsoft W2K3 EE 32-bit            9                             10

            Microsoft W2K8 32-bit                  0                             1

            Ubuntu                                            11                           15

            Suse Linux ES 32-bits                 0                             1

             

             

            What I am thinking about:

             

            • 1. I would like to use VLAN for these WebServers.

            • 2. Configure the PowerConnect 5324 and NIC's for these VM's for Jumbo Frames.

             

            Is there something else from this point what you can suggest?

             

             

            Looking forward to your answer.

             

             

            Regards,

             

             

            Jim

             

             

             

             

             

             

             

             

            • 3. Re: Best solution for up- and downloading movies in a Guest-OS
              polysulfide Expert

               

              I agree that you should test non-virtual before you try virtual.

               

               

              Then I would do the following on your Virtual Machine

              Make SURE you're using the LSI scsi driver

              If the movies are on a VMDK and not an RDM, make sure that you align the partition when you create it using diskpart

              create partition primary align=64 (This can make a huge I/O difference)

              Make sure you're using the vmxnet NIC driver

              Increase your TCP window and MTU sizes

              Consider using a file transfer protocol other than SMB/CIFS

              Consider archiving the larg file into a collection of smaller files that are easier to resume

               

              These are just best practices I would reccomend before you start too deep into troubleshooting.

               

               

               

               

               

               

               

              If it was useful, give me credit

              Jason White - VCP

               

               

              • 4. Re: Best solution for up- and downloading movies in a Guest-OS
                Anders Gregersen Expert

                Assuming that the 800GB isn't a typo, bandwidth to the storage could easily be the problem. iSCSI can handle IOPS just fine, but it limited by the 1Gb wirespeed. For that kind of bandwidth requirements I would recommend that the customer switch to FC based storage or 10GbE iSCSI. You can check this in esxtop or check utilization on the switches.

                • 5. Re: Best solution for up- and downloading movies in a Guest-OS
                  kbk00 Hot Shot

                  How does your storage look when you are doing a transfer?  Disk utilization, proc and mem performance, too...  I would frankly be surprised to find that your environment is being brought to its knees by NIC throughput.  My guess is that it's something storage related.  Someone mentioned disk alignment which is something to look into as well. 

                   

                  Whenever I've exerienced this kind of issue myself, it's always been storage related.  We use NFS but it still uses copper like iscsi does.

                   

                  It'll be interesting how this turns out...




                  If you found this posting to be useful, great.  I didn't waste your time.