1 2 3 Previous Next 39 Replies Latest reply on Jun 30, 2012 6:30 AM by icando9

    Disable host file cache for guest machine

    mrJoe Novice

       

      Hello,

       

       

      Is there a way for vmware to not cache the vmdk hard disk files? I run some benchmarks in vmware-servers and I need all the files to be read from the disk all the time.

       

       

      The host is linux ubunutu. The guest is windows XP. I have allready added the following entries to the .vmx file, but they don't seem to make any difference.

       

       

      diskLib.dataCacheMaxSize = "0"

      diskLib.dataCacheMaxReadAheadSize = "0"

      diskLib.dataCacheMinReadAheadSize = "0"

      diskLib.dataCachePageSize = "4096"

      diskLib.maxUnsyncedWrites = "0"

       

       

      My guess is that the *vmdk files are cached by the host, is there any way to tell vmware that the host should not cache the vmdk files. Or is there another way to avoid this?

       

       

      Thanks

       

       

        • 1. Re: Disable host file cache for guest machine
          Peter_vm Guru
          mrJoe wrote:

           

          Is there a way for vmware to not cache the vmdk hard disk files? I run some benchmarks in vmware-servers and I need all the files to be read from the disk all the time.

           

          ? What are your exact findings? What do you mean by "read from the disk all the time"? They are read as needed.

          It is host operating system which does the caching.

          • 2. Re: Disable host file cache for guest machine
            mrJoe Novice
            Peter_vm wrote:

            ? What are your exact findings? What do you mean by "read from the disk all the time"? They are read as needed.

            It is host operating system which does the caching.

             

            What I meant is, how can I prevent the host operating system from caching the virtual harddisk files.

            • 3. Re: Disable host file cache for guest machine
              Peter_vm Guru

              Since virtual disk is just a file, check in Ubuntu forums how to disable OS file caching for specific files.

              That has nothing to do with VMware.

              But even if you manage that, performance will be worse, not better. Is that what you want it achieve?

              • 4. Re: Disable host file cache for guest machine
                mrJoe Novice
                Peter_vm wrote:

                But even if you manage that, performance will be worse, not better. Is that what you want it achieve?

                 

                Yes, that is exactly what I want.

                AFAIK an application (in this case vmware) can tell the OS to not cache certain files, so that is why I was hoping there is such a setting in vmware.

                • 5. Re: Disable host file cache for guest machine
                  Peter_vm Guru

                  Do you know any other application running on Ubuntu, that can "tell the OS not to cache certain files"?

                  • 6. Re: Disable host file cache for guest machine
                    mrJoe Novice
                    Peter_vm wrote:

                    Do you know any other application running on Ubuntu, that can "tell the OS not to cache certain files"?

                     

                    I wouldn't know if there are actually other programs which use this, this is actually the first time I needed something like this. However, I know there is a function in linux (this is not ubuntu specific) to tell the OS that a file is not needed using the posix_advice function:

                     

                           POSIX_FADV_DONTNEED  attempts  to  free  cached  pages associated with the specified region.

                          This is useful, for example, while  streaming  large  files.   A  program  may  periodically

                          request  the  kernel  to  free  cached  data that has already been used, so that more useful

                          cached pages are not discarded instead.

                     

                          Pages that have not yet been written out will be unaffected, so if the application wishes to

                          guarantee that pages will be released, it should call fsync(2) or fdatasync(2) first.

                     

                    I'd have to search which programs actually use this function

                    • 7. Re: Disable host file cache for guest machine
                      mrJoe Novice

                      I have modified virtualbox (another virtualization product, virtualbox.org , for which anyone can download the source code) to use posix_fadvise on sync and on read, and it works exactly the way as expected.

                      The windows xp guest now actually performs the same as on real hardware (ie no super speedy harddrive because it is reading from cached memory)

                      I hope there is such an option in vmware.

                      • 8. Re: Disable host file cache for guest machine
                        devzero Master

                        do you have any performance values for comparison?

                        • 9. Re: Disable host file cache for guest machine
                          mrJoe Novice
                          devzero wrote:

                          do you have any performance values for comparison?

                           

                          What kind of?

                          The time it takes to boot windows with and without caching? Or the number of used buffers with or without the modification?

                          • 10. Re: Disable host file cache for guest machine
                            devzero Master

                            oh - please tell us all of them

                            • 11. Re: Disable host file cache for guest machine
                              mrJoe Novice

                              Results for a windows xp machine configured with 192 MB. The laptop is equiped with an 4200 RPM drive (so fast boot times are not expected)

                               

                              *** Before vbox start
                              
                              jeroen@jeroen-laptop:~$ free -m 
                                           total       used       free     shared    buffers     cached
                              Mem:          2013       1078        935          0         93        575
                              -/+ buffers/cache:        409       1604
                              Swap:         4996          0       4996
                              
                              *** vbox started, buffers 'disabled'
                                           total       used       free     shared    buffers     cached
                              Mem:          2013       1116        897          0         95        596
                              -/+ buffers/cache:        425       1588
                              Swap:         4996          0       4996
                              
                              
                              ********************
                              ** First windows run
                              30 secs
                              jeroen@jeroen-laptop:~$ free -m
                                           total       used       free     shared    buffers     cached
                              Mem:          2013       1116        897          0         95        596
                              -/+ buffers/cache:        425       1588
                              Swap:         4996          0       4996
                              
                              (I think I made a copy and paste error here, as they are equal before the VM started)
                              
                              
                              ** windows stop
                              jeroen@jeroen-laptop:~$ free -m
                                           total       used       free     shared    buffers     cached
                              Mem:          2013       1132        881          0         98        602
                              -/+ buffers/cache:        431       1582
                              Swap:         4996          0       4996
                              
                              
                              ********************
                              ** Second windows run
                              
                              29 secs
                              jeroen@jeroen-laptop:~$ free -m
                                           total       used       free     shared    buffers     cached
                              Mem:          2013       1400        613          0        100        607
                              -/+ buffers/cache:        692       1321
                              Swap:         4996          0       499
                              
                              ** Windows stop
                              jeroen@jeroen-laptop:~$ free -m
                                           total       used       free     shared    buffers     cached
                              Mem:          2013       1138        874          0        102        602
                              -/+ buffers/cache:        434       1579
                              Swap:         4996          0       4996
                              
                              
                              ********************
                              ** Third windows run
                              30 secs
                                           total       used       free     shared    buffers     cached
                              Mem:          2013       1411        601          0        103        608
                              -/+ buffers/cache:        699       1314
                              Swap:         4996          0       4996
                              
                              ** Windows stop
                              jeroen@jeroen-laptop:~$ free -m
                                           total       used       free     shared    buffers     cached
                              Mem:          2013       1155        858          0        106        603
                              -/+ buffers/cache:        444       1568
                              Swap:         4996          0       4996
                              
                              
                              
                              *** VBOX recompiled with buffers 'enabled'
                              jeroen@jeroen-laptop:~/src/vbox$ free -m
                                           total       used       free     shared    buffers     cached
                              Mem:          2013       1404        609          0        122        815
                              -/+ buffers/cache:        466       1547
                              Swap:         4996          0       4996
                              
                              
                              ********************
                              ** First windows run
                              27 secs
                              jeroen@jeroen-laptop:~/src/vbox$ free -m
                                           total       used       free     shared    buffers     cached
                              Mem:          2013       1805        208          0        123        956
                              -/+ buffers/cache:        726       1287
                              Swap:         4996          0       4996
                              
                              
                              ** Windows stop
                              jeroen@jeroen-laptop:~/src/vbox$ free -m
                                           total       used       free     shared    buffers     cached
                              Mem:          2013       1509        504          0        124        917
                              -/+ buffers/cache:        468       1545
                              Swap:         4996          0       4996
                              
                              ********************
                              ** Second windows run
                              10 secs
                              jeroen@jeroen-laptop:~/src/vbox$ free -m
                                           total       used       free     shared    buffers     cached
                              Mem:          2013       1823        190          0        125        971
                              -/+ buffers/cache:        727       1286
                              Swap:         4996          0       499
                              
                              ** Windows stop
                              jeroen@jeroen-laptop:~/src/vbox$ free -m
                                           total       used       free     shared    buffers     cached
                              Mem:          2013       1514        498          0        125        919
                              -/+ buffers/cache:        470       1543
                              Swap:         4996          0       4996
                              
                              
                              ********************
                              ** Third windows run
                              11 secs
                                           total       used       free     shared    buffers     cached
                              Mem:          2013       1830        183          0        126        974
                              -/+ buffers/cache:        729       1284
                              Swap:         4996          0       4996
                              
                              ** Windows stop
                              jeroen@jeroen-laptop:~/src/vbox$ free -m
                                           total       used       free     shared    buffers     cached
                              Mem:          2013       1517        496          0        127        919
                              -/+ buffers/cache:        470       1542
                              Swap:         4996          0       4996
                              
                              
                              

                               

                               

                              The results of cached memory would be better to see if I knew how to tell my host to drop all clean buffers, which I don't know unfortunately. So the results are a little bit 'dirty' right now

                              • 12. Re: Disable host file cache for guest machine
                                devzero Master

                                thanks, but thats a little  bit hard to read.

                                could you post the essential part to see the performance win ?

                                 

                                >The results of cached memory would be better to see if I knew how to tell my host to drop all clean buffers,

                                maybe http://www.linuxinsight.com/proc_sys_vm_drop_caches.html ?

                                • 13. Re: Disable host file cache for guest machine
                                  devzero Master

                                  >Do you know any other application running on Ubuntu, that can "tell the OS not to cache certain files"?

                                   

                                  every application which opens a file with the O_DIRECT flag.

                                   

                                  http://unixfoo.blogspot.com/2008/01/what-is-direct-io.html

                                   

                                  it`s often used with oracle - but i think oracle isn`t often used with ubuntu

                                  http://www.dba-oracle.com/art_orafaq_oracle_direct_io.htm

                                   

                                  so - one may get the whole ubuntu source and grep inside for O_DIRECT

                                   

                                  • 14. Re: Disable host file cache for guest machine
                                    mrJoe Novice
                                    devzero wrote:

                                    could you post the essential part to see the performance win ?

                                     

                                    It is not about peformance win, (actually more a performance loss). Its about having a more realistic virtual machine.

                                    The problem I am having now is that the first time something is read from the virtual harddisk this is normal hd speed. The second time this information is read it is super fast (because the host has cached the virtual harddisk file).

                                     

                                    A side effect is that this caching of virtual harddisk file has a negative impact on the host machine as the host machine/OS has to drop files from the buffer which might actually still be used very soon by the host.

                                     

                                    I have quicky rerun the test with dropped caches:

                                    With buffer caching 'enabled' (default behaviour unless an application tells linux to do otherwise) the memory usage is like this (memory usage is including buffers):

                                    Before: 721MB used

                                    Windows started: 1131 MB used

                                    After: 834MB used

                                     

                                    So after the guest machine is stopped there is 120MB 'left' in the buffers.

                                     

                                     

                                    With caching 'disabled' using posix_fadvise:

                                    Before: 706MB

                                    Windows started: 983MB used

                                    After:  720MB used

                                    (So 14 MB of other kind of files are cached, which might be any program running on the background, including X11)

                                    1 2 3 Previous Next