Linux hugepages: There needs to be a ramPath= .vmx option

[BitOBear]

So I know all about tmpDirectory= and I know about useNamedFile=false but in no combination do these two options let vmware on a Linux host OS efficently use "hugepages". There needs to be a way to put the ram0 files, and _just_ the ram0 files, into a particular directory.

Background: The linux kernel allows allocation of "huge" memory pages, use of which greatly improves memory access speeds for very large applications. The system administrator makes this available by allocating some memory to the huge page system and then mounting a hugetlbfs psudo filesystem. Applications then create and mmap() files in this file system to allocate huge pages. This is a win because thes pages are typically multiple-megabytes each in size instead of just 4k each. This greatly reduces virtual memory costs associated with loading/reloading page descriptors. (Fortechnical details look up TLB a.k.a Translation Lookaside Buffers.)  Uising this method some applications that use large amounts of memory can be accelerated substantially. The win for huge pages is, well, huge; for example the qemu-kvm open-source virtualization system will gain nearly 10% execution speed running a windows guest in a linux host when huge pages are used.

VMWare Workstation is already compatible at a system call level wiht the huge page allocation system. That is, they way the products mmap() the ram0 (etc) files is exactly what is needed. But since this always happens in "tmpDirectory" or always happens in /tmp if you use the useNamedFile option means that either the files end up in the wrong place or are accompanied by a lot of files log files etc. that don't deserve multi-megabyte memory allocations.

I am hoping there is a secret ramPath= (or similar) setting that can redirect _just_ the ram file mappings to a particular location. (Directories in the huge page file system are okay so it is okay if the option works like tmpDirectory in creating a subdirectory, as long as the stupid log files and other noise dont go there too.)

Is there a secret option of this type or someone to nag to make this happen?

In the alternate, it would be magically special if vmware had a "use huge pages" check box and would then just look for a mounted and writable hugetblfs file system if that box were checked.  Just saying...

[continuum]

did you notice that Workstation 7 on Linux no longer reads mainmem.useNamedFile ?

I explained the changes here on my site
http://faq.sanbarrow.com/index.php?solution_id=1075

> I am hoping there is a secret ramPath=

nope - if it exists it is really top secret 😉

[BitOBear]

I did some more research and if the VMware products would use the MAP_HUGETLB flag for all their memory mappings of the ram backing file this would magically improve performance system wide regardless of the location of the file and remove any remaining issues and be compatable with or properly ignorecd by every currently in-use linux kernel.

The 2.6.38 kernel with Transparent Hugepages set to "always" will scavenge up these mappings into hugepages but it isn't quite as efficent at optimal mapping as woudl be the case with explicit use of the flag since the leading and trailing pages, if not on the hugepage boundaries (which would happen automagically with the MEM_HUGETLB flag set) lead to either rounding or over-allocation or under-use of the feature [don't know which]).

I haven't dug into the module soruce.to see if this can be coerced there as the real-world project I am on just barely gives me time to notice and gripe about this issue.  8-)

NOTE: with transparent hugepages activated some things finish faster in a VM then they finish in native windows. In particular I have been using the latest "MyDefrag" just to exercise the disk and memory "aggregate feel' timings, since it does memory intensive disk intensive system access and between the Host OS caching and condensation of some of the disk flushs and the better memory behaviors, well its kinda stunningly fast.