VMFS block size

The filesystem designed by VMware can use different block size during the datastore initialization. Each size defines some limits.

For more information see: VMFS block size

 

One interesting aspect of VMFS3, is the sub-block allocation as described in KB 1003565: sub blocks are used for directories and small files with size smaller than 1 MB. When the VMFS uses all the sub block (4096 sub blocks of 64 KB each), file blocks will be used. For files of 1 MB or higher, file blocks are used.

 

But the number of 64K sub-blocks is limited, reached this limit also small files or directory entries will use "full" blocks.

 

-


Finding the sub-block limit

Seems that this limit is not documented (or not too clear documented), so how many sub-blocks are available?

One way could be try to debug VMFS metadata or from the first 20MB of each datastore partition (see for example: http://blog.laspina.ca/ubiquitous/understanding-vmfs-volumes or http://kb.vmware.com/kb/1020645) or by using the linux VMFS "driver" (http://code.google.com/p/vmfs/).

Seems that "usual" vmware commands (like vmkfstools) do not give this info (but I've not tried too much).

 

So I've tried a simple approach to find this limit: just make a lot of small file and see when they do not use sub-block size.

In my case I've used ESX (just to have the COS and full Linux commands) and for each volume I've used those commands:

cd /vmfs/volumes/DATASTORE_NAME
echo "Find the number of sub-blocks currently used"
du -ah | grep "^64K" | wc -l
echo "Find the number of directory in the datastore"
ls -l | wc -l
echo "Start building small files"
mkdir delme ; cd delme
for FILE in $(seq 1 3999); do dd if=/dev/zero bs=1k count=1 of=$FILE 2> /dev/null; du -sh $FILE; 

After than, remember to remove the "delme" folder.

 

The results are interesting:

 

datastore1 - 100 existing sub-blocks - 15 dirs

64K     3845

1.0M    3846

 

datastore2 - 212 existing sub-blocks - 27 dirs

64K     3721

2.0M    3722

 

 

So seems that for each datastore the magic number (by adding also the number of directory) is 3960.

Block size seems that does not change this number (datastore1 is 1MB and the other is 2MB).

But probably the number is just a coincidence (due to the usage of only 2 datastores), because I've repeat the same test on some local datastores, and the number isn't the same (but it's always really near this value).

Not sure if minor version of VMFS3 could change this value.

 

-


Conclusion

The number of sub-blocks seems reasonable for a VMFS3 datastore.

Usually the number of folders and small files not not too big, because the number of VMs per datastore is limited (and for each VMs the number of related files is also limited).

This mean than using large block size is not a serius problem in disk usage.