I'm keen to know more on how the read cache operates in hybrid deployments. How is it decided what data is stored on the caching layer for the read cache?
The purpose of the read cache in hybrid configurations is to maintain a list of commonly accessed disk blocks by VMs. This reduces the I/O read latency in the event of a cache hit; that is, the disk block is in cache and does not have to be retrieved from magnetic disk. The actual block that is being read by the application running in the VM may not be on the same ESXi host where the VM is running. In this case, DOM picks a mirror for a given read (based on offset) and sends it to the correct component. This is then sent to LSOM to find whether the block is in the cache.
If it transpires that there is a cache miss, the data are retrieved directly from magnetic disk in the capacity tier, but of course this will incur a latency penalty and could also impact the number of input/output operations per second (IOPS) achievable by vSAN.
This is the purpose of having a read cache on hybrid vSAN configurations, as it reduces the number of IOPS that need to be sent to magnetic disks. The goal is to have a minimum read cache hit rate of 90%. vSAN also has a read ahead cache optimization where 1 MB of data around the data block being read is also brought into cache.