How do VDDK's asynchronous I/O function for NBD worked?

[takitsuki]

Recently, we have been optimizing the NBD transfer rate for VDDK and came across two asynchronous transfer functions in the article "Best Practices for NBD Transport": VixDiskLib_ReadAsync and VixDiskLib_WriteAsync. We could only find limited information about these functions and still do not fully understand the principles of asynchronous operation. Here are my three personal guesses:

1. VMware mentioned that when NBD reads data, it temporarily stores it in the server's buffer before sending it over the NBD link. The next chunk of data is not read until after the current one has been sent. My guess is that asynchronous operations are implemented internally within the server (although our validations seem to suggest otherwise).

2. These two functions may achieve asynchrony by executing callback functions locally, thus separating the read/write operations from the execution of the callbacks.

3. They could perform multiple read or write operations at once, without waiting for each read to complete entirely, and then wait for all asynchronous operations to finish together at the end.

Another point I would like to understand is the use of the VixDiskLib_Wait function. In the vixDiskLibSample.cpp, it is placed outside the for loop, which means it is not necessary to wait for a ReadAsync or WriteAsync operation to complete fully before proceeding to the next iteration of the loop. All asynchronous operations can be waited upon collectively at the end. My third guess arises from this observation.

Finally, regarding the speed comparison mentioned in VMware's article, I would like to confirm that this comparison includes the time for data processing, not just the data transfer time:

- One stream read over a 10 Gbps network with async I/O, the speed of NBD is approximately 210 MBps.
- One stream read over a 10 Gbps network with block I/O, the speed of NBD is approximately 160 MBps.
- One stream write over a 10 Gbps network with async I/O, the speed of NBD is approximately 70 MBps.
- One stream write over a 10 Gbps network with block I/O, the speed of NBD is approximately 60 MBps.