Virtual disk, why really zeroing at write?

[rickardnobel]

To prevent old data on a LUN to be accessable for a new guest operating system it shall be overwritten (zeroed) before given to the guest OS. That makes sense.

And from what I understand if the guest does a READ against an unaccessed part of the disk the vmkernel will return zeros without going to the disk. Is that correct?

And (besides eager zeroed thick disks) a zeroing operation will take place at the first guest WRITE against a new location. My question is however, why? If a write is already decided for, would it not be enough to commit that write and potential old data "behind" the new vmdk will be erased anyway?

[Maximenu]

I used the zeroing when i created the Quorum disk for MSCS. But I don´t know why

[rickardnobel]

Javier Galvez wrote:

I used the zeroing when i created the Quorum disk for MSCS. But I don´t know why

I think that when you use MSCS you should have a "eager zeroed thick disk", that is complete zeroed before given to the guest, for best write performance.

However, on an ordinary thick disk - why the need to zero something that shall be overwritten?

[DSTAVERT]

Your opening statement is probably the real answer. If you have decided that

To prevent old data on a LUN to be accessable for a new guest operating  system it shall be overwritten (zeroed) before given to the guest OS.  That makes sense.

then you can take the time (which can be considerable) to zero out the whole drive or you can have each block zeroed as it is accessed. Same ultimate goal.

[rickardnobel]

David Stavert wrote:

... or you can have each block zeroed as it is accessed. Same ultimate goal.

Thanks for your reply. But I actually still not understand the reason for the operation, even if I understand the goal.

If the guest OS is about to commit a write to a sector, why shall the vmkernel take the time to erase the sector and then overwrite it with the guest OS request? It seems kind of not necessary.

Is it know at what sizes the zeroing is done at? That is, how much space is zeroed at the same time?

[DSTAVERT]

There are three thick disk types. Thick, Zeroed Thick and Eager Zeroed Thick. The type you are describing is just a Thick disk. There is no requirement to zero the blocks either initially or as they are written to. A plain thick disk would behave as you suggest. At first write the new data would simply overwrite any existing information on the block.

[rickardnobel]

David Stavert wrote:

There are three thick disk types. Thick, Zeroed Thick and Eager Zeroed Thick. The type you are describing is just a Thick disk. There is no requirement to zero the blocks either initially or as they are written to. A plain thick disk would behave as you suggest. At first write the new data would simply overwrite any existing information on the block.

Are you sure about this? I think there earlier existed disks that might been called "thick disks" in previous versions of ESX that acted the way you describe, but I do think that the vSphere "thick disk" does a zeroing-at-first-write.

[DSTAVERT]

Is it know at what sizes the zeroing is done at? That is, how much space is zeroed at the same time?

The block size would be the block size that your datastore was created with. If you created a datastore with 1M block size then that would be the size of the space that would be zeroed at the time of access. Zeroedthick is the default which zeros the block at the time of access.

[DSTAVERT]

Have a look at http://kb.vmware.com/kb/1022242 for a description of disk types.

[rickardnobel]

Thanks, there is some confusion here regarding the names of the disks, since the old "thick" does not exist, and what the GUI calls "thick" is referred to as "ZeroedThick" in the KB.

Do you have any reference to that the zeroing is done one VMFS block each? That could explain my initial question, as why overwrite something that shall be overwritten.

[depping]

it is one VMFS block at a time indeed. I can confirm it.

-d

[rickardnobel]

Duncan wrote:

it is one VMFS block at a time indeed. I can confirm it.

Great! So that should explain the mystery (to me). If the guest does a 4 KB write against the disk, the Vmkernel deletes all (for example) 8 MB ahead, depending on the VMFS block size.

So even if the zeroing at the first part, say 4KB, is perhaps not necessary, the remaining part of the VMFS block is "secured" and could be given to the guest as well. Is that correct?

[a_p_]

My thoughts about this. IMO the zeroing before actually writing the real data is part of the internal logic of the vmkernel. I guess it checks the block and only writes the real data if it sees the "Z" bit has already been cleared by zeroing the block.

see http://kb.vmware.com/kb/1011170

André.

[depping]

No I/O is not read in "VMFS" blocks. I/O is read in what the Guest OS requests and how the Array is set up. (Array stripe/block size.)

Two things:

1) We zero the full vmfs block as that is the allocation unit on the VMFS level. So we need to allocate the full block for that write and zero all of it.

2) We zero the block on a read as well to avoid people using data recovery tools to steal/recover data that was previously written in that "VMFS Block".

Duncan (VCDX)

Available now on Amazon: vSphere 4.1 HA and DRS technical deepdive

[rickardnobel]

Duncan wrote:

2) We zero the block on a read as well to avoid people using data recovery tools to steal/recover data that was previously written in that "VMFS Block".

You mean the actual zeroing can occur even at a guest READ at a previous non-allocated part of the disk? Everytime I have read about this it mentions that it happens only at first guest write.

Perhaps because most guest operating systems has no use for reading sectors with no logical data in them?

[rickardnobel]

Duncan wrote:

2) We zero the block on a read as well to avoid people using data recovery tools to steal/recover data that was previously written in that "VMFS Block".

I did a quick test today with given a 50 GB thin disk to a Windows 2003 VM. Inside the guest I did a full format of the new partition, which on Win2003 is to create a new file system and do a read against all sectors on the disk. The format was quite quick and the file was not expanded, which leads me to belive that no reads were really done against the disk, or at least, no zeroing was done.

Could there be any difference between the behaviour of a thin or a default (zeroed)thick disk?

[rickardnobel]

I did another test on a default thick disk, where I did a full format with Windows 2003 server, which reads all sectors on the disk. However, I did not really spot any disk writing activity for zeroing in the background, but I might have missed it. It should be very interesting to know if zeroing takes place also when commiting reads against the raw disk?