VMware Cloud Community
esand
Contributor
Contributor

disk alignment in vm guest os

I've noticed some threads online regarding aligning data inside guest oses and have been quite curious about this technique and when to apply it. In all the articles I came across, they only mentioned disk alignment relating to using ESX, but I use Workstation and Server only and I haven't really found any information indicating it should be done or is required on those products (especially since I don't use a SAN - the vm's are on local disks).

For anyone unsure about what I'm referring to, the disk alignment I'm talking about is the offset/start of a partition on a disk (in this case, in the guest os... so pertaining to a vmdk file). I only have Linux vms and in Linux, if you run "fdisk -ul" you will get a list out of all your disks and their partitions including their starting sectors. Typically a normal partition created with fdisk is at sector 63 (one confusing point I'd like to mention is... is that actually 64 sectors from the start of the drive? I'd assume sector counts start at 0, thus aligning data might not even be an issue - read further as to why), but documents state you should align the start at sector 128. This is what the output looks like of an aligned disk:

Disk /dev/sda: 10.7 GB, 10737418240 bytes

255 heads, 63 sectors/track, 1305 cylinders, total 20971520 sectors

Units = sectors of 1 * 512 = 512 bytes

Disk identifier: 0x08f8d4cd

Device Boot Start End Blocks Id System

/dev/sda1 128 273104 136488+ 83 Linux

/dev/sda2 273105 20964824 10345860 83 Linux

Notice the "Start" is set to 128 for the first partition. This means that the start of that partition is 128 blocks from the beginning of the drive (and since blocks are 512 bytes, that's 65536 bytes). Now, the reason to align stuff is that your host operating system filesystem will have a certain block size (I use 4k blocks), and the idea is that (if I understand correctly) when your guest vm os tries to read blocks from it's virtual (vmdk) drive, the block(s) requested have to be translated to the host os so the data can be read from disk. Aligning the data in your guest filesystem to blocks on your host filesystem means that when a request for a block from the guest is made, the host just needs to read a single block as well - not 2 blocks (and stitch them together and return the data requested from the 2 blocks).

So, since in my case my host filesystem uses 4k blocks, I want the start of data in my guest to be multiples of 4k. 65536 bytes is 16 4k blocks, so the start of data would fall on a 4k block boundary and be aligned. That's the thought anyways... Now, if we actually count from zero when doing this stuff, then I probably want the start to be at 127... but then why not keep everything normal and start it at 63 like it is by default? In that case, keeping things as-is starting at sector 63 would give us a starting point of 32768 bytes (64 * 512 bytes) which is also going to have things fall on a 4k boundary (we've just got 8 4k blocks padding now instead of 16).

Another issue is, when dealing with smaller vmdk disks in a guest, you can get errors like this one when you try fiddling with the starting block:

Disk /dev/sdc: 2147 MB, 2147483648 bytes

255 heads, 63 sectors/track, 261 cylinders, total 4194304 sectors

Units = sectors of 1 * 512 = 512 bytes

Disk identifier: 0x6c0f11ec

Device Boot Start End Blocks Id System

/dev/sdc1 128 4192964 2096418+ 82 Linux swap / Solaris

Partition 1 has different physical/logical beginnings (non-Linux?):

phys=(0, 1, 1) logical=(0, 2, 3)

That drive is just 2Gb and I'm guessing that LBA addressing or something isn't being used in this case so that's why there's this message. The partition seems to behave normally (it's swap though, so can't be too sure) but I am worried by that message.

So, my questions in this post are a few...

1) Should we be aligning our guest partitions to block sizes used in our host filesystem?

2) If so, and in my case with 4k blocks, would keeping the standard start of 63 be fine (as in, do we count from 0)?

3) Should we worry with messages like above where the physical and logical beginnings are mentioned to be different? I would much prefer if I didn't have this message show up.

Also, a side note somewhat... I discovered the "c" setting in fdisk... it toggles DOS Compatibility - whatever that is. By disabling it (you have to hit "c" every time you start fdisk - before you do anything else) and partitioning a disk, the starting of the first partition is set to "1", not "63". I haven't seen if this works with a disk that will also have a boot loader installed in the MBR, but my next question is - if we should align disks... should we perhaps use this option instead? Does using this option set the start of the partition data at zero, or is it actually setting it at block 1 (so 512 bytes from the start of the drive) - or even block 2 (if we count from 0 - this 1024 bytes offset)?

Sorry if this is a lot of questions, but this is something that is very obscure in the "vmware world" and I'm trying to get a firm grasp as to when and if this practice should ever be applied today (may have been relevant in the past - but is it still?) or not. My whole reason for investigating this is that I have been trying to optimize my guest vm's as much as possible when it comes to disk i/o, as I have found that seems to be the biggest issue on my systems right now.

Thanks in advance!

Tags (2)
0 Kudos
7 Replies
RParker
Immortal
Immortal

The reason why the SAN is more important is because it's central to your VM's. The more VM's the more I/O. So it's more important that you align the SAN. Local disks aren't important, and really SAN alignment is REALLY noticeable ONLY under VERY HEAVY load (lots and lots of IO). We aligned our disks (templates). But I did a before and after, saw ZERO difference in performance, but hey at least they are aligned now for whatever it's worth.

The SAN has 20 or 30 disks, so alignment could play an integral part in ensuring you don't lose IO (most things like SQL / Oracle databases) on IO intense VM's. Local disks won't see this , since there aren't many affected VM's (less than 10 usually on local). So that's why alignment is basically unneeded on local.

Also the disk offset is Windows OS. Normally you see an offset around 32512 or 32256 something like that, it should be 32768 (equally divisible by 4k) which is the next starting offset, since Windows uses most of the rest for OS / File system stuff.

You can check it with sysinfo32 -> windows -> devices -> disks. That's how you know if you are truly aligned.

0 Kudos
esand
Contributor
Contributor

Yes, so I've gathered from what I've read - though that doesn't mean it shouldn't be done, since not doing it only invites the possibility of it being a problem at some point in time.

As I mentioned, the reason I even looked in to this was that I was/am experiencing high i/o wait times specifically with guest disk access, so I am trying to do everything I possibly can to eliminate the problem, or at least narrow the problem down to some specific portion of what would cause high disk i/o wait times.

Another tidbit of info is that I recently (after my post) read someone mentioning that the vmdk files themselves might be auto-aligned which could make sense since I believe the vmdk files contain vmware data in them before the actual virtual disk data. I'm not sure that auto-alignment could be achieved though unless they assumed a 63 sector, 512 byte per sector offset before the first partition (which could backfire in several cases if they did assume this). Also, is it aligned to the host filesystem? This too could be an issue since I do my development/creation work in Windows Vista x64 with VMWare Workstation 6.5, but I move the guest oses to a 32bit Linux system running VMWare Server 2.x (I don't have 64bit guest oses - just mentioning the host os). I'm pretty certain that the block size used on my Windows system is much bigger than 4k as it is on the Linux system... so any auto-alignment done on the vmdks (if there is any) would be completely wrong for the Linux server I put them on.

Also, this seems to be an issue since back in 2006. We're now almost in 2009, and I would have figured that an issue such as this (which is so obscure and low-level that MANY people would never think of it, and it requires a complete reformat of everything to cure) would have long since been addressed in some way or another - such as having alignment settings present in the disk creation portion of setting up a guest vm so that you can ensure it's aligned to the host properly. I'm pretty amazed that this hasn't happened after 3 years of product development.

0 Kudos
jeremypage
Enthusiast
Enthusiast

The actual alignment you want varies by vendor but generally you want it to be divisible by 32 or 64. 63 is not the 64th block, at least from an alignment standpoint.

0 Kudos
msemon1
Expert
Expert

Also I think it was VMware which said that alignment did not matter with RDM's. Can anyone confirm that?

Mike

0 Kudos
msemon1
Expert
Expert

Also I think this only applies to Linux and Windows 2000/2003. My understanding is that alignment is unnecessary for Windows 2008.

0 Kudos
msemon1
Expert
Expert

Somewhere I read:

Aligning the boot disk in the virtual machine is neither recommended nor required.

Align only the data disks.

Can you confirm?

Sorry. Did not mean to hijack this thread.

Mike

0 Kudos
RUG201110141
Enthusiast
Enthusiast

0 Kudos