Datastore/disk alignment,

[FrostyatCBM]

Just in the process of creating some datastores on a new SAN. Am using the latest VI Client v4.1 and I have read elsewhere that if I create my datastores this way (or through vCenter) then things will be aligned automatically for me. My confusion arises because I also read that the alignment will be at the 64K B boundary. But when I look at my virtual disks on the SAN, I see something like this:

Virtual Disk name: md1200r10vd1
Virtual Disk status: Optimal
Capacity: 1,894.400 GB
Segment size: 128 KB
Capacity reserved for future segment size changes: Yes
Maximum future segment size: 2,048 KB

So if the VI Client is creating things aligned at 64KB, and if the block size is say 1MB, then doesn't this mean that everything will be misaligned?

Or is the VI Client smart enough to read the segment size of 128KB from the SAN and then align at 128KB?

Should I be overriding this and forcing alignment at 1MB instead?

[rickardnobel]

Or is the VI Client smart enough to read the segment size of 128KB from the SAN and then align at 128KB?

I do not think the vSphere Client has any possibility to extract the stripe size from the SAN and it will most likely be aligned at 64KB.

You could try to create a test VMFS store and then check the alignment with the following command from ESX/ESXi:

fdisk -lu

The output will be something like this, unfortunaly hard to read directly as it is not clear how the labels match.

Device Boot Start End Blocks Id System

/dev/disks/naa.600c0ff000d78ae351ebf54a01000000p1 128 19 518 974 9 759 423+ fb VMFS

But here:

Device /dev/disks/naa.600c0ff000d78ae351ebf54a01000000p1

Boot (empty, not bootable)

Start 128 ( x 512 bytes sector size = 64 KB)

End 19 518 974 (ends at this sector. x 512 = ~10 GB for this test partition)

Blocks 9 759 423

Id fb (partition type=vmfs)

System VMFS

[jcwuerfl]

Generally if your using the VMware client your golden when creating VMFS datastores or so keep hearing.

However,

64+64=128

So basically isn't VMware putting 2 into every 1 bock of the san so its still one Read request to the san? The issue with misaligned was more that it had to do 2 Read requests to access 1 block of data ? So if you had a 256k block then it would put those on 2 blocks on your san so then that would be 2 Read requests vs. 1 if they were aligned?

[FrostyatCBM]

I created some VMFS datastores and then activated tech support mode and logged in to run the 'fdisk' as suggested. Here's a subset of the output:

-

---

~ # fdisk -lu

Disk /dev/disks/naa.6842b2b000462bf8000002743b032d06: 1649.2 GB, 1649267441664 bytes

255 heads, 63 sectors/track, 200512 cylinders, total 3221225472 sectors

Units = sectors of 1 * 512 = 512 bytes

Device Boot Start End Blocks Id System

/dev/disks/naa.6842b2b000462bf8000002743b032d06p1 128 3221225279 1610612576 fb VMFS

Disk /dev/disks/naa.6842b2b000462bf80000026e3b032bbe: 858.9 GB, 858993459200 bytes

255 heads, 63 sectors/track, 104433 cylinders, total 1677721600 sectors

Units = sectors of 1 * 512 = 512 bytes

Device Boot Start End Blocks Id System

/dev/disks/naa.6842b2b000462bf80000026e3b032bbep1 128 1677716144 838858008+ fb VMFS

Disk /dev/disks/naa.6842b2b0004665ba0000026d4ca119cf: 2034.0 GB, 2034096510976 bytes

255 heads, 63 sectors/track, 247298 cylinders, total 3972844748 sectors

Units = sectors of 1 * 512 = 512 bytes

Device Boot Start End Blocks Id System

/dev/disks/naa.6842b2b0004665ba0000026d4ca119cfp1 128 3972842369 1986421121 fb VMFS

-

---

From that I can see that all my partitions start at 128, which when multiplied by the sector size of 512 bytes, gives me a start at 64KB as anticipated.

Thinking about this ... if things start at 64KB and my VMFS filesystem is built using 1MB blocks, then the first block will start at 64KB and finish at 64KB1MB ... whereas the SAN stripes start at 0KB and increment in 128KB blocks, thus arriving at 1MB and then 1MB128KB ... so my VMFS filesystem will be smack bang across a boundary.

Have I got that wrong?

Should I be ignoring the 1MB VMFS block size and only consider the 512 byte sector sizes? In which case, its all aligned okay.

I guess I'm still confused.

I have done alignment before with SQL database servers and so on, but that was all within Windows filesystems ... I have never tried to do this before with VMFS.

[rickardnobel · Accepted solution]

I think the block size is used mainly for allocating the disk space when then vmdk file is created (thick disk) and expanded (thin disk). The actual IO size will be the IO size from the guest operating system, which could be perhaps 8KB in average. So I think that the typical IO at the disk will still probably hit inside the stripe size of your SAN.

[FrostyatCBM]

Thanks mate ... that sounds right ... and seems to be confirmed by the following articles:

http://runningvm.wordpress.com/2010/08/17/vmfs-and-vm-guest-disk-alignment-puts-fingers-in-his-ears-...

http://www.yellow-bricks.com/2010/04/08/aligning-your-vms-virtual-harddisks/

I'll run with the current setup, as it seems to be performing in exemplary fashion ... I'm getting over 1100MB/sec throughput, whereas my old SAN is giving me about 60MB/sec-80MB/sec at best (perhaps the old SAN's disks are misaligned?! )

[rickardnobel]

'm getting over 1100MB/sec throughput,

That looks like very good numbers. What kind of SAN do you have, 8 Gbit FC or 10 Gbit iSCSI?

[FrostyatCBM]

Neither :smileyalert: ... its a Dell MD3200 Shared SAS with dual controllers for redundancy ... I only have 3 VMware hosts ... each controller has 4 ports so it suits our small environment. The cabling therefore doesn't require any switching layer, so I'm using Dell's 4x6Gbit/sec cables direct from the hosts to the storage. I'm still in very early stages of testing it, but so far it is blowing away our old iSCSI SAN as well as some servers with 3Gb/sec direct attached storage too. Very encouraging. Hopefully today I will be able to test it with some different workloads (mix of reads and writes of different sizes).