Jon Munday wrote:
We have a fair amount of legacy Windows 2003 servers with misaligned disks in our environment and I would like to resolve this to ensure that we are getting the best out of our storage, and not sufferring unnecessary performance degredation with something that can easily be avoided.
I did some testing to see how much of a difference IO alignment makes, and since the results are pretty significant, I would like to deal with this ASAP.
See results below;
So my questions are;
- What is the best way to programatically go through all of the VM's and check for disk alignment - I don't want any manual process (if possible).
- What are the options for aligning the misaligned disks?
- What is the impact of each option (ie. How long does it take, Is Downtime required, etc.)
- What are the limitations and / or known issues in doing such remediation tasks.
Thanks in advance,
1 best way to check woud either be locally like diskpart, or use a tool that your san has.
Im sure you might be able to script something with the msinfo data
How to check Drive alignment in windows:
- run “System Information” (msinfo32.exe in Start-> Run)
- and select Components .. Storage .. Disks
- scroll to the bottom and you will see the Partition Starting Offset information.
- This number needs to be perfectly divisible by 4096.
- The default .vmdk, you will see the Partition Starting Offset set to 32,256 (32,256 / 4096 = 7.875)
- and thus this file system is not correctly aligned.
2: options are v2v using vmware converter, or valigner.
vmware converter makes a new vm,
vlaigner converts teh actual disk.
The problem with valigner is that if you ever extended the os or have 2 more more patitions on a disk it wont work
since its really a disk conversion byte by byte they all take the same time as a clone
Hey Matt, thanks for this ... it look like exactly what I'm after
I'll test it out and provide feedback.
1 person found this helpful
Hope it helps. If you have questions, its author (Nick Weaver) is a friend and quite active on twitter as well (@lynxbat).
1 person found this helpful
For a quick check of starting partition offset values from Windows, I created a Powershell script that queries vCenter for the name of all Windows VM's, then uses WMI to query these servers for their starting offset values. I used 64KB as my multiplier - it flagged it as not correctly aligned if not divisible by 64KB.
Another tool you can use for correcting partition misalignment is VMware Converter Standalone (must at least 5.x version). You can do a V2V to the same vCenter, and copy to the "destination" VM using volume based copy - there is an option to create an optimized layout which will align everything at 1024KB. This also has the advantage of being able to resize your disks if you are too much or too little space allocated to your windows drives, and you can move from a single vmdk with multiple partitions to multiple vmdk's, and vice versa, if you like.
Whichever option you go with, you will incur downtime as the VM will have to be offline.
Thanks very much, this will save me writing a script - I will test this tomorrow.
The bottom line is EACH VM has to be done, it's going to take a LONG time and its very tedious.. extended downtime PER VM.
So you can probably do 2 VM's per LUN (on Netapp) and maybe 4 concurrently on your SAN, but even with the script it copies the information in the VM to a new disk, and it takes around 1 minute per GB (estimated).
So take that into account. I have done many of these... it's not fun, the best thing for you is to make sure the templates (all of them) are done first, that way ALL new VM's are deployed properly, and existing VM's you can tackle as needed.
I would start with Databases first, since they have the highest IO. The high IO is where you will see the most dramatic affect. Linux is higher priority than Windows, because Linux kills in poorly aligned VM's, but if you have Windows, some of that is diminished by caching... also do NOT underesestimate defrag the guests either. Since they are not aligned, defrag will be even more crucial, but it will still be a factor once they are aligned.
This topic has been discussed to the nth degree, so I will not get into it, but since you are aligning disks, don't discount the important of a good defrag while you are at it.
Since each VM needs to be off.. not in snapshot mode, will not work... you should:
1) Run registry scan / fix on VM's
2) update / patch VM's as needed (including tools upgrade during this cycle)
3) shutdown the VM after steps 1 and 2
4) perform alignment on VM disks
5) defrag each disk / each VM.
Just my suggestion, but that will give you best performance.
Thanks for the tips, I'll definitely factor this in. My first step is to see how many VM's are affected out of the ~650 windows servers and ~250 unix servers and then start planning to do the tedious work.
... I do have 20TB of Flash storage that I can abuse for this job, so can hopefully crunch through it
Rparker, why is defragging needed/recommended, especially when its a vmdk on a san? From what I've read usually a defrag isnt needed.
"Typically you are going to have multiple VMs running together on a VMFS or NFS volume. Therefore the overall I/O to the underlying LUN is going to be random so defragmenting individual Guest OS'es is not really going to help performance." -chogan
NetApp doesn't recommend defragging the guest at all, because their WAFL feature supposedly places each block in the most optimal location, and defrag can interfere with that.
Also, in general you have to be careful with defrag when you are working with thin provisioning, because you will have to provision more disk in order to execute the defrag. Other than that, I don't know of any other reason why you wouldn't defrag if Windows is reporting excessive fragmentation. Defrag and the other tips that RParker provided should improve performance, biggest impact being when higher-IO vm's are corrected, of course.
doesn't the vmware converter create a vm and so a new nic, which has a new mac address.
Yes. It's a V2V so it would be a new VM object with a unique MAC address. It tries to keep the same network configuration (static IP address, etc) but is not always successful in doing so. Usually if you're chaging hardware version or type for virtual NIC being used, from what I have encountered.
You can change the MAC address back to the original MAC address if you have software that is sensitive to that (FlexNET licensing is one example).
Actually, it can be done more efficiently than NetApp's method:
NetApp has 2 methods. You can use their mbralign utility which does work but requires the VM to be down. It does not support any GPT disks so you have to watch for that. Since all of our storage is NFS, we spun up a guest on the same jumbo-frame network as our datastores, mounted the datastores, and aligned from there. It's not quick and requires enough free space in the datastore as the size of the VMs you're aligning - it basically reads from one file and creates a new one. It's not quick but it's cheap and it almost always works.
The 2nd method is in the VSC and only works if you're using iSCSI disk (unless the most recent version addressed that). What it does is create a purposely misaligned datastore which reversed the offset of the VMDKs. This should be only used as a last resort since it's guaranteed to bite you later on. If you put a Windows 2008 VM on this datastore - and they're aligned by default - you'll end up with split I/Os because the datastore is misaligned.
okay it was some ago I tried with vconverter 5.0 beta , so it seems still no change. Maybe in the next verson of converter.
Here ar some informations about netapp alignment.
1. Top 10 session vmworld 2012
2.) Netapp VSC Online Alignment versus Realigment
Instead of realigning the virtual machine, Netapp is creating a new datastore specifically built to align itself with the unaligned virtual machine. These new datastores are considered “Functionally aligned” because they make the vm appear to be aligned.
This is not an alignment of the ntfs partition.
3.) Keep in mind to take a backup of your vm before you hardcore realign the ntfs partitions. Microsoft does not give support.
Realignment needs downtime.
4.) If you buy a new storage maybe look at modern storage which are eventually not affected by this.
6. Orchstrator session
2049 How OCLC Used vCenter Orchestrator to Automate the Disk Alignment of 1500 + Virtual Machines