New RAID5 array with 5x2TB

[thanatica]

I'm about to configure my ESXi server with a new RAID5 array that contains 5 disks of 2TB each. I need the storage.

I've read my stories about larger-than-2TB-configurations and how ESXi doesn't want more than 2TB no matter how hard you pursuade it to.

So, once and for all, what is THE way to go?

I was thinking this:

1) Create the array as usual on the controller

2) Make the controller chop the array back into 5 virtual disks

3) Add all 5 virtual disks as separate LUNs into ESXi

4) Create 5 separate VM storages for the guest OS

5) In the guest OS, join them back together with a software-RAID0.

But there is SO MUCH overhead in this... There must be a "good" way to go here. But I don't want a NAS, because that's what my VM is supposed to be in the first place (among other things).

Extends then? Dunno how they work, but I've read that people think the world will end if you use them. So maybe not a good option.

Please help

[DSTAVERT]

I would suggest that if you need a NAS create a NAS. There isn't a reason to make it virtual. Just because it is possible to virtualize things doesn't mean everything should be virtual. Very large VMDKs are not necessarily easy to deal with. They become exceedingly dificult to move especially when they are globbed together using extents or virtual RAID disks.

[mcowger]

Given that everything is on the same RAID set anyways, extents isn't a bad way to go.  It will get you the larger VMFS store without intriducing any real additional risk.

However, it still wont allow you to present a larger-than-2TB VMDK to a VM.  You'll still need to use LVM inside the VM to combine all the VMDKs into one larger disk.

[thanatica]

Ok great, so how do I add those extents? I tried this:

1) Add the volume with default option, which says a capacity of some 7.5TB. Resulting datastore is only 1.28TB. An "add extent" or similar option is nowhere to be found.

2) Add the volume with a maximum capacity of exactly 2000GB, which should be under the allowed maximum of 2TB-512B. Works. But then, still no "add extent" option anywhere.

In the datastore properties, there's a "increase" button, but I'm being bombarded with cryptic errors if I enter anything above 2TB. Entering anything below 2TB, like "another" 2000GB, doesn't actually do anything at all, let alone add an extent.

So it boils down tothis: how to add extents?

Google doesn't seem to know either...

--

Friendly note: a simple implementation of GPT partitioning by the ESXi team @ VMware would solve most of these rediculous problems. 3TB disks do exist, so more users are bound to face issues like these.

[DSTAVERT]

The documantation contains all the answers. Make sure that you break the array into chunks smaller than 2TB (be safe with 1.99) Exactly doesn't work. http://pubs.vmware.com/vsphere-esxi-4-1-embedded/wwhelp/wwhimpl/js/html/wwhelp.htm#href=server_confi...

[thanatica]

Aha, so I still need the carving function that's in my RAID controller... Good thing I have that then.

But in that case, I don't really see a functional difference between 4 files on 4 LUNs, and 4 files on 1 LUN-with-3-extents...

Strange, all this. Moreover because back in the day when I used Hyper-V, none of this would have been a problem, since Windows (Hyper-V) supports single volumes well over 2TB (wasn't it something like 16PB? Immensily large at least) and single files also well over 2TB (also somewhere in the exabyte range) for the virtual disks.

Plus, in Windows, it's very esy to install the drivers for the RAID card. No fiddling with obscure commands and console outputs. Also I could install the management software that comes with it.

I hope issues like these are going to be addressed, because after all, we do live in a world of increasing storage capacity demands.   

[mcowger]

Exactly - theres no real difference between the 2 methods, although multiple VMDKs on 1 large VMFS seems mildly easier to me.

Hyper-V does have an advantage here (although your comment about GPT is off base - the limit isn't GPT, its the underlying LVM and Read(10) commands being used.  I suspect you will see this limit corrected soon enough.

As far as the drivers, you are only encountering this because you are using very uncommon (for ESX deployments) RAID card - 90% of the supported cards don't require extra drivers at all.

[pgoggins]

Depending on how many VM's are accessing the system you could also create one large array and then use RDM to map the 7.5TB+ to a single VM. Can't vmotion it but if it's a stand-alone fileserver it would work.