VMware Cloud Community
Brad_Borgald
Contributor
Contributor

2TB limit on Dell PERC6?

I'm evaluating ESX3i on a pair of my 2950's. One has a PERC5 and 5x146GB drives and the other has a PERC6 with 5x750GB drives. I'm running them in raid10. The dell with the PERC5 works fine, but the one with the PERC6 tells me that I do not have any persisitent storage. It detects the scsi controller as a LSI Logic MegaRAID SAS1078R, and it sees it as a 2.05TB disc, but if I try and create a datastore on it (the install on the PERC5 created the datastore automatically), it says that I only have 45GB free. AKA, it's like it wrapped at 2TB so I have that .05TB free that I can create a partition with.

Has anyone else installed on a PERC6 with an array over 2TB?

Tags (3)
Reply
0 Kudos
18 Replies
Dave_Mishchenko
Immortal
Immortal

It's a bug with 3i. I can confirm that on different hardware that ESXi ignores the first 2 TB. In my case, fdisk run at the console shows a 800 GB disk instead of 2.8 TB. You would be OK if you installed ESX 3.5. With both ESX 3.5 and ESXi you are limited to a 2 TB vmfs partition so in your case you'd have a bit of unusable space if you went with ESX 3.5. Given that you ended up with 2.05 TB, did you actually create a RAID 5 with a hotspare? RAID 10 with the 5 drives should give you 1.5 TB with a hotspare.

Reply
0 Kudos
Dave_Mishchenko
Immortal
Immortal

As another option, if the PERC6 supports it, you could create 2 arrays on the set of 5 x 750 GB drives. I tried it tonight on 3 * 1 TB creating first a 2047 GB array (RAID 5) and then another RAID 5 with the remaining space on the 3 drives. ESX 3i was able to see and use both arrays.

Reply
0 Kudos
bssitton_dell
Contributor
Contributor

I work for Dell.

The 2TB limit is a limitation of ESX, and ESXi.

If you are using simple raid types (0,1,5,6) then you can create multiple virtual disks on each raid group.

1) Press <ctrl> R during the boot process to enter the PERC configuration utility.

2) Create a new virtual disk. After you have selected your RAID level (must be 0, 1, 5, or 6), and your physical disks; enter 2097152 or smaller for the VD Size in MB. When you proceed to the VD name, the utility will automatically subtract 1 from the VD size.

3) Finish creating the new virtual disk.

4) From the main screen, expand "space allocation" and select "Free Space".

5) Press <enter> to create a new virtual disk, and enter a disk size less than or equal to 2097152 for the 2nd virtual disk.

6) You may repeat steps 4 and 5 to create up to 16 virtual disks for each Disk Group.

The other option, which someone else posted, is to create multiple raid groups. This also works.

Reply
0 Kudos
weinstein5
Immortal
Immortal

2 TB more specifically is a limitation of the VMFS-3 file system - a single VMFS-3 extent maximum size is 2 TB - to make a larger VMFS-3 datastore you will need to add addiitonal extents -

If you find this or any other answer useful please consider awarding points by marking the answer correct or helpful
Reply
0 Kudos
stvkpln
Virtuoso
Virtuoso

2 TB more specifically is a limitation of the VMFS-3 file system - a single VMFS-3 extent maximum size is 2 TB - to make a larger VMFS-3 datastore you will need to add addiitonal extents -

Nope, that's very inaccurate. > 2TB is not currently supported on ESX, whether it be a LUN formatted for VMFS or presented to be an RDM. The VMkernel doesn't deal with anything > 2TB, period.

-Steve
Reply
0 Kudos
weinstein5
Immortal
Immortal

Where are you getting your information - according to config maximums from vmware.com lists largest LUN Size and VMFS VOlume size as 2 TB - and by having 32 extents you can have a VMFS datastore of 64 TB (don't know why you would do this though) - check out

If you find this or any other answer useful please consider awarding points by marking the answer correct or helpful
Reply
0 Kudos
stvkpln
Virtuoso
Virtuoso

Where are you getting your information - according to config maximums from vmware.com lists largest LUN Size and VMFS VOlume size as 2 TB - and by having 32 extents you can have a VMFS datastore of 64 TB (don't know why you would do this though) - check out

I'm not entirely sure how what I said was inaccurate? 2TB is the max for an individual LUN. This also applies to any LUN's you present for use as RDM. It has nothing to do with VMFS, as far as I know. It's a limit within the VMkernel. If you don't believe me, build a LUN that's < 2TB and present it to your ESX host and see if a VM can use it. Every time I've tried to do it -- and I've had a need to do it -- it hasn't worked.

-Steve
Reply
0 Kudos
Brad_Borgald
Contributor
Contributor

I consider the 2TB limit in VMWare's product a serious limitation that they need to address. With the growth in size of arrays they are going to have alot of other customers with complaints as well. I ended up going with 2 raid5 arrays, I would have prefered to stay with raid10, but the PERC6 doesn't allow you to set the total array size like it does with the simple (raid 0-5) types.

Thanks for all the answers, espcially from bssitton_dell as I didn't realize that I could arbitralily set the array size for simple array types.

Brad

Reply
0 Kudos
bssitton_dell
Contributor
Contributor

If you read the configuration maximums document carefully, vmfs-2 has a volume size of 64TB (not lun, extent, or virtual disk size), however, the volume size for vmfs-3 is 2TB. I don't know the reason behind the change. Either way the lun size limit is 2TB.

Reply
0 Kudos
Henk_Nachtegaal
Contributor
Contributor

If I follow these steps I am not abel to expand space allocation. And I use the perc6/E with a MD1000. What am i doing wrong?

Regards

Henk

Reply
0 Kudos
bssitton_dell
Contributor
Contributor

Can you explain more about what you mean by "Expand Space Allocation"? I'm not sure I understand.

Thanks,

Brian Sitton

Dell, Inc.

Reply
0 Kudos
Henk_Nachtegaal
Contributor
Contributor

The 2TB limit is a limitation of ESX, and ESXi.

If you are using simple raid types (0,1,5,6) then you can create multiple virtual disks on each raid group.

1) Press <ctrl> R during the boot process to enter the PERC configuration utility.

2) Create a new virtual disk. After you have selected your RAID level (must be 0, 1, 5, or 6), and your physical disks; enter 2097152 or smaller for the VD Size in MB. When you proceed to the VD name, the utility will automatically subtract 1 from the VD size.

3) Finish creating the new virtual disk.

4) From the main screen, expand "space allocation" and select "Free Space".

5) Press <enter> to create a new virtual disk, and enter a disk size less than or equal to 2097152 for the 2nd virtual disk.

6) You may repeat steps 4 and 5 to create up to 16 virtual disks for each Disk Group.

I mean step 4. There is nothing to expand here

regards

Henk

Reply
0 Kudos
Henk_Nachtegaal
Contributor
Contributor

Foud It. When you are at point 4 press the right arrow on youy keyboard and the you can make your virtual Disks

regards

henk

Reply
0 Kudos
Henk_Nachtegaal
Contributor
Contributor

Okay...Next challange. How do I create a disk that I can access from a guest running windows 2003 RC2 that is as big as 2TB. I can see all the drives i created now in VMWare but I only can create a virtualDisk with a max. size of 256GB. I need at lease 1.5TB of data storage. How can I create this??

Regards

Henk

Reply
0 Kudos
weinstein5
Immortal
Immortal

By default the maximum file size allowed is 256 GB - I can not remember if this is your first disk but either way you will need to remove the VMFS datastore and readd it making sure you choose the mazimum file size to its max of 2TB with block size of 8 MB

If you find this or any other answer useful please consider awarding points by marking the answer correct or helpful
Reply
0 Kudos
Dave_Mishchenko
Immortal
Immortal

You'll need to delete the datastore and then recreate it. When you create it, you'll have the option to set the block size which will determine the maximum size of the VMDK file that you can create. You'll have the following choices

Block size - VMDK size

1 MB - 256 GB

2 MB - 512 GB

4 MB - 1024 GB

8 MB - 2048 GB

It shouldn't be an issue with a 1.5 TB VMDK and a 2 TB lun, but in general you'll need to leave some free space on the datastore of log file, swap files, snap shot files, etc. You wouldn't want to create a 1.5 TB VMDK file on a 1.5 TB datastore. However, for that size of VMDK file you could consider using a raw device mapping (RDM) to present the entire LUN to the VM to use.

Reply
0 Kudos
Henk_Nachtegaal
Contributor
Contributor

Thanks for the quick reply....How do I create a RDM? If I go to the VM and add a harddisk the option for add Raw Device Mapping is grayed out! So how can I add them?

TIA

HEnk

Reply
0 Kudos
Dave_Mishchenko
Immortal
Immortal

Which local storage, you can't use a RDM (you would need FC / iSCSI SAN), but you could use a generic SCSI device - http://communities.vmware.com/message/966401#966401.

Reply
0 Kudos