ESXi 5.0 does not recognize 4TB storage array

[btg4UK]

Hello all!

This is my setup:

HP ProLiant ML350 G5

• 2x XEON E5345 quadcore
• 12GB RAM
• integrated Smart Array E200i connected to single 250GB SATA disk
• Smart Array P400/512MB connected to two 2TB SATA disks in RAID0 (4TB logical array) (please no comments about RAID0, it's for testing only!)
• LSI 3041X-R PCI-X SAS Controller (nothing connected at the moment)
• vSphere 5.0 starting from USB flash drive)

The server is on the HCL for vSphere 5.

My problem:

In vSphere Client, the 250GB hard disk connected to the E200I controller is recognized correctly, and I can create a VMFS5 datastore without any problems. However, the 4TB array only shows up as a 512byte device, and I can't create a datastore on it.

I also deleted the 4TB RAID0 array and created two 2TB JBOD arrays (one per disk). Now the first one shows up as 1.82TB disk, but the second one as 0.00byte.

Just to add, that the setup worked fine with MS Hyper-V Server 2008 R2 which had no problems recognizing the 4TB RAID0 array out of the box.

So why is that? I thought that vSphere 5 was supposed to be finally able to handle LUNs >2TB, so shouldn't I be able to see the full capacity of both disks in vSphere Client andto create a datastore on it? And what can I do to rectify the situation?

BTW: does anyone know when the HP and Dell editions of vSphere 5 will be available for download?

Thanks,

Ben

[DSeaman]

I ran into a similar problem with the DL360 Gen8 and the P420i conroller with ESXi 5.0 U1. I created a volume that was 3.2TB, then tried to add it via the add storage wizard in the vSphere client. I got part way through the wizard (where it did see the full 3.2TB), but when it was trying to read the partition table the vSphere client hung. The only solution was to reboot the ESXi host, but I was still unable to add the LUN on subsequent tries, even though it saw the full LUN size. I had to delete the logical volume in the smart array software and create a 2TB and 1.2TB volumes, then all was well.

So it seems there's a >2TB issue, although a bit different, with the P420i controllers. 

[Kaptain1]

Hello,

I have a P400 controller on ProLiant ML350G5 server with ESXi 5.0u1 (hp image). How can I set-up RAID10? When i create a Logical Volume in the HP Controller BIOS, it adds all 4 hard-drives (2tb each) to the RAID1+0 , and shows total spac eof 4Tb. Of course, in ESXi 5.0, this logical volume just shows up as 512B, and I can't use it at all.

How can I have RAID10, but logical volume of 2Tb (so that ESXi 5) works? I don't mind loosing 2Tb of space, but I want RAID10 performance. The only option I see so far, is to just use Raid1, but I don't want that because that's what i had before, and it's slower...

Thank You

[a_p_]

I'm afraid you are out of luck with this. A lot of RAID controllers do not support to split RAID x0 levels into logical volumes. If you do have a 512MB BBWC kit attached to the controller you may not really see a huge difference in disk performance between RAID 10 and RAID 5 for normal operations with the 4 disks you've got.

André

[mervincm1]

I believe you can accomplish what you are looking at doing, you just can't do it with the build in bios version of the array configuration utility.

Find a spare SATA HDD, install windows 2008$2 server, Trial version will do, install the HP array driver and the graphical array config utility.

Using the Windows version of the array config tool, then create your 4x2TB RAID 10 array, and then the sub 2TB volumes you need for ESXi to be able to correctly see them.  At this point within ESXi, you can merge the pair of sub 2TB volumes to create a single sub4TB volume.

Yes It is a work around but it should get you going.

[a_p_]

As a quick note. You don't need to install any OS to run the array configuration utility, it is included on the bootable SmartStart CD/DVD. However, as mentioned earlier, I doubt the P400 supports splitting the RAID 10.

André

[mervincm1]

This works if you run the P400 in an HP server.  Some of us use them in a whitebox server (I run an ASUS server board) and smartstart doesn't work.

In anycase, I am 95% sure that (windows version of the ACU tool) is what I did to get my P400 running 4x2TB seagate barracuda LP in RAID 10 working, booting esxi 5 (no update 1) off a USB stick

[Josh26]

André Pett wrote:

As a quick note. You don't need to install any OS to run the array configuration utility, it is included on the bootable SmartStart CD/DVD. However, as mentioned earlier, I doubt the P400 supports splitting the RAID 10.

André

If you refer to producing two logical LUNs out of one large RAID array, it certainly does.

You just need to use the ACU from the SmartStart as mentioned earlier.

[Kaptain1]

Hi All,

An update:

I have downloaded P400 Offline Configuration Utility and burned it to CD, then created a USB (lol it won't let you create it from iso - error message),and it insists on doing sloooow full format on my 16 Gb usb stick, so 1 hour later, i had a bootable USB stick with HP P400 Offline Config Utility.

I put 4 x 2Tb hard-drives into Raid 1+0 (it showed 4Tb of usable space), then i deleted the logical volume (from the same Offline Array utility) and created 1 x 1.9Tb logical volume and 1 x 1.7Tb logical volume. I also re-installed ESXi 5.0 with HP's newest custom image (which still has bug of not recognizing over 4Tb btw), but it did regonize my 2 logical partitions of <2Tb. So... it works - sort of.

Now, the issue i'm having is that with my P400 and 4 x 2Tb WD RE4 64mbCache 7200 rpm hard-drives in Raid 1+0 (Raid10?), I'm getting VERY bad speeds from hdtune!

I did speedtest, and some of my Win2k8r2 servers can't get over 40Mbps speeds via HDtune, and some fresher Win7 and Win2k8r2 are getting about 80Mbps speeds for a few seconds, but THEN, the hdtune test jumps to something around 700Mbps+ and finishes last 3/4 of the test with steady 700Mbps - i think this result is obviously incorrect (or maybe it has to do something with P400 512mb cache?). Before, i had 2 x 2Tb hard-drives in Raid1 and i was getting 100Mbps speeds (higher than now in Raid10).

Acceleration and Cache are all enabled in the Controller settings. I have tried 64k stripe size and 128k, 75%read and 25%write settings on the controller, and now will try with 75%write and 25%read and report back. But it looks very discuraging and I'm not sure what else can be done - i've tried everything already!

How can i get better speeds from my Raid10? I'd think that hard-drives that i got are pretty fast Enterprise Grade hard-drives with 64Mb of cache - should be pretty fast! Maybe the P400 raid controller sucks? I

Any help is greatly appreciated!

[Kaptain1]

Hello,

Well... I've changed the strip size to 128kb and changed Performance Settings on the P400 Array to 75%Write and 25%Read, but i'm still getting same slow speeds via the HDtune - around 40MBps which is very discouraging.

What do I do next? Since i've already tried everything (i belive), where's the bottleneck? Is it the P400 controller?

Thank You

[Josh26]

Kaptain1 wrote:

What do I do next? Since i've already tried everything (i belive), where's the bottleneck? Is it the P400 controller?

Thank You

The obvious question is "do you have the BBWC", or the newer "FBWC"?

[Kaptain1]

Hello,

I have the BBWC.    411064-B21 HP Smart Array P400 512MB   I bought it for as "bulk" for about $150 off DEC Trader...

What are my options to get full speeds of my 4 x 2Tb RE4 hard-drives? New Controller? If so, please recommend one which will surely give me the speeds.

Thank you!

[Kaptain1]

*Bump*

Hi All,

Could you please reply to my previous post... I'm pretty desparate here. I spend $1k on 4 x 2Tb WD RE4 hard-drives and thought that Raid10 would give me much better performance than Raid1, but instead I got 40MB/s speeds now which doesn't make any sense to me.

Shouldn't I be getting something close to 200-300MB/s speeds with these hard-drives?

What can I do to fix this situation? If I need to buy a new hard-drive controller, which one would you recommend?

Thanks in advance!

[Zhuominchen]

Raid 1+0 is double-raiding vs (raid 1 or raid 0) is single raiding. 

Your controller is to be blamed.  

Example:  I had LSI raid controller with 20 X 2 TB disk (6GB/s interface), flat Raid 6 (with biggest blocksize 1 MB)   on supermicro Server.

Result :  730 MB/s  both read/write, that is near 6 GB/s.

[Kaptain1]

Thanks. Which is a good Raid controller would you recommend for Raid10? I'll buy a new one.

[Zhuominchen]

If your motherboard has PCI (X16/X8) , try this one (the smaller PCI plug, the faster it is)

http://www.amazon.com/dp/B0012VL3K2/ref=asc_df_B0012VL3K22038224?smid=ATVPDKIKX0DER&tag=hyprod-20&li...

[Kaptain1]

Thank You.

Yes, it supports the smaller PCI Express plug. Do you know of any "higher-end" controllers with backup battery? And maybe more cache? My goal is to make it as fast as my WD RE4 hard-drives can go in Raid10

I tried doing some research online, but can't find any definite recommendations. Any help is greatly appreciated!

Thank You

[Zhuominchen]

LSI MegaRaid is hign-end product

http://www.amazon.com/dp/B002IT4YH6/ref=asc_df_B002IT4YH62038224?smid=ATVPDKIKX0DER&tag=hyprod-20&li...

http://www.amazon.com/LSI-Logic-MEGARAID-320-2-64MB-3202064/dp/B00008PGJN

4-disks wont get you very far. Single Disk has about 120 MB/s max, you get 500 MB/s at best.

So you may want to look for 6-port/8-port Raid Controller.

Jimmy

[Kaptain1]

Does anyone know a good HD controller for Raid 10... I'm looking at a few HP's new controllers, and they look fine, but did anyone actually had any real-world experience with those? Any fw bugs?

Anyone know of a good controller for ESXi which will support more than 4 HD's and have a good amount of cache with backup battery?

Thank You

[Zhuominchen]

This is whole spec of the system I had (20 X 2 TB disks with 730 MB/s throughput)

RAID-LSI-9260-4i LSI Logic MegaRAID 9260-4i 4-Port SAS 6Gb/s Internal RAID Controller

is the high-performance RAID controller with battery backup option.

ixsystems.com  is OEM but motherboard is built by Supermicro.

Jimmy

-----------------------------------------------------------

iX4236-847E16 4U Orion2 Dual Xeon Rackmount Server - 36 Bay Hot-Swap

SAS/SATA Drive Bays - (24 Drives in front + 12 Drives in the rear of

the chassis) - SAS2 6Gb/s - 1400W Redundant Power Supply (1+1)

- Quick Release Rackmount Rail Kit

4 ea

MB-SM-X8DTI-F Dual Socket Xeon Motherboard (Intel Tylersburg 5520 Chipset) - 12

DIMM Slots (192GB Max Memory) - SATAII + Dual Gigabit Intel

82576 LAN, (4) PCI-E + (2) PCI Expansion Slots + IPMI Dedicated

LAN

4 ea

CPU-INT-XE-2.40-12M-WM Intel Quad Core 2.40Ghz Xeon (Westmere) 12MB cache 5.86GT/s

QPI (80W) (E5620)

4 ea

HEAT-SM-SNK-P0038P Heatsink: Multi Core i7 Passive Copper XEON 2U Heatsink 4 ea

MEM-AP-4GB-D3-1333-ECC/REG 4GB DDR3 1333Mhz ECC/REG 256x8 (12GB Per System) 12 ea

HD-WD-SATA2-2TB-7200/64 Western Digital 2TB Enterprise (RAID Duty) SATA2 7200RPM

64MB Cache RE4 (36 Per Sys)

144 ea

RAID-LSI-9260-4i LSI Logic MegaRAID 9260-4i 4-Port SAS 6Gb/s Internal RAID Controller

4 ea

RAID-LSI-BBU07 LSI Logic MegaRAID Intelligent Battery Back-up Unit For SAS

8880EM2, 9260, 9261, 9280

4 ea

NIC-INTEL-10G-X520-T2-DP Intel 82599EB 10 Gigabit Ethernet Controller, Dual RJ-45 Ports -

E10G42BT

[btg4UK]

Kaptain1 wrote:

Does anyone know a good HD controller for Raid 10... I'm looking at a few HP's new controllers, and they look fine, but did anyone actually had any real-world experience with those? Any fw bugs?

Anyone know of a good controller for ESXi which will support more than 4 HD's and have a good amount of cache with backup battery?

The newer Smart Arrays (P212, P410, P420 etc) all seem to work fine with ESXi. Only the 1st gen SAS controllers (E200, P400, P600, P800) are suffering from a buggy ESXi driver, and considering that they have been replaced by two generations already it is unlikely that they will be fixed.

Generally, the Smart Arrays are most certainly some of the best controllers you can get. If you run ESXi then just avoid the 1st gen ones if your local storage is bigger than 2TB.