New Datastore Size v5

[RParker]

So if the new ESXi can handle larger than 2TB VMFS, yet current storage controllers (such as Netapp) are limited to 2TB LUN, creating a 4TB VMFS datastore amounts to the same thing as (2) 2TB LUN using an extent, true or false?

To say that the new vSphere will support greater than 2TB VMFS is a half truth, because the current storage controllers are still pretty much using the old LUN limits..

Unless I missed something?

[mrudloff]

Ha ?

Storage Controller have a 2TB per LUN limit ? We have server here using Adaptec, LSI or 3Ware and each raid card supports 2TB+ - in fact, a Windows Server based on 2008R2 just had a 8TB LUN created to be used as ISCSI NAS ...

[Baddos]

Netapp isn't really a block storage controller. There are plenty of san vendors that have been providing luns > 2tb for years.

[wdroush1]

Heh, the only limitations I've seen on free OS based SANs I've been working heavily with is 32-bit hardware has a 1.2TB LUN limit.

Everything else, sky is the limit (actually there is some theoretical max, but it's stupid high).

Sounds like Netapp is kind of terrible in that regard.

[DanielBoddy]

netapp seems to be limited by the hardware and/or the OS

for example the 2050 cannot go above 2TB but the newer 3020's can but only with ontap 8 (and i think only in certain modes, i.e. not V7 mode)

[RParker]

DanielBoddy wrote:

netapp seems to be limited by the hardware and/or the OS

for example the 2050 cannot go above 2TB but the newer 3020's can but only with ontap 8 (and i think only in certain modes, i.e. not V7 mode)

We have 3070's they can NOT (we are on ontap 8.0).  are you sure about 3020, we are moving to a 3170 next year.. supposedly.. hopefully this limitation will be gone..

[DanielBoddy]

sorry typo there this was the 3200 family (not 3020 they are very old pre 2006)

http://www.netapp.com/us/products/storage-systems/fas3200/fas3200-tech-specs.html

50 or 70TB volume depending on model chosen and i believe this requires them to be in a 64bit mode so might not be a case of enabling Ontap 8 on Ontap 8 supported platforms becuase the datastores would have to be converted from 32bit to 64bit (if thats even possible)

[ZeroGravity2011]

I think you are confused about the limits and associated terminology used. The 2TB limit was a VMFS limit not a storage array limit per se, I don't know of any storage array that can't present more than 2TB LUN to a host. The problem in VMware pre VMFS5 is you can't go beyond 2TB when creating a VMFS datastore using block based storage, i.e. fibre channel, iscsi or internal/direct attached disks. When using NFS you can go beyond 2TB without a problem. However, in both cases there is a 2TB VMDK limit regardless of the storage protocol. VMFS5 removes the 2TB limit on block based storage but the 2TB VMDK limit still applies.

[RParker]

ZeroGravity wrote:

I think you are confused about the limits and associated terminology used. The 2TB limit was a VMFS limit not a storage array limit per se, I don't know of any storage array that can't present more than 2TB LUN to a host.

You do now!.  Netapp 3000 series.  Just to be clear we are talking a SINGLE LUN.  I can present multiple LUN's and make them extents (I don't want to do that) but a SINGLE lun cannot be bigger than 2 TB on Netapp.

The limit is a STORAGE CONTROLLER limit, not VM Ware.  I didn't realize there were so many storage controllers that did NOT have this limit, I assumed it was an industry strandard.

The Netapp is limited to 2TB, but apparently we are in the minority (changing that soon).

[DanielBoddy]

he is correct in that Netapp do (did) have 2TB lun limits for certain models and OS versions of ontap - we are using FAS2050's in two installations and these very definately have a 2TB limit for dedup luns - i know we tried to enable it on test luns 1.8 TB no problem dedup ok, 2TB - Ontap 7 refuses to enable it - so therefore we are limited to using LUNs below the 2TB limit if we are to use this hardware with ESXi 5.

[mcowger]

There are tons of controllers that do > 2TB LUNs.

Symmetrix, EMC VNX, 3PAR, Compellent, etc can all do it.

[jayctd]

Netapps current limit for all filers including the 3000 series is 16 TB for maximum LUN size

There  are limits on the maximum volume sizes with deduplication enabled that  depend on the filer model. I have also been seeing reports that you cant  expand a LUN more than 10x its origional size maybe that is what you  are hitting?

I  just tested in my testlab and had no issues making a 6TB LUN on my 2040  (8.0.1P3) this is all the spare disk I had avaliable for test in the  lab right now

[EdWilts]

DanielBoddy wrote:

sorry typo there this was the 3200 family (not 3020 they are very old pre 2006)

http://www.netapp.com/us/products/storage-systems/fas3200/fas3200-tech-specs.html

50 or 70TB volume depending on model chosen and i believe this requires them to be in a 64bit mode so might not be a case of enabling Ontap 8 on Ontap 8 supported platforms becuase the datastores would have to be converted from 32bit to 64bit (if thats even possible)

Today you can't convert a 32-bit aggregate to a 64-bit aggregate today but it's pretty easy to create a 64-bit aggregate and storage vMotion your datastores.  We've done a bunch of that.  Of course you need to have the spare spindles to do this but a good time would be at your next disk purchase.

The next release of OnTap is rumored to allow an in-place conversion of the aggregate from 32-bit to 64-bit.  I'm not sure I'd want to be one of the first to try that though...  Better a live chicken than a dead duck.

With 32-bit aggregates, the maximum aggregate size is 16TB which is still pretty darn big.  You want to go with bigger aggregates to spread your load across more spindles though.  With 600GB SAS drives, it doesn't take very many before your aggregate is full.

[RParker]

Netapps current limit for all filers including the 3000 series is 16 TB for maximum LUN size

Show me the documentation on that, because it's NOT accurate (for 3000 series).  Let's do a step by step shall we?

Create the volume (check).  A VOL can be 16TB in size yes.. depending on 32-bit Aggregate or 64-bit Aggregate.

Create the LUN.  If I choose 4TB (example) for the LUN size I get an error (LUN Maximum size CANNOT be greater than 2TB).  PERIOD.

This concludes the steps.  Show me where you can create a LUN (not the VOLUME) bigger than 2TB when the Filer will NOT let you do it.

I tried Netapp System Manager, the Web Console, or Command line... *ALL* of them have the same 2TB LIMITS.

Doesn't matter AIX, HP, VM Ware, Windows.. ALL LUN types have the same Limits.

[EdWilts]

If you have the right NetApp hardware and software, you can create a LUN > 2TB.  This example show a creation of a 3TB LUN on a FAS6210 with a 64-bit aggregate.

stp1nas04a> version
NetApp Release 8.0.2 7-Mode: Mon Jun 13 14:13:45 PDT 2011

stp1nas04a> vol create /vol/ewilts aggr1_sata 4000g
Wed Aug 24 10:57:03 CDT [stp1nas04a: export.auto.update.disabled:warning]: /etc/exports was not updated for ewilts when the vol create command was run. Please either manually update /etc/exports or copy /etc/exports.new to it.
Creation of volume 'ewilts' with size 4000g on containing aggregate
'aggr1_sata' has completed.
stp1nas04a> lun create -s 3t -t vmware /vol/ewilts/vmware
stp1nas04a> lun show /vol/ewilts/vmware
        /vol/ewilts/vmware             3t (3298534883328) (r/w, online)

[RParker]

Ed Wilts wrote:

If you have the right NetApp hardware and software, you can create a LUN > 2TB.  This example show a creation of a 3TB LUN on a FAS6210 with a 64-bit aggregate.

What Filer do you have?  A FAS6210.

What Filer do *I* have?  A FAS3070.

with a 3000 SERIES Netapp you CANNOT CREATE > 2TB LUNS!

jayctd incorrectly said..

Netapps current limit for all filers including the 3000 series is 16 TB for maximum LUN size

. . . but it doesn't.

just because you have a later version that does STILL does not help me does it?

That is my point.  I *KNOW* there are Netapp filers that support 2TB and larger for LUN's, but *WE* don't have don't have it.  Until we get a new filer I am stuck with a 2TB limitation.

[jayctd]

Dont know what to tell you

Volume Created at 6 TB

Lun Created at 3 TB no snapshot reserve

Edit: Just for refference this is a 2040

Nothing fancy

May be worth asking the netapp communities maybe there is an option maximum setting or some upgrade path that imposes an older lower limit.

As well as other forums stating the limit is 16 TB as well

http://communities.netapp.com/thread/14909

http://forums.citrix.com/message.jspa?messageID=1481170

[RParker]

Jered wrote:

Dont know what to tell you

Volume Created at 6 TB

Lun Created at 3 TB no snapshot reserve

Nothing fancy

May be worth asking the netapp communities maybe there is an option maximum setting or some upgrade path that imposes an older lower limit.

As well as other forums stating the limit is 16 TB as well

http://communities.netapp.com/thread/14909

http://forums.citrix.com/message.jspa?messageID=1481170

OK, apparently we have a language barrier.

Netapp 3000 Series.  I don't care what Ontap 8.x supports.

Netapp 3000 Series, does NOT allow > 2TB LUN's.  I am not talking OS, not talking about File systems, not talking about Volumes or Aggregates or 32-bit vs 64-bit

*YOU* said the 3000 Series Netapp supports greater than 2TB LUN's and *I* am telling you it does NOT.

[EdWilts]

RParker wrote:

So if the new ESXi can handle larger than 2TB VMFS, yet current storage controllers (such as Netapp) are limited to 2TB LUN, creating a 4TB VMFS datastore amounts to the same thing as (2) 2TB LUN using an extent, true or false?

To say that the new vSphere will support greater than 2TB VMFS is a half truth, because the current storage controllers are still pretty much using the old LUN limits..

Unless I missed something?

There appear to be 2 parts to this question.  Current NetApp controllers - all of them sold today - are not limited to 2TB LUNs, as I've demonstrated in another post.  To say that the vSphere 4 will not support greater than 2TB VMFS is not a half truth - VMware is not saying they support the limits on ALL controllers - we haven't even seen the compatibility matrix yet.

Your controller may be limited to 2TB but that doesn't mean that ESXi is limiting it.

The FAS3070 you're using is already 2 generations behind - it was first replaced by the 31xx line and then the 32xx line.  Current storage controllers are not "pretty much using the old LUN limits".

I don't think you're missing anything - what you can do within vSphere 5 is now limited to what your filer can support.  With vSphere 4, it was a limit of vSphere, not the filer.

There will always be limits on what the hardware will support and what vSphere can support.  You appear to have hit your hardware limit and there's nothing VMware can do about that.

[EdWilts]

RParker wrote:

Netapp 3000 Series.  I don't care what Ontap 8.x supports.

The FAS3070 supports OnTap 8.x:  https://now.netapp.com/NOW/cgi-bin/software/?product=Data+ONTAP&platform=V3070

I don't know if your FAS3070 with OnTap 8.x would allow you to create a LUN > 2TB.