athlon_crazy
Virtuoso
Virtuoso

iSCSI Datastore Best Practice

Hi there, just a quick one. My environment :

  • MSA 2000i iscsi with 4TB RAW size

  • 11x virtual machine (6 run oracle database) = 760GB size after resize.

  • ESX configure with iscsi initiator & 2x nic + own service console

Since, I'm beginning to follow VMware best practice documentation, what is the best way, option & things need to consider such distribute VMs among datastore, performance, I/O bottleneck, RAID 10 & etc before I create datastore for this 11x VMs?.

I'm thinking to have atleast 2x datastore with max 500GB in size each.

Thanks

VMware newbie..

Zen Systems Sdn Bhd

www.no-x.org

http://www.no-x.org
0 Kudos
19 Replies
kjb007
Immortal
Immortal

Typical best practice applies to any block-based datastore. 300-500 GB LUN sizes, or 10-20 vm's per LUN. RAID type depends on data requirements, and not really on VMware best practice, per se. You want to minimize if not completely isolate the iSCSI network to itself. Isolate, at least virtually using VLANs, if you can't isolate physically with separate physical NICs, etc.

-KjB

VMware vExpert

vExpert/VCP/VCAP vmwise.com / @vmwise -KjB
chriswaltham
Enthusiast
Enthusiast

KjB: I don't mean this the way it sounds, but do you have a reference for as to how/why you should size LUNs -- either in GB/LUN terms of VM/LUN terms? I am curious as to how and why 'ideal' numbers come up; I'm in the process of re-structuring our datastores so I'd love to take a look at some detailed information. Thanks!

0 Kudos
kjb007
Immortal
Immortal

Here's something to start with. https://www.vmware.com/pdf/vmfs-best-practices-wp.pdf

Mind you that hard numbers are harder to come by, but you can at least see the theory here.

-KjB

VMware vExpert

vExpert/VCP/VCAP vmwise.com / @vmwise -KjB
0 Kudos
chriswaltham
Enthusiast
Enthusiast

Thanks! I'll give that a read. When using iSCSI datastores and VMotion HA, where should I put the VM swap files? iSCSI LUNs, local (i.e., VMware host) disk, NFS shares..? The NetApp best practices guide just recommends separating swap out, but it doesn't say where is the best place to put it (or how the best place to mount it is.)

0 Kudos
kjb007
Immortal
Immortal

You can put swap files on local disk, but it will impact performance, and vmotion will be a problem. HA may be fine since you're talking about restarting and not live move. I'd leave them with the vm.

-KjB

VMware vExpert

vExpert/VCP/VCAP vmwise.com / @vmwise -KjB
0 Kudos
ablej
Hot Shot
Hot Shot

A lot of storage design decisions will come from what kind of performance and recovery agreements you're company has. But best practice is to use no more than 10-20 VM's per VMFS volume because of SCSCI reservation conflicts. So you will need size you're LUNS accordingly.






David Strebel

www.holy-vm.com

If you find this information useful, please award points for "correct" or "helpful"

David Strebel www.david-strebel.com If you find this information useful, please award points for "correct" or "helpful"
0 Kudos
chriswaltham
Enthusiast
Enthusiast

Sorry, I always get the acronyms mixed up! We're doing VMware DRS, not HA.

0 Kudos
JoJoGabor
Expert
Expert

We do a lot of deployments on NetApp and I really like the following method:

Use NFS for system volumes, then you dont need to worry about SCSI reservations, Use no more than 700GB volumes with no more than 20VMs. Use iSCSI LUNs and use the MS iSCSI initiator within the VM to map directly to those to take advantage of the SnapManager utilities. Size these LUNs depending on application. Use RAID-DP throughout with as many disks as possible in the aggregate.

0 Kudos
kjb007
Immortal
Immortal

Then you want to keep your swap files with your vm. Don't change the default here.

-KjB

VMware vExpert

vExpert/VCP/VCAP vmwise.com / @vmwise -KjB
0 Kudos
athlon_crazy
Virtuoso
Virtuoso

OK bout scsi reservation, how do we measure or can we monitor the scsi reservation process?

VMware newbie..

Zen Systems Sdn Bhd

www.no-x.org

http://www.no-x.org
0 Kudos
kjb007
Immortal
Immortal

Hmmm. Monitor the process directly? I'm not sure. You can run esxtop and watch the CMDS/s gong to/from your "hba".

-KjB

VMware vExpert

vExpert/VCP/VCAP vmwise.com / @vmwise -KjB
0 Kudos
depping
Leadership
Leadership

I wouldn't be to worried about scsi reservations.

my suggestion would be make at least two LUNs maybe even three depending on the averag VM size. I'd rather have 3 or 4 LUNs than just 2. (Single point of failure, performance, locking etc)

Duncan

VCDX | VMware Communities User Moderator

-


Blogging:

Twitter:

If you find this information useful, please award points for "correct" or "helpful".

0 Kudos
athlon_crazy
Virtuoso
Virtuoso

2 or 3 LUN per datastore is it what you mean depping or else?

VMware newbie..

Zen Systems Sdn Bhd

www.no-x.org

http://www.no-x.org
0 Kudos
Yattong
Expert
Expert

Always try and keep 1 LUN per datastore.

I agree with the above that you should have multiple datastores, 2+.

If you already know your I/O requirements for dB servers, you can try different RAID configs and see what kind of performance you can get from it.

You can use tools like SQLIO or IOMeter to check the array performance...



If you found this or any other answer useful please consider the use of the Helpful or correct buttons to award points

~y

If you found this or any other answer useful please consider the use of the Helpful or correct buttons to award points ~y
0 Kudos
BUGCHK
Commander
Commander

On the MSA2012i, create at least two 'virtual disks' (= RAID containers for those who are not familiar with this array's terminology), because one controller manages a virtual disk and all the volumes it contains. Then you can create one or more 'volumes' (the storage reservation which is presented as SCSI LUNs to the hosts).

0 Kudos
athlon_crazy
Virtuoso
Virtuoso

On the MSA2012i, create at least two 'virtual disks' (= RAID containers for those who are not familiar with this array's terminology), because one controller manages a virtual disk and all the volumes it contains. Then you can create one or more 'volumes' (the storage reservation which is presented as SCSI LUNs to the hosts).

Glad to hav someone here which already had an experienced with MSA2012i specially on port group / vdisk. Few questions for you if you don't mind :

1) Does MSA 2000i is same product with MSA 2012i?

2) can we create different RAID on single vdisk which contains 8x500GB disk on MSA 2000i?

3) If can't, can MSA 2000i contains 2x vdisk with different RAID (RAID10 & RAID 5) plus global hotspare?

VMware newbie..

Zen Systems Sdn Bhd

www.no-x.org

http://www.no-x.org
0 Kudos
athlon_crazy
Virtuoso
Virtuoso

Thanks for sharing the tools Yattong. Yep I always keep on using one LUN per datastore since 2x LUN per datastore meaning I have to extend the datastore which not recommended I think. BTW, can someone tell me why datastore extend is not good?

VMware newbie..

Zen Systems Sdn Bhd

www.no-x.org

http://www.no-x.org
0 Kudos
JoJoGabor
Expert
Expert

I dont like using extents purely from a management point of view. Its too easy to forget that 2 LUNs are making up a single VMFS. Remember if you lose a LUN for whatever reason you lose the entire datastore. Usually losing a LUN is purely from somebody deleting it accidentally. If you plan the LUN design well enough there's rarely any reason to create extents, particularly now we have Storage Vmotion to live migrate Virtual mchine files around between datastores without service interruption.

On the MSA side of things, I think the 2012 is the same as an MSA 2000 - I think its just product terminology but I'm not sure, as far as I am concerned it is the same product, it behaves in the same way.

On the MSA2000 a vDisk refers to groups of disks which you assign a RAID level to, so no, a single vDisk can only have one RAID setting. The MSA1500 was different in that you create one group of disks and then create LUNs with different RAID groups. The MSA2000 is more like other SANs now.

Yes you can create separate groups of disks (or vDisks) with different RAID settings. For example for dedicated database LUNs you may want Raid10 for logs for sequential writes, and a RAID-5 for random-access data files.

athlon_crazy
Virtuoso
Virtuoso

Thanks for your explaination. Credits goes to whoever deserved it.. I just got chanced today to configured MSA2000i with 2x vdisk (RAID5) plus one global hotspare. Well, with good planning the setup was running as what I planned b4 with acceptable performance. ESX hosts smoothly detect the LUN from MSA2000i. The only minor thing is, by default second vdisk will go to controller B which didn't comewith my MSA2000i. I have to changed the owner to controller A & modify the volume mapping table, only then ESX can detect the remaining LUN on second vdisk.

VMware newbie..

Zen Systems Sdn Bhd

www.no-x.org

http://www.no-x.org
0 Kudos