TiBoReR
Enthusiast
Enthusiast

16TB of database data to configure

Hi guys,

I'm planning a project with 3 oracle databases, one of 8TB and two of 4TB.

I want to have advices please.

What I'm planning now:

LUN and VMFS datastore named VMFS_Oracle_DB1_01 of 2TB

LUN and VMFS datastore named VMFS_Oracle_DB1_02 of 2TB

LUN and VMFS datastore named VMFS_Oracle_DB1_03 of 2TB

LUN and VMFS datastore named VMFS_Oracle_DB1_04 of 2TB

LUN and VMFS datastore named VMFS_Oracle_DB2_01 of 2TB

LUN and VMFS datastore named VMFS_Oracle_DB2_02 of 2TB

LUN and VMFS datastore named VMFS_Oracle_DB3_01 of 2TB

LUN and VMFS datastore named VMFS_Oracle_DB3_02 of 2TB

I know i would have been able to do a big VMFS of 16TB with extents but I saw a lot of performance issues with extents in forums.

Some people told me to do RDM instead of VMFS, but I can't see advantages.

All my OS partitons are in other LUNs like VMFS_OS_01 and VMFS_OS_02.

0 Kudos
13 Replies
MR-T
Immortal
Immortal

That's pretty serious storage.

It's not only about the VMFS side of things, the storage vendor normally makes recommendations about optimal sizes for LUNS.

Also, to see 2TB in one VMFS volume you'll need to format this with a large block size, not the default of 1MB.

What storage are you using?

BTW - Good call for not using extents

0 Kudos
Dave_Mishchenko
Immortal
Immortal

On this thread the last 2 posts have a few suggestions of why they prefer RDM over VMFS.

http://www.vmware.com/community/thread.jspa?messageID=639420&#639420

0 Kudos
esiebert7625
Immortal
Immortal

Yes, extents are not recommended....here's some reading for you...

ESX Server Raw Device Mapping - http://www.vmware.com/pdf/esx25_rawdevicemapping.pdf

LUNS - http://www.vmware.com/community/thread.jspa?messageID=333672

LUNS Size - http://www.vmware.com/community/thread.jspa?threadID=36725&start=0&tstart=0

SAN Configuration Guide - http://www.vmware.com/pdf/vi3_esx_san_cfg.pdf

SAN Conceptual and Design Basics - http://www.vmware.com/pdf/esx_san_cfg_technote.pdf

SAN System Design and Deployment Guide - http://www.vmware.com/pdf/vi3_san_design_deploy.pdf

Extending a VMFS3 data store - http://www.vmware.com/community/thread.jspa?threadID=65156&tstart=0

To use extents or not? - http://www.vmware.com/community/thread.jspa?threadID=81494&tstart=100

Choosing and Architecting Storage for your Environment - http://download3.vmware.com/vmworld/2006/adc0135.pdf

fyi...if you find these posts helpful, please award points using the Helpful/Correct buttons....thanks

0 Kudos
TiBoReR
Enthusiast
Enthusiast

That's pretty serious storage.

Yes it is.. It's why I need advices from the more people it's possible to have it working well with good performances.

It's not only about the VMFS side of things, the

storage vendor normally makes recommendations about

optimal sizes for LUNS.

Okay, I will take a look if they have recommandations.

Also, to see 2TB in one VMFS volume you'll need to

format this with a large block size, not the default

of 1MB.

Yes sure, it will be 8MB blocks.

What storage are you using?

It's a EMC DMX3.

BTW - Good call for not using extents

Happy to see I'm planning the right thing.

0 Kudos
Rumple
Virtuoso
Virtuoso

Are these Oracle databases big performance based databases or archival/low utilization systems.

For that type of storage you are looking at creating some pretty serious RAID groups across alot of disk to get the spindle count up if its performance based.

I'd have to be convinced pretty hard to virtualize something like that.

0 Kudos
TiBoReR
Enthusiast
Enthusiast

Are these Oracle databases big performance based

databases or archival/low utilization systems.

I would say mid performance databases.

For that type of storage you are looking at creating

some pretty serious RAID groups across alot of disk

to get the spindle count up if its performance

based.

Sure, I'm planning to put the more disks possible to my array to get better performances having a lot of disks heads.

I'd have to be convinced pretty hard to virtualize

something like that.

I can't understand that point. Why the physical way would be better in that case ?

0 Kudos
Rumple
Virtuoso
Virtuoso

The primary goal of Virtualization is to enable you to bring up the overall utilization of hardware by combining a bunch of low utilizated systems onto a single piece of hardware.

The secondary goal is typically to gain hardware independance for applications.

The third goal is typically High Availability

With oracle databases this big combined with the performance requirements you need to be able to get out of the disk for regular activity as well as backups, as well as the processor and RAM requirements that you'd also be running alot of other VM's on the same hardware unless your current ESX farm is pretty massive.

In my environment I could probably virtualize something like that but I'd probably then have to pay about 70k to put in another esx server to regain the virtualization space used up by a the 3 oracle instances as well as the

On the other hand a 20k server including O/S with a couple quad core intel processors and 32GB of RAM would more then suffice for the Oracle environment and leave me room to virtualize another 10 VM's or more in my current environment.

Not saying you shouldn't do it, but you have to really consider what you are trying to gain with the virtualization and if the added cost both from a monetary (ESX isn't cheap) and virtualization performance loss provides a big enough Return on investment to virtualize.

In my environment I have 3 solaris servers running oracle. I would have an outage if I lost any one of the servers, but within 4 hours I could rezone/remount the oracle luns on any of the other servers and recover the databases and suffer no performance loss (although we have 2 hour response on the physical hardware anyhow).

Not as fast as using ESX HA, but in our environment, the virtualization capacity we'd lose by virtualizing the oracle was of more importance then the cost of the oracle servers.

TiBoReR
Enthusiast
Enthusiast

Most important reasons why we will use virtualization in that scenario is about HA and one you forgot DR.

For your information, there are 4 servers 2 X dual-core 3Ghz with 36GB RAM in that project.

We will see in production but I'm planning performances from CPU and RAM to be good. My real concern is about disk space capacity and performance.

It's always appreciated to get comments like yours. Thanks again.

0 Kudos
CWedge
Enthusiast
Enthusiast

You shouldn't have to many problems on the disk side.

I'm running a DMX3000 25% filled with great performance.

moneill
Enthusiast
Enthusiast

Your preference for virtualisation makes a lot of sense to me. The bigger the db the longer it takes

to rebuild all the views - avoid shutting it down at all costs. I haven't built anything this large,

but the investigation for an aborted project lead me to understand that rdms in virtual compatibility

mode are the best option. Vmotion (inc HA/DRS) capable with some SAN features for backup etc.

Performance no less than vmfs but much more flexible.

I would also be making sure that my hosts were 64bit capable and I would be running 64bit

guests to handle addressing this volume of data.

Mike

0 Kudos
petedr
Virtuoso
Virtuoso

We are running Oracle database a virtuals and the performance had not been an issue however our size is nowhere near what you have, DB is just under 100GB. With the smaller database on one vmfs, we do split the application code on another vmfs, and the operating system on a third. Good luck with your project.

www.thevirtualheadline.com www.liquidwarelabs.com
0 Kudos
mikemohr
Contributor
Contributor

I have a project to assign three TB of space to a VM for an application/SQL Server. The SAN behind the scenes is a Sym DMX. We were told by our internal storage team that the best practice was three, one TB LUNs pointed to the server. So I formatted each of those with 4 MB blocks and let VMFS do its job. I just mapped them to the server directly, no extents, no RDMs. Inside the VM, I created dynamic volumes based off the sizes recommended by the vendor for the application, ironically also EMC. My only concern with this setup is the inability to align the volumes at the VM level because they already have a file system as opposed to RAW disks. Hopefully this won't cause many issues as this isn't highly utilized, just needs a TON of storage.

0 Kudos
TiBoReR
Enthusiast
Enthusiast

So, should I double my number of virtual disks splitting them to 1GB ? Will performances be better ?

0 Kudos