Large Storage Server - Virtual or Not?

[DamienStevens]

I understand this is possible, the question is: Is it advisable to virtualize a 16TB Volume with 1+ Billion files?

What we're doing in Hardware today:

Front-end Servers (many virtual) - Linux Front-ends that receive data and write to to the back-end server over NFS. The front-end servers receive files over the Internet at up to 100 mbps and write to the back-end server.

Back-end Linux Server running NFS w/ 16TB RAID6 Array (1 Large Volume, about 14TB) - Around 1 Billion files and growing (will grow array to 24TB)

Questions:

1a) Should we run all of the above via ESXi as 1 large storage server (with extra RAM and Procs) OR Leave the storage server as Hardware Only? (will be performance be sufficient)

Restated, should we:

1b) 1 x 16TB Storage Server Hardware Only w/ NFS + 1 x Multi-proc & RAM Server with 3-4 Guest VMs accessing the storage server over NFS (2 physical servers).

OR

1c) 1a) 1 x 16TB Storage Server w/ Multi-proc & RAM and run 3-4 Guest VMs accessing the storage guest over local VM network NFS (1 physical server).

2) Do we do this by creating a linux guest OS that has the entire drive/volume and runs NFS and allowing the other guests access over NFS (on the same sever) OR is there a better way?

3) Does it make sense to use VMFS for this level of files and growth in addition to NFS and RAID6?

[weinstein5]

Welcome to the Forums - it is not possible to use ESX/ESXi to virtualize a datastore that large - ESX/ESXi has a maximum LUN size and virtual disk size of 2 TB - this is for SAN storage. I am not sure if this holds true if you were to use /NAS/NFS attached storage -

If you find this or any other answer useful please consider awarding points by marking the answer correct or helpful

[DamienStevens]

Wouldn't it be possible to setup multiple 2 TB virtual disks and use extents and something like LVM to tie them into one large volume?

[Ken_Cline]

Wouldn't it be possible to setup multiple 2 TB virtual disks and use extents and something like LVM to tie them into one large volume?

Yes, you could do that. Each volume (from an ESX perspective) is limited to 2TB, but you could use LVM within the guest to make it as large as the guest can support. ESX/i should be able to handle the load (assuming HW - especially storage - is configured properly).

Question - how do you back this thing up?

Ken Cline

VMware vExpert 2009

VMware Communities User Moderator

Blogging at: http://KensVirtualReality.wordpress.com/

[DamienStevens]

This is actually a replication target for several production environments - this is the backup.

Would you recommend that I do this inside VMware / VMFS or the storage via Hardware Only and Run VMWare on a separate machine?

[depping]

I'm just wondering why you would want to virtualize a server like this. What are reasons to do it?

Of course you can use extends to have the 16TB available as a VMFS volume, but it would make more sense to leave it as NFS in my opinion if you would at one point in time want to return to physical it would be a lot easier. So it offers more flexibility when using NFS directly.

Duncan

VMware Communities User Moderator | VCP | VCDX

-

---

Blogging:

Twitter:

If you find this information useful, please award points for "correct" or "helpful".

[DamienStevens]

I wanted to see if you recommended running the storage in VMFS as their will be extra HW available to run some VMs, but I think you're point is it will be easier to leave it as hardware.

Thank you.

[kcollo]

I would not attempt it, but if doing ESX, you can do scsi pass through and run the server as a VM with a dedicated storage card (SCSI or FC) connected to and external array. Then you could use the OS to format the externally attached disks (transparent to vmware) however you want besically bypassing VMFS/using VMware to manage the actual disks. I havent done this myself, but would love to hear from people who have.

Kevin Goodman

kevin@colovirt.com

http://blog.colovirt.com