VMware Cloud Community
shebang
Contributor
Contributor

ESXi local storage recommendation

If using local storage for ESXi, should this be raid storage or is a simple harddisk sufficient. What is recommended? P.S. The VM's are on the SAN.

Reply
0 Kudos
55 Replies
shebang
Contributor
Contributor

Thanks for all response. For this ESX host I´ll go for local raid-1 storage, but I'm certainly going to try, on a less critical server, the possibilty of using flashmedia for ESXi.

Tnx,

Reply
0 Kudos
DSTAVERT
Immortal
Immortal

Yes, USB has a small amout of data writes, but if there will be small I/O and also lets remember that it has a really big amout of data writes, but with insufficient write speed. So, I don`t think that if will store on SAN you will have big overhead.

I don't know what big data writes ESXi has. Other than log files which, since they don't survive reboots, should be reconfigured to write to a log server or a datastore what big writes?

ESXi was designed to be SMALL so that it could be installed on flash. That was the whole idea. NO drives in the host. Spend your money on shared storage.

-- David -- VMware Communities Moderator
Reply
0 Kudos
AntonVZhbankov
Immortal
Immortal

>Other than log files which, since they don't survive reboots, should be reconfigured to write to a log server or a datastore what big writes?

Log problem is easily solved by VM with linux and syslog-ng as centralized log-server.


---

MCP, MCTS, VCP, VMware vExpert '2009

http://blog.vadmin.ru

EMCCAe, HPE ASE, MCITP: SA+VA, VCP 3/4/5, VMware vExpert XO (14 stars)
VMUG Russia Leader
http://t.me/beerpanda
Reply
0 Kudos
DSTAVERT
Immortal
Immortal

I was asking the question about why those who don't think flash is an acceptable Enterprise level installation medium since writes are almost NIL especially when you do use a logserver. I don't understand the fear of lots of writes.

-- David -- VMware Communities Moderator
Reply
0 Kudos
Formatter
Enthusiast
Enthusiast

I have been running off CF card for about 1.5 years, No problems except the ones I caused. Upgraded from 3.5 to 4 and no problem I do all the patches and again no problem. I have the logs writing tothe CFand have had no problem. Changed the configuration lots also.

I also dont understand why you would run it any other way then on CF.

Reply
0 Kudos
vidals
Contributor
Contributor

I heard during the last VMUG meeting that ESX 4 (vSphere) will be the last version that will be available with a hard drive install. In the future, the embedded ESXi version will be the preferred method. I understand that the VMware developers will be emphasizing ESXi and de-emphasizing ESX as time goes by.

ESXi is more stable and easier to upgrade. Because it doesn't have a hard drive, it is more stable. Because it doesn't have a service console, it is much easier to upgrade. Recall that most of the vmware updates are for service-console related packages, so if there is no console, then it is simpler to upgrade. In order to access the OS of ESXi, I've started using Vma (VIMA).

#####################

Gil Vidals, VCP

www.vmracks.com

ESX VMware Hosting

Dedicated VI Leasing

Server Security

support@vmracks.com

San Diego, CA

#####################

##################### Gil Vidals, VCP www.vmracks.com ESX VMware Hosting Dedicated VI Leasing Server Security support@vmracks.com San Diego, CA #####################
Reply
0 Kudos
s1xth
VMware Employee
VMware Employee

I have been using flash drives on two hosts with 4.0 esxi since it came out. I am actually moving all my hosts to this in the next few months. Not a single issue. I use good flash drives (sandisk) and everything runs great. There is no reason not to use flash, I am done buying local storage. Smiley Happy

http://www.virtualizationimpact.com http://www.handsonvirtualization.com Twitter: @jfranconi
Reply
0 Kudos
chrisaug
Contributor
Contributor

ESXi is more stable and easier to upgrade. Because it doesn't have a hard drive, it is more stable.

I find the fact that just by installing ESXi on USB media, as opposed to hard drives, makes it more stable.

And why not gain the extra performance advantage of a 15K SAS drive with throughput of 3 Gb/s as opposed to a max throughput of 480 Mb/s. Couple that with the capability to configure a redundant RAID array with my boot drives. It sounds like I will be the last person on Earth still installing their ESX on local storage. Quite honestly, I'll configure a boot-from-SAN configuration before I install on Flash media.

Reply
0 Kudos
DSTAVERT
Immortal
Immortal

Where is the performance other than a slight difference at startup. ESXi loads into RAM and done. ESXi was designed to run from flash. That was the design goal for ESXi. It was how it was originally delivered. Installing on local media is a waste of money. Eliminate the controller and local drives.

-- David -- VMware Communities Moderator
Reply
0 Kudos
TobiasKracht
Expert
Expert

Need to note that ESXi does`nt support boot from SAN

StarWind Software R&D

StarWind Software R&D http://www.starwindsoftware.com
Reply
0 Kudos
AlbertWT
Virtuoso
Virtuoso

OK, so in this case of USB ESXi 4 install i shall use a proper brand with "ware" leveling.

has anyone ever got problem with USB malfunction in the ESXi install ? I wonder what would happens in the realworld datacenter scenario if the ESXi write so many logs into the USB ?

Kind Regards,

AWT

/* Please feel free to provide any comments or input you may have. */
Reply
0 Kudos
DSTAVERT
Immortal
Immortal

A quality USB stick should do leveling.

-- David -- VMware Communities Moderator
Reply
0 Kudos
Dave_Mishchenko
Immortal
Immortal

ESXi writes it's logs to /var/logs which is not on the USB flash drive. When it boots up it creates a ram drive and then mounts the various partitions as required. /bootbank, /altbootbank , /store etc are on the USB flash drive but other folders like /var/logs /etc are in memory. When you boot from USB flash drive you also have to configure the swap file for ESXi on a datastore. I've been using a couple of cheap USB drives for a couple of years (light usage) with no issues. HP on the other hand shipped a bunch of faulty one early on.




Dave

VMware Communities User Moderator

New book in town - vSphere Quick Start Guide -http://www.yellow-bricks.com/2009/08/12/new-book-in-town-vsphere-quick-start-guide/.

Do you have a system or PCI card working with VMDirectPath? Submit your specs to the Unofficial VMDirectPath HCL - http://www.vm-help.com/forum/viewforum.php?f=21.

Reply
0 Kudos
AlbertWT
Virtuoso
Virtuoso

Wow, I've never knew that. but Yes, thanks for the explanation Dave now I know how the ESXi USB install works in general.

I'm now getting the 16 GB USB stick for my office. (I know it is way too big, I've done my home test environment with just 1 GB USB).

Cheers.

Kind Regards,

AWT

/* Please feel free to provide any comments or input you may have. */
Reply
0 Kudos
s1xth
VMware Employee
VMware Employee

I have been using Sandisk USB drives without a problem. 4GB since it's

tough to get a GOOD 1GB anymore. My Dell r710s shipped from Dell with

Kingston SD cards, generics. No issues with them.

Don't forget, ESXi was designed from the ground to be embedded.

Sent from my iPhone

On Nov 12, 2009, at 8:18 PM, AlbertWT <communities-emailer@vmware.com

http://www.virtualizationimpact.com http://www.handsonvirtualization.com Twitter: @jfranconi
Reply
0 Kudos
AlbertWT
Virtuoso
Virtuoso

Yes, exactly so hard to get a small one nowadays.

That is why I'm now end up with 16 GB which is far too generous.

I wonder if this empty space can be used for storing ISO file and templates Smiley Happy does anyone ever try this ?

Kind Regards,

AWT

/* Please feel free to provide any comments or input you may have. */
Reply
0 Kudos
vidals
Contributor
Contributor

When I was testing out HA, I decided to pull out the USB flash drive from my ESX 3.5 host. Well, to my amazement, the ESX host kept chugging along! Obviously the earlier post in this thread is correct. The embedded version of ESX is loading everything into RAM and runs safely from there. The USB isn't needed (at least not during my "test"). I don't know if eventually, the server would have required to write to the USB, but it didn't seem to. I have a feeling that it would have run fine without the USB stick, which gave me more confidence in the embedded version.

I do feel better knowing the USB flash drive is plugged in just in case I have to reboot the server Smiley Happy

#####################

Gil Vidals, VCP

www.vmracks.com

ESX VMware Hosting

Dedicated VI Leasing

Server Security

support@vmracks.com

San Diego, CA

#####################

##################### Gil Vidals, VCP www.vmracks.com ESX VMware Hosting Dedicated VI Leasing Server Security support@vmracks.com San Diego, CA #####################
Reply
0 Kudos
s1xth
VMware Employee
VMware Employee

You wont have any problems with the USB. Like you said, if the USB drive fails, just slap in another one and you are back in business!! Smiley Happy

http://www.virtualizationimpact.com http://www.handsonvirtualization.com Twitter: @jfranconi
Reply
0 Kudos
DSTAVERT
Immortal
Immortal

Another reason to have it in there is that once an hour it copies the configuration to the alternate bootbank partition.

-- David -- VMware Communities Moderator
Reply
0 Kudos
truo1van
Contributor
Contributor

Gents,

Speaking from experience: DO NOT run ESXi on USB key.

We have had numerous issues of ESX hosts: symptom#1 (in HA cluster) reporting "HA Agent error". Attempts to "Reconfigure HA" works most of time.

However, when it did not work, the prob aggrevateed to "Failure to install Virtual Centre Agent", the affected ESX host dropped out of the VC. You can all imagine what that would mean!, VMs could be running fine, but ESX host is now stand alone

Raised a SR with VMWare support, rebuilt ESX did not help. Had vendor in to replace new USB key flash, reinstalled and thing is back to normal ==> faulty USB flash. VC failed to deleted and re-create the vpxa and aam subdir when jioing VC and HA cluster ==> symptons of USB key corruption/failure, its actually revealed in /var/log/messages.

Another sympton#2 we just had 3 days ago: "HA Agent error", VMs running on the affected host started to have network drop in/outs ie, responses to pings are intermittent. Same thing: "Reconfigure HA" attempts failed and dropped out of VC ==> outage. Reboot seemed to sort of re-jion back to VC but still have that HA error.

I know of another organisation having exact same issue + symptoms like above and their running their ESX host on HP (btw, we run on Dell 2950 II).

To me, these are reasons NOT to use USB key:

1) Little or not ESX DOES writes to USB key - USB is not a reliable, not as reliable as a RAID1 HDs setup, a media for read/write IO for critical OS platform

2) There is a official release from HP on the USB keys inside HP boxes due to internal heat environmentals

3) From personal experience above. Note: We have run 3.i and upgdared 4i in test/dev farm WITHOUT ANY ISSUE

4) Symptom#2 is a "deadly one" - its quite unexpected the ESX USB flash issue manifested itself in VM losing network connections - its like vmnic driver TCPIP stack overflow of some sort - but NO, it was not.

Anyway. We decided to slowly move our ESX hosts off USB key and onto HDs. In hindsight it was a good decision to buy these hosts with RAID1 15K disks Smiley Happy).

Thats my experience and interpretation of the issue.

Good luck guys.

Reply
0 Kudos