If using local storage for ESXi, should this be raid storage or is a simple harddisk sufficient. What is recommended? P.S. The VM's are on the SAN.
Hi,
@truo1van, Yes i don't recommend it as well to install ESX on USB, but in my production environment, I'm using ESXi 4 on Dell Power Edge 2950-III I plug the Corsair Voyager GT Hi-SPeed 16 GB USB2 stick and it runs flawlessly very fast and reliable since I'm plugging the USB in the external port (behind the server).
So far no problem with ESXI 4 on USB as long as you choose the good brand. FYI Dell comes with Kingston 1 GB USB but I'm not using it in production.
Kind Regards,
AWT
Re HP etc withdrawing support for internal USB - I wonder if the crux of the issue is exactly why these same servers all use registered DIMMS instead of the cheaper unbuffered - namely electrical noise. Mounted externally the device should be immune to that at least.
Please award points to any useful answer.
Guys... I am going to have to disagree about the USB drives. I run all
embedded USB ESXi in production across 25 hosts. Never had a single
problem and you shouldn't. There are very few writes made to the
drive, also once the server is booted the firmware is loaded entirely
into memory and runs from the server...NOT the USB drive!! If you are
having issues you need to look at your hosts. USB is entirely support
by VMware.
Sent from my iPhone
On Feb 4, 2010, at 5:42 AM, AlbertWT <communities-emailer@vmware.com
Gents,
I know ESXi can run on USB, supported by VMware etc.. thats why we run on USB from day on our Prod!.
For me, the ultimate question to ask is: if I had a safer (and I do) choice, albeit conventional less "flashy", more reliable would I take it?.
Its risk management.
USB stick dangling off front or back of host in datacentres is not an option for us. Dont believe large organisations or law enforcement agencies with security mandates/ceritfications.. would allow you to do that.
Corrections: We run Dell 2950 III in Prod and 2950 II + R610 in tes/dev.
When things runs OK, all good, but when they dont.. thats why we need to minimize if cant eliminate the chances.
Btw: lets throw in the mix ==> vShield zones to be used for hosts in DMZ! Imagine that playing up!
For me, need things proven and solid - verified by own experience.
chrs,
VanT
The new Dell r610 servers have embedded SD cards inside the server
cases. Fully meeting all security/hippa requirements.
Heck you can PULL a USB drive from a running server and it still
function.
Sent from my iPhone
On Feb 4, 2010, at 6:01 PM, truo1van <communities-emailer@vmware.com
Think about all the routers and switches that you depend on. Run from flash. Have since day one. I posted a link to a reliability report on USB flash drives. Can't find it right now but it basically said something like if you wrote to it constantly the drives would last many many many years.
We are a very small organization but I replace several dozen spinning hard drives drives every year. Hard Drives ABSOLUTELY will fail. Most of my problems with USB disks are drying them out after they have gone through the washing machine.Over the years I have had very very few failures. Even the ones that have gone through the washing machine.
Added bonus USB contributes very little to the heat load and the power load. Those are good things for a server.
Yes David,
I Agree with you, one of the reason I go down with using USB is that I can now very confident that with better brand such as Corsair Voyager GT it would last very long time around 10 years of non stop read write operation.
Plus I also got miraculous incident where I accidentally put my Corsair USB into washing machine and found out that the data inside of it still intact nothing is lost until now
Hope this helps.
Kind Regards,
Albert Tedjadiputra
I agree about the thumb drives causing less heat and all, but again, why introduce a single point of failure into a production environment? Sure, the raid controller is a single point of failure, but so is the USB controller. One task, in building our ESX environment, is to limit single points of failure to the least amount possible to increase uptime.
If you have a thumb drive and it fails or gets pulled out, what's the difference if you have a raid array and a drive dies. Nothing except ESX can still write logs and whatever else it needs to the remaining raid array. And secondly, what kind of notification do you have if a thumb drive fails? If what you say is true about how everything gets loaded into RAM after boot, then you won't know the thumb drive failed until the next reboot. Things keep working with a raid array and I can see the amber light when a drive goes. I can replace it and let the array rebuild without any downtime.
I can see your point...but you have to undetstand if you are this
concerned about redunancy than you should have your servers in an HA
cluster. There are many points of failure besides just where esxi runs
from that have higher failure rates. (ex. Switches, controllers, SAN
storage, nic cards, back planes etc). The new cisco Ucs servers have
no hard drives and run all embedded esxi. VMware designed esxi to run
via a flash memory device. All of my r610 servers have zero hard
drives and run embedded esxi from Dell (built in custom sd chip
connected directly to mobo.. All 3rd gen 2950s also have a internal
USB plug on them on the side that you can use).
Like I said, if you are so concerned about HA looking at esxi is least
of your worries. I have been running 3.5 on flash drives before it was
even supported and on all servers running 4.0 using Sandisk flashes or
using the Dell Embedded if they are new boxes.
I trust the flash drives over hard drives any day.. Out of 600 servers
I have 20 drive failures a year.
Just my opinion, just giving you all the information from my exp.
Sent from my iPhone
On Feb 4, 2010, at 9:33 PM, chrisaug <communities-emailer@vmware.com
Gents,
Check this thread out. Exact same symptoms as I had.
http://communities.vmware.com/message/1470996
This is real life ESX Prod experience!
Think about all the routers and switches that you depend on. Run from flash. Have since day one...
This of course is absolutely true. However they also (traditionally at least) have a lot less going on inside, for example a router typically has one or two unbufferred DIMMS. Those just don't cut it in a high-density ESX box like (say) an R610.
Perhaps what is needed is EMI shielded flash.
The moral of your story should be not to buy cheap USB drives instead of not running ESXi on USB drives.
There has been an influx of bad USB drives as of late as all the good NAND chips are going into SSD drives.
Currently, non-reputable manufacturers are building USB drive out of NAND rejects (not cut out for SSD usage).
Buying from a reputable manufacturers is more important now than ever.
Last, you should try to buy SLC thumb drives (like the Corsair Voyager GT) and not the MLC versions.
SLC chips last 10x longer than the MLC chips and are faster too.
Interesting post folks
learnt a lot from here and will be installing my ESXi on a USB Drive for sure
'Technically speaking' ESXi 3.5 was never designed to run on flash. It was later (in its development and life cycle) supported by certain vendors to run off flash. It was until vSphere it was FULLY supported and is actually the RECOMMENDED method of installation.
Everyone is entitled to their own opinions and own research/experience but vSphere is 100% safe on USB and FULLY supported.
I should also add that USB drives have a data retention time limit.
Old good USB drives made from SLC chips had a 10 years data retention limit.
New cheap MLC drives can have as little as 1 year data retention limit.
Sadly, most manufacturers have stopped making SLC thumb drives in favor of the cheaper to make MLC ones.
The largest SLC thumb drives that were ever made topped at 4GB and production stopped around 2007.
Any drive made after 2007 is certain to be MLC.
If you have a cheap drive, it would be a good idea to re-load it from an image once every year or so (if you don't upgrade your ESXi install passed that time).
So, start hunting those old good SLC thumb drives if you want to hold onto a particular install for several years.
I'm coming in late to this conversation, but I have the same sort of configuration as those above and the VMware response (at least the sales arm) hasn't really made me feel good. I really don't care about the HDD v SD card/Flash reliability issue too much (if I have a quality SD card). To me the pros outweigh any cons by going Flash/SD.
The bigger deal is the support. Are user installed SD cards supported? Right now I'm configuring a new R710 with the internal SD reader module and my support is through VMware and not Dell. I was told (by Vmware sales) that VMware does NOT support SD card installs of ESXi unless it is installed thru the OEM directly. But that seems strange. I have Essentials so I don't get support unless I pay $300 per incident anyway. But I just want to make sure that IF I call VMware they don't hang up on me because I use an SD card to boot ESXi. I really like installing on flash/SD since I think the management is far superior to local HDD installs (as mentioned in previous posts). Unfortunately, now I don't know what to think regarding support... There is no mention in the HCL of recommended or supported Flash or SD cards which is really horrible exclusion IMO since the ESXi 4 U1 install documentation refers you to the HCL for Flash devices that are supported. Seems like there a lot of confusion about this issue from the support standpoint(?) Can anyone clear this up once and for all? I wish VMware would.