ESXi in SD card or standard hard disk

[ChevUribe]

Most of the VSAN ready nodes are using SD card for the ESXi partition.

From a VMware standpoint, would you recommend installing ESXi in an SD card?

Can we instead ask the vendor to include 2 x 300GB disk in Raid 1 config as ESXi partition?

Would VMware still support that since some vendors are saying that VMware will not support the machine since it is considered a non-VSAN ready node if the additional 2 disks are included.

thanks!

[admin]

Greetings!

Most of the VSAN ready nodes are using SD card for the ESXi partition.

From a VMware standpoint, would you recommend installing ESXi in an SD card?

I would say that It's not about VMware stand point. It's about how much do you want to spend for VMware vSAN deployment. Most of the vSAN ready nodes are using SD card for ESXi partition to lower the cost of vSAN deployment. For vSAN deployments, it makes most sense to install ESXi on an internal USB/SD card to avoid the unnecessary cost of local disks for ESXi partition. ESXi installation foot-print is very small and it does not make sense to waste entire disk for ESXi installation as you cannot partition your SSD or HDD(s) to use it for vSAN datastore because vSAN will only, and always, claim entire disks.

Can we instead ask the vendor to include 2 x 300GB disk in Raid 1 config as ESXi partition?

Would VMware still support that since some vendors are saying that VMware will not support the machine since it is considered a non-VSAN ready node if the additional 2 disks are included.

Yes, you can ask the vendor to include 2 x 300 GB in RAID-1 config as ESXi partition but only If you really want to install ESXi on a local disk. If you ask me, It would be an unnecessary cost for vSAN deployment.

There are two ways to build a Virtual SAN cluster:

1) Build your own based on certified components

2) Choose from list of Virtual SAN Ready Nodes

If you are asking vendor to include extra disks for ESXi installation then you are going with point (1) mentioned above and yes, VMware supports this type of configuration.

Hope this answers your queries.

_________________________

Was your question answered correctly? If so, please remember to mark your question as answered when you get the correct answer and award points to the person providing the answer. This helps others searching for a similar issue.

Cheers!

-Shivam

[ChevUribe]

we would like to include the hard disk since in Raid 1 configuration, ESXi is still usable even a hardware issue is encountered.

Unless VSAN ready nodes has 2 SD cards per host and is redundant if 1 SD card fails

[admin]

I would suggest to use SD card If your vendor provides hardware modules for redundancy at SD card level.

There are solutions available for SD card level RAID.

Example:

1) Dell Internal Dual SD Module (IDSDM)

1) HP Dual 8GB MicroSD Enterprise Midline USB.

These SD card modules provides data redundancy through a mirrored RAID 1 configuration and are used for booting embedded hypervisors like ESXi.

You can use this link to check vSAN ready nodes - http://vsanreadynode.vmware.com/RN/RN

Virtual SAN Compatibility Guide - http://www.vmware.com/resources/compatibility/pdf/vi_vsan_rn_guide.pdf

Hope this helps you deciding further.

_________________________

Was your question answered correctly? If so, please remember to mark your question as answered when you get the correct answer and award points to the person providing the answer. This helps others searching for a similar issue.

Cheers!

-Shivam

[ChevUribe]

the server will be cisco

[admin]

Cisco has similar controllers for SD card mirroring - Cisco FlexFlash SD Controller.

[danleejccs]

I've been using single SD cards and/or USB (cheap $10 ones) for ESXi installations for 7 years on dozens of hosts for critical production loads.  Being that ESXi only reads the disk on boot up and then loads and runs all operations from memory; the only time the RAID drives or SD/USB disk is written too again is when you make a change to your host configuration. From an admin experience standpoint; unless something drastically changes in the future with how ESXi operates, I'll never install ESXi on RAID drives again. Too costly and zero performance benefit and negligible redundancy gains as a host profile can easily be used to recover a failed SD Card on a fresh install.

Cheerio!

Dan

[zdickinson]

Good morning, agree on SD/USB; disagree on single.  For the small added cost of doing dual SD cards, I would do it.  Thank you, Zach.

[roman79]

Hi All,

Just a couple of links to support the decision of adding ESXi on SD cards:

• VMware support for USB/SD devices used for installing VMware ESXi (1010574) | VMware KB
• What happens when the ESXi boot device fails (SD card or usb).

I have had similar consideration before and ended up using the hardware vendor's SD cards in RAID1 configuration. No issues so far.

[virtualg_uk]

There was a great VMworld session on this I'm sure by Cormac Hogan but I cannot find the session ID!

Full deep-dive details here which will let you decide what is best for your configuration:

• https://blogs.vmware.com/virtualblocks/2016/03/18/virtual-san-design-considerations-for-booting-from...
• VSAN considerations when booting from USB/SD - CormacHogan.com

[roman79]

Thanks, mate! Valuable links.

[dhanarajramesh]

SD card is better but make sure you have forwarded you syslogs and core dump collection is to another centralized  servers. most of the time customers are choosing bigger drive for esxi installation bcz of they just leave all these syslog and coredump collection settings as default where it's configured to the same disks.

[sayho]

Hello~

I suggest SD or USB when you use less than 512GB memory Host.

but you must set location global logs.[scratch/log].

[FM19999999]

We setup our 2x300GB drive for the OS on a seperate RAID card so we can configure a RAID-1.

Then we leave the whole front side of our hosts (24 bays) free for vSAN.

I wanted to push us toward SD cards but the whole core dumps and log file getting wiped was an issue.

When you install ESXi on an sd card, the /scratch partition doesn’t get created on the sd card and symlinks /tmp/scratch on ramdisk.  The scratch directory is used for storing logs, temp files, and a handful of other things, that’s what’s not persistent.

Also the chassis we were going to go with didn't have redundant SD card slots.

Our long term goal is go with SATADOM .

[Bill_Oyler]

Also, I suggest looking into the "SSD Boot Drives" available from most vendors.  For example Dell charges about $115 for a 120GB SSD Boot Drive, which isn't a huge price delta from the cost of the old, small, slow SD-Cards.  The benefits of the SSD Boot Drives are:

- ESXi installation is blazing fast compared with SD-Cards.

- Plenty of room for scratch partition and diagnostic partitions.

- Better reliability than SD-Cards.

- No issues with ESXi hosts that have 512 GB+ RAM (which is more and more common).

- RAID 1 and hot-plug support (compared with SD-Cards which generally are not).

The price of these "SSD Boot Drives" keep going down so I expect these to become the de-facto standard in the near future. 

Note that VMware/Dell/EMC's turnkey VSAN solution VxRail uses "SATADOM" drives (64 GB) for booting ESXi, which is very similar to using "Boot SSD Drives."  Similar endurance, performance, and benefits.  I would highly recommend considering either SATADOM or "SSD Boot" drives.

Bill

[elerium]

There are some caveats with with Dell and using a SSD boot drive if the drive is hooked up to the H730 raid controller.

Best practices when using VSAN and non-VSAN disks with the same storage controller (2129050) | VMwar...

- If boot disk is RAID1 using H730, your VSAN setup must use RAID instead of passthrough

https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=21363...

- No VMFS partitions (including scratch partition/logging/coredump) can be created on the same raid controller

Just be aware of those or face H730 instability/crashes

[Bill_Oyler]

Those are some really great reminders on using the same PERC controller for boot and for vSAN.  I suppose the "ideal" solution is to use a 64 GB SATADOM SLC drive, which appears to leverage its own controller (does not use PERC) and is as fast as "regular" SSD drives, bigger than SD-Cards, and support the scratch partition for ESXi.  I know Duncan mentioned SATADOM SLC drives being supported over a year ago with vSAN, and it seems that is what VxRail is using also.  Here's the blurb on SATADOM for Dell R730 servers:

http://www.dell.com/support/manuals/us/en/19/poweredge-r730/SATADOM%20Techsheet_Pub/Installing-and-R...

Does anyone know how much one of these 64 GB SATADOM SLC drives costs?  I can't seem to find a price online...