how to boot vSAN from local disk

[suhag79]

Hi,

have a question about booting device for vSAN node. suppose, if we go for vSAN ready node with 400 GB SSD, 9*1.2 TB capacity disk and 64 GB of SD card. We still want to use the local disk (which is not participating in disk group) to boot the ESXi. how can we do it ? obviously, we should not use capacity tier disk 1.2 TB for this....so what are the other option ? i also dont want to use remote log server to store any log....

can we ask vendor to add 2*300 GB of SAS disk (to boot ESXi) while we order for vSAN ready node ? 

regards,

[virtualg_uk]

My understanding is that you need a separate storage controller for the boot disks as per this KB: Best practices when using vSAN and non-vSAN disks with the same storage controller (2129050) | VMwar...

However SD/USB/SATADOM are more preferred options depending on where you want to log to and the size of the cluster & amount of memory in the host.

This document should give you more detailed information: Using Boot Devices and Virtual SAN

[suhag79]

Yes, i have gone through this link Using Boot Devices and Virtual SAN.

Since i want some persistence device to store the boot and other logs, i am looking for local disk. So question is, can we customize the vSAN ready node SKU hardware configuration as per our requirement ?

other question - SATADOM is a part of vSAN ready node ? how big it is ? and its works like local disk only? i.e persistence ?

Regards,

[virtualg_uk]

Usually you can customise the ready nodes with the vendor but then they might no longer be classified as ready nodes for support purposes. So it might be worth speaking with your vendor about this.

SATADOM SD and USB are all valid persistent storage locations for OS and logs. Simply select the best option based in your vSAN requirements.

Hope this helps?

[suhag79]

"SATADOM SD and USB are all valid persistent storage locations for OS and logs"

When you boot ESXi from a USB or SD device, log information and stack traces are lost on host reboot because the scratch partition is on a RAM drive. Use persistent storage for logs, stack traces and memory dumps.

this is what mention on,

Using Boot Devices and Virtual SAN

thats why i got confused. i need to check with vendor then how they will support if we customize the vSAN ready node config

[virtualg_uk]

Yes that is correct. Sorry my last statement was ment to refer to SATADOM.

SD and USB of course work as persistent boot options but logs are stored on scratch as they are not suited for writing logs.

There was a very good VMworld session last year about vSAN and logs. I'll try to find it and post it here for you. I know Cormac Horgan has a good blog post too if you need more information.

[suhag79]

thanks a lot,

i will check Cormac Horgan blog while you get the VMworld session data.

[suhag79]

just for reference....

just found this,

What You Can (and Cannot) Change in a vSAN ReadyNode™

https://blogs.vmware.com/virtualblocks/2017/03/14/can-cannot-change-vsan-readynode/

[TheBobkin]

Hello suhag79,

It is supported to log to a disk on the same controller but it is not best practise and is not supported for LSI3108-based controllers (e.g. H730)

"If the non-vSAN disks are in use for VMFS, the VMFS datastore should be used only for scratch, logging and coredumps."

kb.vmware.com/kb/2129050

However H730 can be used for boot from same controller as vSAN disks provided that the VMFS is removed after installation:

kb.vmware.com/kb/2136374

This is the best resource I know of for comparison of the different options here:

blogs.vmware.com/virtualblocks/2016/03/18/virtual-san-design-considerations-for-booting-from-a-flash-device/

Specifically this which sums up the pros and cons:

As per supported additions to ReadyNodes - adding an additional controller for a disk is supported, these don't need to be high performance devices.

Though consider how many drive slots are available on the ReadyNodes you are looking at as losing a drive slot or two may limit further expansion, so SD for boot (RAID1 redundant pair if possible) and SATADOM for logs is a preferred option.

Bob

[suhag79]

Thanks,

so the bottom line of using SSD (SAS or SATA) as a boot device is to lose HDD slot. we are fine with this in my environment.

We are planning to add "2*UCS-SD120GBKS4-EB = 120 GB 2.5 inch Enterprise Value 6G SATA SSD (boot)" in vSAN ready node as boot device. Since we are exploring CISCO c240 as vSAN ready node, it will a "Cisco 12G SAS Modular Raid Controller".

as per the KB "kb.vmware.com/kb/2129050" you and Graham shared with me,

ESXi host installation is permitted on non-vSAN disks attached to same controller so we can install the ESXi on SATA SSD and we can use the same for storing the scratch, logging and coredumps. i dont have any recommendation/best practice of using Cisco 12G SAS Modular Raid Controller with vSAN and non-vSAN disk, as you provided for H730.

i hope, this will work and is also comply with VMware std.

thought ?

[suhag79]

Hi,

did you get a chance to check my last reply. That is the only pending query i have before we finalize the solution.

regards,

[virtualg_uk]

Hello.

I cannot talk on behalf of Cisco regarding the need for a mother controller. I would suggest talking with your service provider or Cisco account manager to see what their requirements are. This means that you will have an official response regarding something critical to vSAN functioning correctly.

Sorry I cannot be more helpful. Maybe others will have Cisco experiance with this area?