11 Replies Latest reply on Sep 25, 2017 3:45 PM by Graham

    how to boot vSAN from local disk

    suhag79 Enthusiast

      Hi,

       

      have a question about booting device for vSAN node. suppose, if we go for vSAN ready node with 400 GB SSD, 9*1.2 TB capacity disk and 64 GB of SD card. We still want to use the local disk (which is not participating in disk group) to boot the ESXi. how can we do it ? obviously, we should not use capacity tier disk 1.2 TB for this....so what are the other option ? i also dont want to use remote log server to store any log....

       

      can we ask vendor to add 2*300 GB of SAS disk (to boot ESXi) while we order for vSAN ready node ? 

       

      regards,

        • 1. Re: how to boot vSAN from local disk
          Graham Expert
          User ModeratorsvExpert

          My understanding is that you need a separate storage controller for the boot disks as per this KB: Best practices when using vSAN and non-vSAN disks with the same storage controller (2129050) | VMware KB

           

          However SD/USB/SATADOM are more preferred options depending on where you want to log to and the size of the cluster & amount of memory in the host.

           

          This document should give you more detailed information: Using Boot Devices and Virtual SAN

          • 2. Re: how to boot vSAN from local disk
            suhag79 Enthusiast

            Yes, i have gone through this link Using Boot Devices and Virtual SAN.

             

            Since i want some persistence device to store the boot and other logs, i am looking for local disk. So question is, can we customize the vSAN ready node SKU hardware configuration as per our requirement ?

             

            other question - SATADOM is a part of vSAN ready node ? how big it is ? and its works like local disk only? i.e persistence ?

             

            Regards,

            • 3. Re: how to boot vSAN from local disk
              Graham Expert
              User ModeratorsvExpert

              Usually you can customise the ready nodes with the vendor but then they might no longer be classified as ready nodes for support purposes. So it might be worth speaking with your vendor about this.

               

              SATADOM SD and USB are all valid persistent storage locations for OS and logs. Simply select the best option based in your vSAN requirements.

               

              Hope this helps?

              • 4. Re: how to boot vSAN from local disk
                suhag79 Enthusiast

                 

                "SATADOM SD and USB are all valid persistent storage locations for OS and logs"

                 

                When you boot ESXi from a USB or SD device, log information and stack traces are lost on host reboot because the scratch partition is on a RAM drive. Use persistent storage for logs, stack traces and memory dumps.

                 

                this is what mention on,

                 

                Using Boot Devices and Virtual SAN

                 

                thats why i got confused. i need to check with vendor then how they will support if we customize the vSAN ready node config

                • 5. Re: how to boot vSAN from local disk
                  Graham Expert
                  User ModeratorsvExpert

                  Yes that is correct. Sorry my last statement was ment to refer to SATADOM.

                   

                  SD and USB of course work as persistent boot options but logs are stored on scratch as they are not suited for writing logs.

                   

                  There was a very good VMworld session last year about vSAN and logs. I'll try to find it and post it here for you. I know Cormac Horgan has a good blog post too if you need more information.

                  • 6. Re: how to boot vSAN from local disk
                    suhag79 Enthusiast

                    thanks a lot,

                     

                    i will check Cormac Horgan blog while you get the VMworld session data.

                    • 7. Re: how to boot vSAN from local disk
                      suhag79 Enthusiast

                      just for reference....

                       

                      just found this,

                      What You Can (and Cannot) Change in a vSAN ReadyNode™

                       

                      https://blogs.vmware.com/virtualblocks/2017/03/14/can-cannot-change-vsan-readynode/

                      • 8. Re: how to boot vSAN from local disk
                        TheBobkin Virtuoso
                        VMware EmployeesvExpert

                        Hello suhag79,

                         

                         

                        It is supported to log to a disk on the same controller but it is not best practise and is not supported for LSI3108-based controllers (e.g. H730)

                        "If the non-vSAN disks are in use for VMFS, the VMFS datastore should be used only for scratch, logging and coredumps."

                        kb.vmware.com/kb/2129050

                        However H730 can be used for boot from same controller as vSAN disks provided that the VMFS is removed after installation:

                        kb.vmware.com/kb/2136374

                         

                        This is the best resource I know of for comparison of the different options here:

                        blogs.vmware.com/virtualblocks/2016/03/18/virtual-san-design-considerations-for-booting-from-a-flash-device/

                         

                        Specifically this which sums up the pros and cons:

                        http://blogs.vmware.com/virtualblocks/files/2016/03/Boot_Comparison.png

                         

                        As per supported additions to ReadyNodes - adding an additional controller for a disk is supported, these don't need to be high performance devices.

                         

                        Though consider how many drive slots are available on the ReadyNodes you are looking at as losing a drive slot or two may limit further expansion, so SD for boot (RAID1 redundant pair if possible) and SATADOM for logs is a preferred option.

                         

                         

                        Bob

                        2 people found this helpful
                        • 9. Re: how to boot vSAN from local disk
                          suhag79 Enthusiast

                          Thanks,

                           

                          so the bottom line of using SSD (SAS or SATA) as a boot device is to lose HDD slot. we are fine with this in my environment.

                           

                          We are planning to add "2*UCS-SD120GBKS4-EB = 120 GB 2.5 inch Enterprise Value 6G SATA SSD (boot)" in vSAN ready node as boot device. Since we are exploring CISCO c240 as vSAN ready node, it will a "Cisco 12G SAS Modular Raid Controller".

                           

                          as per the KB "kb.vmware.com/kb/2129050" you and Graham shared with me,

                           

                          ESXi host installation is permitted on non-vSAN disks attached to same controller so we can install the ESXi on SATA SSD and we can use the same for storing the scratch, logging and coredumps. i dont have any recommendation/best practice of using Cisco 12G SAS Modular Raid Controller with vSAN and non-vSAN disk, as you provided for H730.

                           

                          i hope, this will work and is also comply with VMware std.

                           

                          thought ?

                          • 10. Re: how to boot vSAN from local disk
                            suhag79 Enthusiast

                            Hi,

                             

                            did you get a chance to check my last reply. That is the only pending query i have before we finalize the solution.

                             

                            regards,

                            • 11. Re: how to boot vSAN from local disk
                              Graham Expert
                              User ModeratorsvExpert

                              Hello.

                               

                              I cannot talk on behalf of Cisco regarding the need for a mother controller. I would suggest talking with your service provider or Cisco account manager to see what their requirements are. This means that you will have an official response regarding something critical to vSAN functioning correctly.

                               

                              Sorry I cannot be more helpful. Maybe others will have Cisco experiance with this area?