14 Replies Latest reply on Sep 27, 2013 4:58 PM by Drumboy444

    How to disable VMDirectPath and remove controllers

    sdeshpande Novice

      Hello,

       

      I  successfully enabled VMDirectPath for USB controller and unfortunately  also for SATA AHCI controller. As SATA AHCI is also enabled, I can not  see anymore my locally attached disk and so the datastores.

       

      1. Even if I try to disble it and try to reboot after that, it comes again "Enabled", and no matter what do.

      2.  /etc/vmware/exs.conf,  I changed passthru devices to vmkernel, but  after reboot, it again comes again as passthru. No change at all.

       

      I do not see VMkernel.Boot.noIOMMU option in Software -> Advanced Settings -> VMkernel.

       

      Host: Dell PowerEdge T110 II, ESXi 5.

       

      Please suggest.

       

      Many thanks in advance.

       

      Sameer

        • 1. Re: How to disable VMDirectPath and remove controllers
          VTsukanov Virtuoso

          Hi

           

          try to edit file /etc/vmware/esx.conf from console
          In esx.conf should be a string of the form "/device/000:31.2/owner = "passthru",
          1) delete this string
          2) after rescan storage adapters from VI client, you should see your storage adapter,
          3) after "Rescan All" on tab storages you should see all your storages.
          Now you will need store the new configurations to bootbank partitions (exec "/sbin/auto-backup.sh" from console) and copy state.tgz to your old bootbank partitions (should be mounted after successfull step 3 to /vmfs/volumes/....)
          1 person found this helpful
          • 2. Re: How to disable VMDirectPath and remove controllers
            sdeshpande Novice

            Thanks for the reply. After deleting the string (complete line itself with passthru) from /etc/vmware/esx.conf file, devices still appear in passthru.

             

            1.

            Original file contents:

             

            /advUserOptions/options[0018]/type = "int"
            /device/000:000:26.0/device = "1c2d"
            /device/000:000:26.0/owner = "passthru"
            /device/000:000:26.0/vendor = "8086"
            /device/000:000:29.0/device = "1c26"
            /device/000:000:29.0/owner = "passthru"
            /device/000:000:29.0/vendor = "8086"
            /device/000:000:31.2/device = "1c02"
            /device/000:000:31.2/owner = "passthru"
            /device/000:000:31.2/vendor = "8086"
            /device/000:000:31.2/vmkname = "vmhba0"
            /device/000:001:00.0/vmkname = "vmnic0"
            /device/000:001:00.1/vmkname = "vmnic1"
            /device/000:002:00.0/vmkname = "vmnic2"

             

            Modifed esx.conf

             

            /advUserOptions/options[0018]/type = "int"
            /device/000:000:26.0/device = "1c2d"
            /device/000:000:26.0/vendor = "8086"
            /device/000:000:29.0/device = "1c26"
            /device/000:000:29.0/vendor = "8086"
            /device/000:000:31.2/device = "1c02"
            /device/000:000:31.2/vendor = "8086"
            /device/000:000:31.2/vmkname = "vmhba0"
            /device/000:001:00.0/vmkname = "vmnic0"
            /device/000:001:00.1/vmkname = "vmnic1"
            /device/000:002:00.0/vmkname = "vmnic2"

             

             

            2. Rescan of adapters.

             

            /etc/vmware # esxcli storage core  adapter list
            HBA Name  Driver     Link State  UID            Description
            --------  ---------  ----------  -------------  ----------------------
            vmhba33   iscsi_vmk  online      iscsi.vmhba33  iSCSI Software Adapter
            /etc/vmware #
            /etc/vmware #
            /etc/vmware # esxcli storage core  adapter rescan --all
            /etc/vmware #
            /etc/vmware #
            /etc/vmware # esxcli storage core  adapter list
            HBA Name  Driver     Link State  UID            Description
            --------  ---------  ----------  -------------  ----------------------
            vmhba33   iscsi_vmk  online      iscsi.vmhba33  iSCSI Software Adapter
            /etc/vmware #

             

            3. Rescan all from the storage tab (ESX client) does not show either :-(

             


            • 3. Re: How to disable VMDirectPath and remove controllers
              VTsukanov Virtuoso

              1) try to remove

              /device/000:000:31.2/device = "1c02"
              /device/000:000:31.2/vendor = "8086"
              /device/000:000:31.2/vmkname = "vmhba0"

              2) Check "vmkload_mod -l" that you have achi module loaded (something like "vmkload_mod -l | grep achi"), if it not loaded load it by command "vmkload_mod ahci"

              3) Try rescan storage adapters

              ....

              1 person found this helpful
              • 4. Re: How to disable VMDirectPath and remove controllers
                sdeshpande Novice

                1. Removed the lines as mentioned.

                 

                2. Loaded the AHCI module which was not loaded before.

                 

                /etc/vmware # vmkload_mod -l | grep ahci
                /etc/vmware #
                /etc/vmware # vmkload_mod ahci
                Module ahci loaded successfully
                /etc/vmware #

                 

                3. Rescan did not change

                 

                /etc/vmware # esxcli storage core  adapter list
                HBA Name  Driver     Link State  UID            Description
                --------  ---------  ----------  -------------  ----------------------
                vmhba33   iscsi_vmk  online      iscsi.vmhba33  iSCSI Software Adapter
                /etc/vmware #
                /etc/vmware #
                /etc/vmware # esxcli storage core  adapter rescan --all
                /etc/vmware #
                /etc/vmware #
                /etc/vmware # esxcli storage core  adapter list
                HBA Name  Driver     Link State  UID            Description
                --------  ---------  ----------  -------------  ----------------------
                vmhba33   iscsi_vmk  online      iscsi.vmhba33  iSCSI Software Adapt
                er

                 

                 

                 

                4.  Devices still appear as passthru

                 

                /etc/vmware # vmkchdev -l


                000:000:00.0 8086:0108 1028:04de vmkernel
                000:000:26.0 8086:1c2d 1028:04de passthru
                000:000:28.0 8086:1c10 0000:0000 vmkernel PCIe RP[000:000:28.0]
                000:000:28.4 8086:1c18 0000:0000 vmkernel PCIe RP[000:000:28.4]
                000:000:29.0 8086:1c26 1028:04de passthru
                000:000:30.0 8086:244e 0000:0000 vmkernel
                000:000:31.0 8086:1c52 1028:04de vmkernel
                000:000:31.2 8086:1c02 1028:04de passthru vmhba0
                000:000:31.3 8086:1c22 1028:04de vmkernel
                000:001:00.0 8086:10c9 8086:a03c vmkernel vmnic0
                000:001:00.1 8086:10c9 8086:a03c vmkernel vmnic1
                000:002:00.0 14e4:165a 1028:04de vmkernel vmnic2
                000:003:03.0 102b:0532 1028:04de vmkernel

                 

                5. Assigned the devices to vmkernel

                 

                /etc/vmware #
                /etc/vmware # vmkchdev -v 000:000:26.0
                /etc/vmware # vmkchdev -v 000:000:29.0
                /etc/vmware # vmkchdev -v 000:000:31.2

                 

                6. List again

                 

                /etc/vmware #
                /etc/vmware # vmkchdev -l
                000:000:00.0 8086:0108 1028:04de vmkernel
                000:000:26.0 8086:1c2d 1028:04de vmkernel
                000:000:28.0 8086:1c10 0000:0000 vmkernel PCIe RP[000:000:28.0]
                000:000:28.4 8086:1c18 0000:0000 vmkernel PCIe RP[000:000:28.4]
                000:000:29.0 8086:1c26 1028:04de vmkernel
                000:000:30.0 8086:244e 0000:0000 vmkernel
                000:000:31.0 8086:1c52 1028:04de vmkernel
                000:000:31.2 8086:1c02 1028:04de vmkernel vmhba0
                000:000:31.3 8086:1c22 1028:04de vmkernel
                000:001:00.0 8086:10c9 8086:a03c vmkernel vmnic0
                000:001:00.1 8086:10c9 8086:a03c vmkernel vmnic1
                000:002:00.0 14e4:165a 1028:04de vmkernel vmnic2
                000:003:03.0 102b:0532 1028:04de vmkernel
                /etc/vmware #

                 

                7. Adaper list without rescan, All adapter are available again.

                 

                /etc/vmware # esxcli storage core  adapter list
                HBA Name  Driver     Link State  UID            Description
                --------  ---------  ----------  -------------  ---------------------------------------------------------------------
                vmhba38   ahci       link-n/a    sata.vmhba38   (0:0:31.2) Intel Corporation Cougar Point 6 port SATA AHCI Controller
                vmhba0    ahci       link-n/a    sata.vmhba0    (0:0:31.2) Intel Corporation Cougar Point 6 port SATA AHCI Controller
                vmhba33   iscsi_vmk  online      iscsi.vmhba33  iSCSI Software Adapter
                vmhba34   ahci       link-n/a    sata.vmhba34   (0:0:31.2) Intel Corporation Cougar Point 6 port SATA AHCI Controller
                vmhba35   ahci       link-n/a    sata.vmhba35   (0:0:31.2) Intel Corporation Cougar Point 6 port SATA AHCI Controller
                vmhba36   ahci       link-n/a    sata.vmhba36   (0:0:31.2) Intel Corporation Cougar Point 6 port SATA AHCI Controller
                vmhba37   ahci       link-n/a    sata.vmhba37   (0:0:31.2) Intel Corporation Cougar Point 6 port SATA AHCI Controller

                 

                8. Rescan the adapter list

                 

                /etc/vmware # esxcli storage core  adapter rescan --all
                /etc/vmware #

                 

                9. All the adapters are show as in no. 6

                 

                /etc/vmware # esxcli storage core  adapter list
                HBA Name  Driver     Link State  UID            Description
                --------  ---------  ----------  -------------  ---------------------------------------------------------------------
                vmhba38   ahci       link-n/a    sata.vmhba38   (0:0:31.2) Intel Corporation Cougar Point 6 port SATA AHCI Controller
                vmhba0    ahci       link-n/a    sata.vmhba0    (0:0:31.2) Intel Corporation Cougar Point 6 port SATA AHCI Controller
                vmhba33   iscsi_vmk  online      iscsi.vmhba33  iSCSI Software Adapter
                vmhba34   ahci       link-n/a    sata.vmhba34   (0:0:31.2) Intel Corporation Cougar Point 6 port SATA AHCI Controller
                vmhba35   ahci       link-n/a    sata.vmhba35   (0:0:31.2) Intel Corporation Cougar Point 6 port SATA AHCI Controller
                vmhba36   ahci       link-n/a    sata.vmhba36   (0:0:31.2) Intel Corporation Cougar Point 6 port SATA AHCI Controller
                vmhba37   ahci       link-n/a    sata.vmhba37   (0:0:31.2) Intel Corporation Cougar Point 6 port SATA AHCI Controller

                 

                10. All the adapers and VM are shown.

                 

                But as I know, vmkchdev -v is not persistent and it will go back to passthru once rebooted. How shall I make persistent so that it will show me all the adaperts and not in passthru again?

                 

                Thanks

                • 5. Re: How to disable VMDirectPath and remove controllers
                  VTsukanov Virtuoso

                  You need create new state.tgz (/sbin/autobackup.sh should be create it, take a look at "cat /sbin/auto-backup.sh") and place it to /bootbank & /altbootbank (should be mouted in /vmfs/volume/... you can determine their by files boot.cfg, state.tgz ).

                  • 6. Re: How to disable VMDirectPath and remove controllers
                    sdeshpande Novice

                    I created a new state.tgz using /sbin/autobackup.sh as mentioned. The state.tgz file was old in /albootbank and copied over it with the new one. Rebooted the host and all went fine. Thanks for the immediate help. All the devices removed from passthru and finally disabled VMDirectPath :-) I could recover all my VMs.

                     

                    Many thanks

                     

                    Sameer

                    • 7. Re: How to disable VMDirectPath and remove controllers
                      VTsukanov Virtuoso

                      No problem, could you please mark the thread as answered and consider to mark correct and/or helpfull answers.

                      • 8. Re: How to disable VMDirectPath and remove controllers
                        sdeshpande Novice

                        I already marked answered with points :-)

                        • 10. Re: How to disable VMDirectPath and remove controllers
                          kikistan Novice

                          I have tried everything you explain.

                          Success till .... state.tgz

                           

                          I copied from sbin the auto-backup.sh to /vmfs/volume/XXX

                           

                          I don't have a clue where is the bootbank and altbootbank

                           

                          When I reboot ... confi is lost

                           

                          I need some help.

                           

                          Thnaks

                          • 11. Re: How to disable VMDirectPath and remove controllers
                            VTsukanov Virtuoso

                            You can identify bootbank & altbootbank partitions by size (~ 250Mb in output command "df -h").

                            • 12. Re: How to disable VMDirectPath and remove controllers
                              Drumboy444 Novice

                              I too have the Dell T110 II with ESXI (5.5 to be exact but have the same problem with 5.0).  No matter what I do in configuration I have the error that says Host does not support passthrough configuration.  Can you tell me what BIOS version you have on your machine for that might be my problem.  I have virtualization enabled on the BIOS but ESXI still says on the summary tab "The hardware of the host does not support DirectPath I/O.  My CPU CPU is an Intel Core i3-3220 3.3ghz.  What did you need to do special just to pass the USB ports through.  I want to use an external USB drive array and pass it to my virualized file server but for the moment have to keep using one of the internal SATA connections.  I want to use all of the internal SATA connections for raiding my ESXI build and have all other drives be external.

                               

                              Any help would be appreciated.

                               

                              Thanks,

                              Stephen Bailey

                              • 13. Re: How to disable VMDirectPath and remove controllers
                                sdeshpande Novice

                                If I am not wrong, do you want to connect USB device (external hard drive) to VM? If yes, you do not need to pass USB ports through. You can plugin external USB device to main server, i.e PowerEdge and add USB controller to VM (VM settings and so on) and then external device will appear to VM.

                                 

                                If my understanding is correct, follow the link - http://pubs.vmware.com/vsphere-50/index.jsp?topic=%2Fcom.vmware.vsphere.vm_admin.doc_50%2FGUID-3E6E58AA-2AD4-4370-9955-07E5FBDE800D.html

                                 

                                HTH

                                 

                                Sameer

                                • 14. Re: How to disable VMDirectPath and remove controllers
                                  Drumboy444 Novice

                                  No this does not work, according to that link you provided and the links on that page these must be passthrough USB so I believe these ports need to show on Configuration and advanced settings.  I did just try adding all available USB adapters to my VM but it never recognized my drive plugged into the host.