13 Replies Latest reply on Jan 4, 2016 5:48 AM by BonnoB

    ql_ima.log file

    daboud Lurker

      Hello Everyone.

       

      I've discovered that a log file named ql_ima.log is growing over time.  Eventually, it gets big enough to fill the /tmp directory.  This of course causes problems with any other process that needs to write to the /tmp directory.  I've tried turning off verbose logging with "esxcfg-module --verbose 0 qla4xxx" which I think probably works for that module, but don't know if the ql_ima.log file belongs to that module.  There is a ql_ima_sdm.log file that seems to have a limit on size configured somewhere because the ql_ima_sdm.log_old file exists (always at 4.8MB).  Of note is that a "ql_ima.log_old" file does NOT exist.

       

      Does anyone know how I can disable this ql_ima.log file completely or at least cap it at a few MBs?  I don't believe I've been able to find the right person at VMware Technical Support that knows anything about this file or it's configuration.  If anybody knows anything, I would greatly appreciate it.

       

      ESXi 5.5 build 1331820

       

      Thanks.

        • 1. Re: ql_ima.log file
          SteveAntle Enthusiast

          Hello,

           

          I am having the same issue on all of my ESXi hosts:

           

          Product: VMware ESXi

          Version: 5.5.0

          Build: Releasebuild-1474528

          Update: 0

           

          The two files in question are:

           

          188154205 May 30 11:25 ql_ima.log

              3195746 May 30 11:25 ql_ima_sdm.log

           

          Located in /tmp directory.

           

          I am having issues now with VDP backups - one by one the host management interfaces are crashing during a VDP backup process  and the hosts disconnect from vCenter Server. This only started happening since the /tmp folder started to fill up (related? not sure, but /tmp having 0% free space is an issue for sure).

           

          Help anyone?

          • 2. Re: ql_ima.log file
            daboud Lurker

            Steve.

             

            I did get an answer from VMware.  You need to update the Qlogic driver.  I downloaded qla4xxx-634.55.28.0-1682448.zip and extracted it to get the off-line bundle file.  I then uploaded it to a datastore and ran these steps to fix the issue.  After I was finished, the ql_ima.log and accompanying files were gone (deleted or moved I don’t know).  But anyway, this fixed the issue of the /tmp directory filling up and causing me problems.

             

            Place host in Maintenance Mode

             

            Navigate to datastore directory

             

            # cd /vmfs/volumes/<Datastore name>

             

             

            List the files

             

            # ls

             

             

            Copy the offline bundle to /var/log/vmware

             

            # cp qla4xxx-634.55.28.0-offline_bundle-1682448.zip /var/log/vmware

             

             

            Install the driver

             

            # esxcli software vib install -d /var/log/vmware/qla4xxx-634.55.28.0-offline_bundle-1682448.zip

             

             

            Reboot

             

            # reboot

             

             

            Mark

            • 3. Re: ql_ima.log file
              daboud Lurker

              Steve.

               

              I did get an answer from VMware.  You need to update the Qlogic driver.  I downloaded qla4xxx-634.55.28.0-1682448.zip and extracted it to get the off-line bundle file.  I then uploaded it to a datastore and ran these steps to fix the issue.  After I was finished, the ql_ima.log and accompanying files were gone (deleted or moved I don’t know).  But anyway, this fixed the issue of the /tmp directory filling up and causing me problems.

               

               

              Place host in Maintenance Mode

               

               

               

              Navigate to datastore directory

               

               

               

              1. cd /vmfs/volumes/<Datastore name>

               

               

               

               

               

              List the files

               

               

               

              1. ls

               

               

               

               

               

              Copy the offline bundle to /var/log/vmware

               

               

               

              1. cp qla4xxx-634.55.28.0-offline_bundle-1682448.zip /var/log/vmware

               

               

               

               

               

              Install the driver

               

               

               

              1. esxcli software vib install -d /var/log/vmware/qla4xxx-634.55.28.0-offline_bundle-1682448.zip

               

               

               

               

               

              Reboot

               

               

               

              1. reboot

               

               

              Mark

              • 4. Re: ql_ima.log file
                SteveAntle Enthusiast

                Mark,

                 

                Thanks for the reply, the drive update worked like a charm!

                 

                Steve

                • 5. Re: ql_ima.log file
                  MatthewCEvans Novice

                  I had a number of hosts where the log files were being created in the /tmp directory but a number that didn't have the logs, even though everything was patched to the same version and same updates.  Turned out that these log files are also related to the QLogic iSCSI IMA Plug-IN.  To turn off logging from the QLogic iSCSI IMA Plug-IN, delete this file: /etc/ql_ima.cfg.  This information is from this QLogic PDF: http://filedownloads.qlogic.com/files/driver/79829/release_iscsi_adapter_driver_vmware_esx_esxi_41x.pdf.

                  • 6. Re: ql_ima.log file
                    HippoCoar Lurker

                    Mark,

                     

                    Thank you so much for this. We didn't even realize we had a problem with our /tmp directory until we tried to run a data collection tool that used that directory to run the script. Our /tmp directories were completely full before this ran.

                    • 7. Re: ql_ima.log file
                      krktech Lurker

                      I have this issue on a few hosts and many hosts that don't have the issue, all with similar setup, vSphere 5.5 and the same QLogic version. ( ima-qla4xxx  500.2.01.31-1vmw.0.3.040121).   So why are only a few hosts getting the ql_ima.log files created and filling up /tmp?   The only thing I can find is that the hosts with the issue have iSCSI connections to Nimble storage (hosts with only iSCSI to NetApp don't have the log files created).  

                       

                      Thanks to Mark for the fix.

                      • 8. Re: ql_ima.log file
                        Darkschneidr Novice

                        Interesting... I have HP servers with iscsi connections to Nimble as well, and I'm seeing it.

                        • 9. Re: ql_ima.log file
                          doubletwist13 Lurker

                          Add us to the list - we are connected to a Nimble storage and have this issue. We will try the driver update and see if that helps.

                          • 10. Re: ql_ima.log file
                            musc_county_is Novice

                            We recently had this issue, which caused a host to basically become unusable and needed to be physically powered off.

                             

                            We are running ESXi 5.5.0 2718055 on Dell R720 hosts and connect to an EqualLogic SAN.

                             

                            When the server was back online, the VMware support engineer that we worked with deleted the ql_ima.log and ql_ima.sdm.log files in the /tmp directory and mentioned that we needed to reboot the hosts "regularly" to help prevent this form happening, but did not offer any resolution or mention the solution that Mark posted.

                             

                            It would be nice to know if that solution would work for our systems or if there is something else that should be done.

                             

                            After running the host has been running for 3.5 days, this is what we show for file sizes:

                             

                            7695511 Aug 31 13:49 ql_ima.log

                            2946749 Aug 31 13:49 ql_ima_sdm.log

                            5000027 Aug 31 07:13 ql_ima_sdm.log_old

                            • 11. Re: ql_ima.log file
                              systimax Enthusiast

                              Same problem with us and same scenario. HP servers connected to a Nimble SAN.

                               

                              Do you guys suggest installing the drivers from Qlogic instead of updating from HPs embedded ESX driver repository or delete the CFG file that does the logging?

                              • 12. Re: ql_ima.log file
                                Darkschneidr Novice

                                What I wound up doing was to push out a new baseline esxi image that I knew didn't include that driver, which solved the issue for me. I don't have qlogic hardware in my hosts.

                                • 13. Re: ql_ima.log file
                                  BonnoB Enthusiast

                                  Just had contact with Nimble support. Indeed, for us Nimble users it is the same.

                                  Host in maintenance mode.

                                  ssh into host

                                  delete /tmp/ql_ima* log files,

                                  delete /etc/ql_ima.cfg file (file existence is responsible for creating log files)

                                  update driver (download it from MyVMware, instructions see other mesage)

                                  reboot host

                                  Exit maintenance mode