VMware Cloud Community
jrsmith61
Contributor
Contributor

enabling iSCSI software adapter on Lenovo System x Custom Image for ESXi 6.5a causes problems with persistent scratch location

I have installed the recently released Lenovo System X Custom Image for ESXi 6.5a on three separate physical servers and one nested ESXi image. When I enable the iSCSI software adapter and reboot, I see the following warning on each host

System logs on host ... are stored on non-persistent storage

The ScratchConfig.ConfiguredScratchLocation advanced system setting is pointing to the 4 GB vfat partition that was created at installation time, but ScratchConfig.CurrentScratchLocation is pointing to /scratch which I believe is in a ramdisk.

I reviewed the log files in /scratch/log and found the following entry in syslog.log

2017-03-13T14:47:27Z jumpstart[66440]: using /vmfs/volumes/58c2ed55-f98c6822-a783-e41f1344b0f4 as /scratch

However, less than 1 second later, I find errors in other log files about non-persistent storage

syslog.log:2017-03-13T14:47:30Z jumpstart[65896]: executing start plugin: scratch-storage

vmkernel.log:2017-03-13T14:47:30.246Z cpu10:65942)Activating Jumpstart plugin scratch-storage.

vmkernel.log:2017-03-13T14:47:30.447Z cpu3:66558)ALERT: Logs are stored on non-persistent storage.  Consult product documentation to configure a syslog server or a scratch partition.

vmkernel.log:2017-03-13T14:47:30.470Z cpu10:65942)Jumpstart plugin scratch-storage activated.

vmksummary.log:2017-03-13T14:47:30.447Z cpu3:66558)ALERT: Logs are stored on non-persistent storage.  Consult product documentation to configure a syslog server or a scratch partition.

vmkwarning.log:2017-03-13T14:47:30.447Z cpu3:66558)ALERT: Logs are stored on non-persistent storage.  Consult product documentation to configure a syslog server or a scratch partition.

vobd.log:2017-03-13T14:47:28.638Z: [GenericCorrelator] 43572261us: [vob.user.scratch.partition.unconfigured] No scratch partition has been configured. Recommended scratch partition size is 4095 MiB.

vobd.log:2017-03-13T14:47:28.638Z: [UserLevelCorrelator] 43572261us: [vob.user.scratch.partition.unconfigured] No scratch partition has been configured. Recommended scratch partition size is 4095 MiB.

vobd.log:2017-03-13T14:47:28.638Z: [UserLevelCorrelator] 43572494us: [esx.problem.scratch.partition.unconfigured] No scratch partition has been configured. Recommended scratch partition size is 4095 MiB.

I assume there is something wrong with the Lenovo vibs, but I don't know which one is causing the problem. VMware's 6.5a build works just fine.

Any suggestions?

Reply
0 Kudos
8 Replies
simonkf
Contributor
Contributor

Did you ever manage to resolve this?

I have the same issue.

TIA

Reply
0 Kudos
ITaaP
Enthusiast
Enthusiast

I've always moved my Syslog to one of the datastores which will get rid of that message you are receiving.

Syslog.global.logDir - [Datastore01] Logs

Syslog.global.logDirUnique - Checked

https://tactsol.com https://vmware.solutions
TechMassey
Hot Shot
Hot Shot

Hmmm, well it looks like the custom ISO from Lenovo has a preset on the scratch config location. No big deal.

In fact, since you have iSCSI. I highly recommend creating a dedicated scratch datastore for scratch/syslog. The advantages are huge and VMware support loves you when you can just grab those logs from the datastore instead of being tied to local host storage.

With vSphere 6.5, it is easier then ever to configure the global scratch config. Go to advanced settings on the host, filter by "scratch" . Choose the Syslog.global.logDir and put in the datastore of choice. Any issues, just let me know.


Please help out! If you find this post helpful and/or the correct answer. Mark it! It helps recgonize contributions to the VMTN community and well me too 🙂
simonkf
Contributor
Contributor

We're using a virtual SAN type solution so unfortunately we can't migrate the logs to a central datastore.

What are the impacts of moving just the logs to a local directory? I thought this could cause issues patching if scratch didn't have enough space and was left as the default?

Reply
0 Kudos
UmeshAhuja
Commander
Commander

Hi,

You receive this message when your system stores its log files on non-persistent storage.

Example

When you use the advanced system parameter Syslog.global.logDir to configure the system log location, if you configure the system log location as temporary, for example [] /tmp, you get the esx.problem.syslog.nonpersistent message.

OR

When the location for system logs is not configured, your system uses the scratch partition by default. If the scratch location is set to nonpersistent storage, such as /tmp, you get the esx.problem.syslog.nonpersistent message.

Can you try to cross verify the location of the system logs and let us know ?

Verifying the Location of System Logs in vSphere Web Client

  1. Browse to the host in the vSphere Web Client navigator.
  2. Click the Manage tab, then click Settings.
  3. Under System, click Advanced System Settings.
  4. Ensure that Syslog.global.logDir points to a persistent location.
  5. If the field Syslog.global.logDir is empty or points to a scratch partition, make sure that the field ScratchConfig.CurrentScratchLocation shows a location on persistent storage.

Note: You must reboot the host for the changes to take effect.

Note: To log to a datastore, the Syslog.global.logDir entry should be in the format of [Datastorename]/foldername. To log to the scratch partition set in the ScratchConfig.CurrentScratchLocation, the format is blank or []/foldername.

Thanks n Regards
Umesh Ahuja

If your query resolved then please consider awarding points by correct or helpful marking.
Reply
0 Kudos
TechMassey
Hot Shot
Hot Shot

You should be fine, a lot of VMware Admins use the local storage for scratch config and typically it is the default 10K or 15K drive that came with the server. Because ESXi 6.5 for instance only needs 1GB for the installation and 4GB on the scratch location, most default drives have sufficient space. If you run off a USB flash drive, then you need to consider ESXi storage minimums and potential scratch/dump file growth.


Please help out! If you find this post helpful and/or the correct answer. Mark it! It helps recgonize contributions to the VMTN community and well me too 🙂
Reply
0 Kudos
simonkf
Contributor
Contributor

The servers have no local drives. It's all USB.

I have an open ticket with support. Let's see what happens with that.

Reply
0 Kudos
ITaaP
Enthusiast
Enthusiast

With this virtual SAN type solution, do you not have any Datastores added to your ESXi hosts? As long as each ESXi host has at least one datastore, then you can move your logs. I had the same issue with Dell servers that used 4 GB USB drives.

Before:

Default_Syslog.png

After:

Updated_Syslog.png

https://tactsol.com https://vmware.solutions
Reply
0 Kudos