"Server Error" when attempting to view Activity\Jobs

[MrCheesecake]

Good afternoon!

I manage a few different AppVol environments and most seem to have an issue where clicking on the Jobs tab under Activity in the AppVolumes management interface results in a "Server Error".  In the System Messages tab, we see an entry of "Unhandled request exception:  undefined method 'titleize' for nil:nilClass".

After poking around the SQL database, I think I see the problem but wanted to see if anyone has dealt with this before.

SQL Table in Question:  dbo.job_logs

Working System:  Row id matches the 'id' column.  In AppVol 2.18, we have 21 rows/jobs- last job is id 21.

Angry System:  Row 21 has an 'id' of 22 and the second-to-last job is id 20 (21 is not present).

Because opening support cases is almost as much fun as a trip to the dentist, I'm thinking about backing up the database and changing the out-of-sequence ID to see if that resolves the issue.  I'm wondering if anyone else has experienced the same thing and if this may be a course of action that resolves it.

Thanks!

[sjesse]

Use at your own risk, but I had something similar once, and I just cleared out all the jobs. Look at

VMware App Volumes Troubleshooting

at page 35, and at 

VMware Knowledge Base

you can get on the ruby console and run the command to delete any jobs, I'm not sure if the pending ones are your problem or not, but it worked in my case. I'd do this after making sure you have backups.

[MrCheesecake]

Thanks, for the link-  We have some stuck jobs in another environment so that will come in handy!

We started poking around the DB this morning (after creating a backup!) and found the following:

Changing the key/id number did not matter.

However... We found another inconsistency.  On the angry systems, there was a job called 'SweepVMsJob'.  On the happy systems, that value was 'SweepVmsJob'.

Apparently, the calls to the database are case sensitive and, for some reason, the value changed or was not updated with our last upgrade.  Changing the value and restarting the AppVol service resulted in the jobs to be displayed correctly.

Weird!

[sjesse]

Open an SR, I think the jobs are the Ruby names which are case sensitive, so you may be running into a bug.

[aejgd]

This resolved the issue for us and we didn't even need to restart the services.

[julatoski · Accepted solution]

MrCheesecake​,

You are exactly right. This noisy, yet benign issue is caused by two jobs (SweepVMsJob and RefreshVMMJob) that were renamed in 2.14. The database schema is updated upon upgrading the first App Volumes Manager (AVM) server.  The issue can happen when all AVMs aren't powered down when upgrading from 2.13.x to 2.14 or later, and one or both of these jobs is created in the updated database by an older manager.  In this case, the new AVMs do not know how to process these jobs.

While in practice I cannot endorse editing the database without specific guidance from our support engineers, I can recommend that any customers having this issue to please call our global support team and they will be able to provide guidance on how to resolve the issue.

Generally, this type of upgrade issue was resolved in 2.18 with the addition of a rolling upgrade feature.  Please also note that feature is not supported for major version upgrades such as the upgrade from 2.18 to App Volumes 4.  Please review VMware App Volumes 2.18 Release Notes and Upgrade App Volumes Manager documentation respectively for more details.

Thanks and good catch!

[aejgd]

What is the correct name for these jobs now in version 2.18?

Thanks

[julatoski]

SweepVmsJob

RefreshVmmJob

[aejgd]

We do not have one named RefreshVmmJob - is that an issue?

Also what are the default time settings for all of these scheduled jobs?

Thanks

[julatoski]

Hi aejgd​

Pending activities (delayed_jobs) are ephemeral and only exist in a queue for the duration they are schedule and executed.  It sounds like you have no cause for alarm.

More importantly, to ensure you are not invalidating supportability, if you are having an issue I recommend calling our global support team as they will be able to assess your situation and help with a prescribed solution.

Thank you.