VMware Communities: BIG bug in ESX 3.5 Update 2 - If you're using 3.5u2 read this now! -

Reply Re: BIG bug in ESX 3.5 Update 2 - If you're using 3.5u2 read this now! - A general system error occurred: Internal Error Aug 12, 2008 2:05 AM

in response to: robnik

LudoS 7 posts since
Aug 12, 2008

Hi,

Luckily only 2 out of our 18 servers (the two I recieved last week) are running 3.5u2... The rest of the infrastructure is on 3.5U1 (95350).

It seems that the bug is affecting only the destination server (i.e. if the target is 3.5U2) because I was able to move out the 3+4 VM I had on the 3.5U2 servers back to 3.5U1 servers using VMotion and put the 3.5U2 in maintenance mode and bring them out of the DRS/HA cluster.

I just prefer not to imagine what could have happen if I finished the upgrade as I planned and the 200+ VM's we have would have been affected .

What really piss me off is the lack of communication from VMware directly as they have full lists of customers who registrated licenses and downloaded the binaries !!!! For cases such as this one I would have expected some kind of direct communication notifying those of us who luckily could benefit from the lessons learned from our colleagues in Australia who were hit first.

Thanks to all early publishers for sharing their experiences and keeping us informed.

BR Ludovic

Reply Re: BIG bug in ESX 3.5 Update 2 - If you're using 3.5u2 read this now! - A general system error occurred: Internal Error Aug 12, 2008 2:12 AM

in response to: FrancWest

JohnTroyer 946 posts since
Aug 30, 2005 VMware

Hi FrancWest,

Everyone is mobilized here at VMware. mjlin, who posted in this thread several hours ago, is the product manager. Support knows what is going on. Someone else has posted our first communication here on this thread (patch should be available within 36 hours). Unfortunately I also can't access the kb, but I assume that posted message is from the kb.

I know we're preparing additional communication, so check that kb and expect more from us as we have more information. I'm sorry we weren't able to reach out to everyone directly yet.

John

Reply Re: BIG bug in ESX 3.5 Update 2 - If you're using 3.5u2 read this now! - A general system error occurred: Internal Error Aug 12, 2008 2:14 AM

in response to: mjlin

deploylinux 59 posts since
Oct 20, 2004

Not to be argumentitive, but 36hrs is way unreasonable to make customers wait. ESX is supposed to be an enterprise product. Enteprise products usually have 4hr SLA's. No one expects vmware to fix, recompile, and distribute ESX patches in under 4hrs...but there is a huge gap between 4hrs and 36.

Reply Re: BIG bug in ESX 3.5 Update 2 - If you're using 3.5u2 read this now! - A general system error occurred: Internal Error Aug 12, 2008 2:21 AM

in response to: deploylinux

joergriether 185 posts since
Sep 17, 2006

There must be a licensing backdoor for emergencies to simply allow everything, NOW would be the right time to give that information out to the this community. Another thing: Earlier you mentioned you´ll provide iso´s and zip´s within 36h but not a patch for fixing EXISTING machines in that timeframe. Also, you didn´t mention at all how you gonna accomplish this. I guess a reboot or maintenance mode is not an option for many users here. This is production environment, this is critical environment, I suggest you take critical measures.

Joerg

Reply Re: BIG bug in ESX 3.5 Update 2 - If you're using 3.5u2 read this now! - A general system error occurred: Internal Error Aug 12, 2008 2:23 AM

in response to: beeswarm

Lars Wolff 1 posts since
Jul 17, 2008

I hope apllying the future Patch will not require any reboot! If I am not able to use Vmotion I have to apply the Patch in the Night....

Cheers

Lars

Reply Re: BIG bug in ESX 3.5 Update 2 - If you're using 3.5u2 read this now! - A general system error occurred: Internal Error Aug 12, 2008 2:25 AM

in response to: sradnidge

ESXDevil 29 posts since
Aug 12, 2008

Hi all

I have the same issue here in Switzerland. Changing the Date solved the problem. Shame on you VMWare! I've thought that i have done a missconfiguration on the ESX Hosts and several hours i searched for a solution to fix this superb error message " a general system error has occurred: Internal error".

Thanks to the others here in this communities that helped me a lot.

Reply Re: BIG bug in ESX 3.5 Update 2 - If you're using 3.5u2 read this now! - A general system error occurred: Internal Error Aug 12, 2008 2:25 AM

in response to: Lars Wolff

ezhosting 26 posts since
Jul 28, 2008

Yes me too.

I think you should prioritize a PATCH for those stuck in this hell, instead of creating new isos.

Reply Re: BIG bug in ESX 3.5 Update 2 - If you're using 3.5u2 read this now! - A general system error occurred: Internal Error Aug 12, 2008 2:29 AM

in response to: ezhosting

mimo17 10 posts since
Aug 4, 2005

Lucky you if you can reboot in the night. We have 24h operation!

I asked the support if a trial would work as a workaround - but poor support girl did not understand.
Any sugesstions from the forum?

Reply Re: BIG bug in ESX 3.5 Update 2 - If you're using 3.5u2 read this now! - A general system error occurred: Internal Error Aug 12, 2008 2:30 AM

in response to: frank_wegner

hughs 4 posts since
Aug 12, 2008

I can't believe this has happened. I'm trialling ESXi before we spend £50k on Virtualising all of our servers. I'm hardly going to go down the VMware route now am I? Idiots.

Reply Re: BIG bug in ESX 3.5 Update 2 - If you're using 3.5u2 read this now! - A general system error occurred: Internal Error Aug 12, 2008 2:32 AM

in response to: deploylinux

Bakafish.com 8 posts since
Aug 11, 2008

As a former VMware engineer I know a bit about the build process, and unfortunately this being tied into the licensing code means that any fix likely touches a lot of different places in the code-base itself. There probably isn't going to be a simple patch, but something along the lines of a 3.5u2v2 bundle with all new components. This is one of the reasons why licensing that hobbles applications sucks, I wish there was a better way for companies to protect their interests without leaving their customers (and therefore themselves) at so much risk. The awful thing is that this bug wasn't reflected correctly in the UI, causing us to really dig to find the answer. I found this thread after about 3 hours of searching through logs and googling, when there was only a few posts on this thread. Let's hope the updated bundles are available soon. Expect Maintenance Mode/Reboot.

Representing Japan.

Reply Re: BIG bug in ESX 3.5 Update 2 - If you're using 3.5u2 read this now! - A general system error occurred: Internal Error Aug 12, 2008 2:33 AM

LB@SGI 38 posts since
Dec 18, 2007

Our company too has compliance requirements, but luckilly we can set our date back.

But here is the better way to do it in my opinion:
Set your date back by 1 or 2 years. ( to a year that your ESX farm didn't exist)

This allows you to repair your logs easilly using a search/replace function later.

Be advised however that MOST compliance related logging occurs on the VirtualCenter server.
DO NOT CHANGE THIS DATE!

VirtualCenter server dates do not have any relation ot the problem.
Furthermore, MOST VC servers are members of Domains, and if you set this time back (even to the 10th) you will not be able to log on to the server, as Active Directory will fail on this server, and/or at the very least, the NTP client on the server will RESET the time back to correct.

Basically, there is no need to reset the VC time, but I reccomend a full year or two rollback on the ESX hosts, for ease of log repair at a later time.

Reply Re: BIG bug in ESX 3.5 Update 2 - If you're using 3.5u2 read this now! - A general system error occurred: Internal Error Aug 12, 2008 2:33 AM

in response to: ezhosting

maishsk 159 posts since
Jun 1, 2005

Forgive me for being the bad guy here.

But as a Administrator, do you not wait for a product to stabilize before installing it into your production environment??

I for one will not install any new software - be it Microsoft Vmware, whatever... - until it has been up and running in the wild for at least a month or two after release date
Same goes for patches by the way..

Do not make the mistake of jumping on the bandwagon for new things on your production systems.

Feel sorry for all of you that were hasty here, but maybe this will be a lesson to us all...

Good luck with your nerves....

Maish
Systems Administrator & Virtualization Architect

Reply Re: BIG bug in ESX 3.5 Update 2 - If you're using 3.5u2 read this now! - A general system error occurred: Internal Error Aug 12, 2008 2:35 AM

in response to: mimo17

frank_wegner 383 posts since
Oct 26, 2005 VMware

Trial licenses will not help, because the licensing check itself is broken.

Reply Re: BIG bug in ESX 3.5 Update 2 - If you're using 3.5u2 read this now! - A general system error occurred: Internal Error Aug 12, 2008 2:39 AM

in response to: LB@SGI

sradnidge 132 posts since
Oct 24, 2005

LB@SGI wrote:

<snip>

Be advised however that MOST compliance related logging occurs on the VirtualCenter server.
DO NOT CHANGE THIS DATE!

<snip>

And for anyone out there running VC in a VM...

Reply Re: BIG bug in ESX 3.5 Update 2 - If you're using 3.5u2 read this now! - A general system error occurred: Internal Error Aug 12, 2008 2:41 AM

in response to: maishsk

Erik Zandboer 671 posts since
Jun 11, 2007

I must say I do not agree with the statement that you should wait before a release stabelizes. No one could have seen this coming. When do you upgrade? When you tested a release for three days, a week, two weeks? We should be thankfull for people upgrading that fast - and unravel any problems with releases.

Unfortunately, today is somewhat a black day for all of those pioneers...

This Question is Possibly Answered

Actions