"General fault caused by file. " after upgrade from ESX2

[greglim]

After upgrading an ESX2 series server to ESX3, and then upgrading the vmfs to vmfs3, nearly all of my VM's are maked "invalid". Only one apparently is still valid. In the "events" tab for the server, I see a whole bunch of errors like this:

Failed to relayout SQLSERVER06: "General fault caused by file. "

I get one per "invalid" vm. The *.vmx files appear to still exist as well as the vmdk's.

I am still investigating, but if anyone knows what this error message means, please answer!

Thanks!

-Greg Lim

[RobMokkink]

did you upgrade the vm's???

[greglim]

When the VM's get in that state, you cannot upgrade them. In fact, the only things I can do are "remove from inventory" and "delete from disk". I think "add permission..." might work, but I am root, and all files are readable/exec by root.

I've managed to get several VMs up again by removing from inventory and recreating them with their vmdk's, but I have approximately 20 VM's here, and the earlier admin has left me with a great undifferentiated pile of unmatched vmx's and vmdk's. It is taking a significant amount of time to properly recreate these.

I also want to make a remark about (re)adding multiple ESX2 vmdk's to these new machines. To actually rebuild the machine, I have to upgrade hardware \*twice. The first time happens because you \have* to add the first ESX2 vmdk, and you cannot add the 2nd vmdk until you upgrade. At that point, you can add the 2nd vmdk, but you will need to immediately upgrade hardware \*again* as you won't be able to edit settings again. aargh.

Summary: I have a workaround, but it's taking forever.

[larryblanco]

I am having the same issue. In migrating the VM from ESX 2.5.x & VMFS 2 to ESX 3 and VMFS3 I am getting the error "General fault caused by file. \[VOLUME] filename.vmdk"

What gives VMWARE???? Bring out the engineers that are hidden in the closet located in the backroom with no windows.

HELP!

[daaniel]

Any news on this issue? I'm having the same issue with nearly all my VM's...

[pyoung]

I'm having the same problem. Anyone else getting the feeling this upgrade wasn't quite ready for primetime? I'm getting negative free space showing up on newly created datastores, servers which fail when trying to upgrade, datastores which show up alternately in VC as VMFS2 and VMFS3, etc. And now this...

[jharper1]

I'm having the same issue here. I was able to migrate 1 guest, but have not been able to move any more. I was also showing negative space on my VMFS 3 partition and getting the insufficient disk space error, but now I'm receiving this error "General fault caused by file."

[larryblanco]

Well, I guess I will post my work around since no VMWARE tech did.

I had to putty into the esx 3 and use "vmkfstools -i srcdisk destdisk" to clone the VM i wanted to move. The clone process took the VM in VMFS2 and moved & converted it to VMFS3. After the clone process was complete I created a new VM on he ESX 3 server. Moved the VMDK files into the directory of the VM and attached the VMDK files to the VM. Now I have a ESX 3 VMFS3 VM converted over from ESX2 and VMFS3. Once I verified that the VM is 100% operational I went ahead and deleted the old VM.

For those of you that use the VIC and cannot see a datastore, It's a bug the VIC and supposedly there will be a patch out soon for it. I have that same isse as well.. You will need to go into the WEB CLIENT for the ESX 3. It displays the datastore properly there.

Larry

[jharper1]

Ran the following command on the service console and now it works fine using the cold migration in VirtualCenter 2.0.

\[root@host ~]# vmkfstools -s vmhba1

\[root@host ~]# vmkfstools -s vmhba2

[Lord_Narcosis]

What I found to work for us with the "General Fault" error when trying to cold migrate from 2.5.3 to 3.0 was to do the migration to the 3.0 server and tell VCenter not to move the files, only the host the machine is on. Then I re-ran the migration from the 3.0 host and told it to migrate to itself and this time to move the files during the migration. After the migration hit 100% it sat the for about 15 minutes and then said "operation timed out" but the files were in the new location and the vm powered up and worked just fine.

When doing this I discovered that there was an old copy of the vm left in its original location so i assume the "operation time out" error was because the 3.0 server could not delete the old files from the VMFS2 partition since it is read-only to 3.0.

This work around has worked flawless for us on 86 VMs so far and the nice thing is the original files are still there if we have an issue.

[wcrahen]

I ran into this same problem tonight as well. The 2 step cold migrate that Lord Narcosis mentions works, but I have found I have to rescan the LUNs everytime before migrating otherwise I have the negetive disk space problem, quite the pain.

[john_flatbush]

I encountered the same problem. Solution was to remove the upgraded volume from all hosts. Remove the volume under datastores. Recreate the volume on an ESX3 server. Once the volume was recreated instead of upgrading the volume, I was able to cold migrate from 2.5.1 to 3.

[admin]

Have any of you reported these problems to VMware tech support. If so, please provide the SR # and I will get someone to follow up with you. We need to get logs from ESX and VC to debug this problem. We could not repro this problem in house.

[pooppo]

This is trivial to reproduce.

Use VMotion to migrate an ESX 2.5 VM to an ESX 3 host, while keeping the data store the same, i.e. the VM disk remains on the VMFS2 partition.

[TVolke]

I had the same problem after upgrading from vmfs2 to vmfs3, but found the following workaround.

\- remove the client from inventory

\- edit clients .vmx file and change the following

scsi0:0.name = ""

scsi0.virtualDev = "lsilogic"

ex.

scsi0:0.name = "EVA0VOL4:TEST1.vmdk"

scsi0.virtualDev = "vmxlsilogic"

changed to

scsi0:0.fileName = "/vmfs/volumes/EVAVOL4/TEST1.vmdk"

scsi0.virtualDev = "lsilogic"

\- register the client

vmware-cmd -s register /home/vmware/TEST1/TEST1.vmx

\- relocate VM files on host

\- Upgrade virtual hardware

\- Power up client

\- Upgrade VMware tools

\- Reboot client

[pooppo]

Thanks for this, but it didn't work.

I can't edit the config file before vmotion, because vmotion then fails.

I cannot edit the config file after vmotion, because then it is too late.

My workaround was to use clone instead of vmotion.

[Hudson_Perkins]

The error occured for me when I was cold migrating vm's from LUNA to LUNB. LUNA and LUNB were originally VMFS2 partitions. I upgraded them to VMFS3.

My workaround was to remove LUNB (as it did not contain any files) and recreate the LUNB VMFS3 volume. This fixed the problem even though the error related to LUNA.

Message was edited by:

Hudson Perkins

[jonhutchings]

We had this, and ended up going the cold migration route. I think in our case all the problems stem from screwed up networking post upgrade.

Couple you might want to try before doing anything more serious.

1) make sure your networking is all good, and reboot the affected ESX servers, then check the networking is still good.

2) check the management agents are running correctly on the hosts (service console, then service mgmt-vmware restart)

People have pretty much covered the steps we went through to get everything sorted.

It's a nasty moment when it happens, but at least in our case, it all came good without any data loss

[Hamer]

Is there an actual fix for this......It is my scenario exactly??

Thanx

[Riles]

Here's my current layout and how I was able to get around the error. I have an ESX2.5.2 host that is part of my old farm. I will migrate my vm's that I want to upgrade to this machine and remove it from the farm. Then I will add it to the new farm that is comprised to 2 AMD ESX3.0.1 host and 2 Intel ESX3.0.1 host. When I add the ESX2.5.2 to the new 3.0.1 farm, it will be grouped with the other 2 Intel server.

So virutal center looks like this:

Hosts & Clusters

\ |-------ESX Farm (Data Center)

\ |----

---

AMD Hosts (Folder)

\ |----

---

ESX3.0.1 (VM Host)

\ |----

---

Intel Hosts (Folder)

\ |----

---

ESX2.5.5 (VM Host)

\ |----

---

ESX3.0.1 (VM Host)

First I will migrate my VM from ESX2.5.2 to ESX3.0.1 that are grouped together in the "Intel Host" folder. I have had one failure doing this and I restarted the migration and it completed the second time. Second I will Upgrade the Virtual Hardware. Third, boot the VM and upgrade the VMWare Tools. Don't ask me why or how it works. But it does for me.