Win2k3 Guest OS Disk I/o errors running on ESX 3 infrastructure

[Wes75]

The organisation I am supporting are running ESX 3 infrastructure, with the VMDK's located on a SAN environment.

We are experiencing on random VM's (Win2k3 SP0-2 OS) Disk I/O errors within the Guest OS, although there are no other issues being reported at either the SAN or Host levels.

For example, I have found that one VM can be rebooted and will run successfully for approx 20 minutes before continually reporting disk I/O errors, including the CI Service (cisvc) reportedly dismounting the C drive volume. This obviously then causes the guest to hang until it is forcably rebooted via VC.

Other guests that are experiencing similar issues, which are located on different SANs and Hosts (although hosts are on the same ESX OS level) but they generally tend to last longer before they experience this issue. However, at the moment it is only a select few guests (approx 20 out of a total 700-odd VMs spread over 4 clusters of 8 hosts per cluster) that are experiencing this issue.

Has anyone else experienced this issue or may know of a possible solution, whether it be a WinOS or VMware fix?

Additional details of the environment can be provided on request.

[dave01]

You will need to provide us with more information such as FC switch models/SAN models etc.

First thing i'd try is to look for something common between those Machines with the issue.

For example...

Are they all going thru the same FC switch? (check your pathing)

[Wes75]

dave01, thankyou for your reply.

Unfortunately I cannot provide specific detailed information regarding the SAN & fabric infrastructure, as the environment is so large that we have a seperate team who specifically look after that infrastructure and unfortunately are not forthcoming with details of it (to the point in the past of claiming that there is no such thing as SAN environment logs for diagnostic reporting). Also, we have a strict lock-down on access to the datacentre, and it is especially difficult to organise approval to lift floor tiles, etc, to trace cabling (as the fibre was laid by the storage mangement group so I dont even know physically which SAN the hosts are connected to.

What I can tell you is that the Virtual machines are resident on seperate ESX hosts, of which are also in seperate clusters. To my knowledge (keeping in mind the above), the SAN storage utilised where the VMs are stored is seperate hardware for each cluster. SAN hardware is EMC Clarrion, although I am not certain of models, etc, used.

ESX hosts are Sun Fire v40z, running ESX 3.0 Update 1 or 2 (depending on cluster).