Partition Update Failure when Adding Extent

[VoipDude]

We had 100GB for our primary VMFS and ran out of space. We increased the LUN size, added an Extent through the VIC and then had 250GB. We are out of space again, so we're trying to process again with another 250GB of increased space on the LUN. But this time we're getting an error when we try to add the Extent.

The VIC (when using VC or direct to the ESX server) sees the Extent, but when we click Finish to finalize the changes we get the error.

I was not the person who created the 2nd Extent. I originally did an fdisk, changed partition type to 'fb' and attempted to add the Extent. But it was not showing up, so I deleted the partition and that's where we are today.

vmkernel, in short, says:

Jul 3 13:13:29 hostname vmkernel: 0:00:25:56.506 cpu4:1039)WARNING: ScsiHost: 303: vml.02000000000690a018f01579255f5e041107003009313030452d30:0: Can't overwrite contents of active fb partition

hostd.log, in short, says:

Error : Error: Read-only file system during write on /vmfs/devices/disks/vml.02000000000690a018f01579255f5e041107003009313030452d30

Unable to commit to disk

I found a KB (2240) on how to manually fdisk (which I did) and format using vmkfstools (which I did not do). However, the KB does not mention how I would add this new space to my existing VMFS datastore. So I'm not sure if that is actually the same problem as what I'm experiencing or not.

Thanks,

Scott

[kjb007]

I would try a reboot on my ESX host first, and then try to re-add the extent before any further troubleshooting. Then, if that doesn't work, I would remove that LUN and re-present it to the server. Is this from a SAN or DAS?

-KjB

[VoipDude]

I would try a reboot on my ESX host first, and then try to re-add the extent before any further troubleshooting. Then, if that doesn't work, I would remove that LUN and re-present it to the server. Is this from a SAN or DAS?

I already attempted a reboot of the ESX server this morning. That did not appear to change anything. I have not removed the LUN yet. Although it was a size expansion, not a LUN creation, so I will need to see if a reduction is even possible. I'm using an iSCSI SAN on the backend.

Thanks,

Scott

[kjb007]

Oh, that is a different matter. You shouldn't attempt to "expand" a LUN that a VMFS volume exists on, because there is no way to extend the VMFS on top of it. So, if you can't reduce the LUN back to the original size, you will have wasted unused space, unless you can partition that LUN . If fdisk can see the additional space as empty, you should be able to create a new empty partition in that space. If you've already done that, run fdisk -l and post here.

-KjB

[VoipDude]

You shouldn't attempt to "expand" a LUN that a VMFS volume exists on, because there is no way to extend the VMFS on top of it.

1. fdisk -l /dev/sdc

Disk /dev/sdc: 536.8 GB, 536881397760 bytes

255 heads, 63 sectors/track, 65272 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/sdc1 1 13054 104856191 fb Unknown

/dev/sdc2 13055 32636 157292414 fb Unknown

We started with 100GB (/dev/sdc1) and later expanded the LUN to add 150GB (/dev/sdc2). We're just trying to run through the process again to add another 250GB.

I manually created /dev/sdc3 initially, thinking I needed to prepare the partition for VMware (which I found out I don't), and I'm not sure if that's the root of my problem?

Per KB 2240 I could create /dev/sdc3 and manually format it with vmkfstools, but I'm not sure if I'd be able to add it to my existing VMFS datastore?

Thanks,

Scott

[VoipDude]

VMware support rep mentioned not having any VMs running on the LUN to stop any I/O activity. Could this possibly help me out? I could see why it might be in a "read only" state when VMs are running, but would purely stopping the VMs remove the "read only" status (if this is actually the case) or would a specific VMware service need to be stopped to release the hold?

I'm thinking about trying to shrink the LUN to the original size and just add a new LUN. Apparently VMware prefers it that way.

Thanks,

Scott

[VoipDude]

VMware support rep mentioned not having any VMs running on the LUN to stop any I/O activity.

Bingo! I suspended all of the VMs on the LUN and the Extent addition worked great.

Thanks,

Scott