VMware Cloud Community
gwsilliman
Contributor
Contributor

esxi 4.1 can not expand datastore lun detected as snapshot

I am attempting to expand an existing datastore located on a direct attached raid 5 array.  I have added space on the array, which esxi recognizes, however I can not expand the datastore to consume the new free space, nor can I create a new datastore with the free space.

I am using 5 250gig drives, so I am not running up against the 2tb barrier.  They are in an external array chassis, attached to an RR2722 sas controller.

When I open the properties of the datastore, the expand open is grayed out.

Any thoughts as to how I can expand my datastore?

Thank you

Reply
0 Kudos
21 Replies
rushdir
Contributor
Contributor

I think your answer is on the screenshot it self...

your VMFS data store is configured for 2MB block size and the maximum storage you can create is 512 GB.

Block size of 1MB                256GB

Block size of 2MB                512GB

Block size of 4MB                1TB

Block size of 8MB                2TB

Refer the maximums doc for more info.

Hope this helps you...

Reply
0 Kudos
gwsilliman
Contributor
Contributor

Unfortunately, I don't think that is it.

The screen shot also shows that the existing datastore is 698gig.  The setting you pointed out have to do with the maximum virtual disk file I can create on the datastore.

Any other thoughts?

Reply
0 Kudos
AWo
Immortal
Immortal

It is not related to the block size, as this restricst the maximum file size, not the datastore size, which is 2 TB - a few KB.

The problem is, that you can have only one VMFS partition per disk but you tried to add an extend which must be a free VMFS partition. So you only can add an extend if you have a VMFS partition on another disk.

What you need to do is to expand (grow) the VMFS partition itself, but AFAIK that is not possible.

Check: http://pubs.vmware.com/vsphere-esxi-4-1-installable/wwhelp/wwhimpl/js/html/wwhelp.htm#href=server_co...

AWo

vExpert 2009/10/11 [:o]===[o:] [: ]o=o[ :] = Save forests! rent firewood! =
Reply
0 Kudos
DSTAVERT
Immortal
Immortal

If ESXi is installed on the same array you probably won't be able to expand the datastore (another reason to install ESXi to USB). You may need to migrate your VMs and start fresh.

-- David -- VMware Communities Moderator
Reply
0 Kudos
gwsilliman
Contributor
Contributor

Thanks for the replies so far...

AWo >  What you need to do is to expand (grow) the VMFS partition itself, but AFAIK that is not possible.

This is really unfortunate for us soho people who are trying to take advantage of the technology.  Online Raid 5 expansion is a viable method of expanding storage.  Without datastore growth, we can not cost effectively grow in to the hardware.

DSTAVERT > If ESXi is installed on the same  array you probably won't be able to expand the datastore

DSTAVERT > (another reason  to install ESXi to USB). You may need to migrate your VMs and start  fresh.

The system is installed on a usb flash drive.   I made that upgrade when  I rebuilt the host for 4.1.  The host is actually diskless now.  All  the spinning media is in the external array chassis, save the dvd drive.

Reply
0 Kudos
DSTAVERT
Immortal
Immortal

Did you rebuild the array when you installed 4.1 to USB or just installed ESXi to to a USB disk and used the existing datastore. Expanding requires that the new space be directly adjacent so that you have a contiguous volume.

-- David -- VMware Communities Moderator
Reply
0 Kudos
gwsilliman
Contributor
Contributor

The array was an addition with the upgrade.

The system was running on 3 of the 250gig drives as independent discs, each their own datastore.  I moved all the critical vm's and file server vmdks to a single drive - which at the time was the boot drive also.  I built the array with 2 new 250gig drives and the 2 old drives, reinstalled esxi on the flash drive.

In order to get the rr2722 to work, I had to install 4.0, install the drivers, then upgrade to 4.1.

Once I had the system stable, I created the existing datastore on the array and moved all the vm's and their associated files to the array.  I have now removed the last drive from the host chassis, initialized it in to the array chassis, and expanded the raid5 array to include this 5th disc.

And now we are up to the current delema... how to make use of this added space on the array.....

Guy

Reply
0 Kudos
DSTAVERT
Immortal
Immortal

You have gone to Configuration -> Storage Adapters and Select the storage adapter done a Rescan All. Check to see that the storage space matches.

Configuration -> Storage Rescan All. Right click the datastore and Properties. Use the Increase button and select the appropriate disk.

-- David -- VMware Communities Moderator
Reply
0 Kudos
gwsilliman
Contributor
Contributor

Yes, all steps exactly as described - in fact while still in maintenance mode after the array expansion.  I have also attempted after exiting maintenance mode, and had vm's active.  All attempts return to the screen shot in the first post.

The device shows 931gig, the partition is 698, and the extent is 698.  The "Increase" button is disabled.

Guy

Reply
0 Kudos
a_p_
Leadership
Leadership

I'm not exactly sure why you cannot increase the VMFS LUN in the GUI. Maybe it's related to the HighPoint controller, maybe to the DAS storage!?

However, you could increase the datastore manually as described for ESX 4.0 (not ESXi though) in http://kb.vmware.com/kb/1009125. There's also a blog post describing the steps for ESXi at http://deinoscloud.wordpress.com/2010/05/03/how-to-increase-the-size-of-a-local-datastore-on-an-esxi...

Caution: Make sure you have a current backup! I only tried the steps for ESXi 4.0 in my lab environment so far and it worked.

André

gwsilliman
Contributor
Contributor

André,

  Thank you so much.  I have not taken the steps detailed but this looks promising.  My only concern is the fdisk tends to be destructive.  Will I loss the existing data on the partition when I alter it?

I know this sounds like an obvious answer with the information you referenced, but I am leery about altering the partition table.

Guy

Reply
0 Kudos
a_p_
Leadership
Leadership

I agree, when I first read about deleting the partition I was confused too. However, as long as you re-create the partition with the same start block and only another (larger) end block, you actually only alter the partition table and do not harm the data in the partition. I'd suggest you read the KB article closely and if you are not sure about it, create e.g. a virtual ESXi host in a VMware Workstation or even on your current ESXi host and try the steps in this test machine.

André

Reply
0 Kudos
DSTAVERT
Immortal
Immortal

It does work but use it with caution. I would practice it several times. If you don't have a physical host to practice with you should be able to recreate this with ESXi as a guest OS. I assume this would work using VMware workstation and ESXi as a guest. And as suggested make sure you have current backups.

-- David -- VMware Communities Moderator
Reply
0 Kudos
gwsilliman
Contributor
Contributor

Ok - I'm good with that.  Last thing.  Should I have all VM's shut down when I do this?

Reply
0 Kudos
a_p_
Leadership
Leadership

I'm not 100% sure, however I don't really see the need to power off the VM's. With the fdisk commands you actually don't delete the partition, since you don't apply the write command before the partition is reconfigured with the larger size. The vmkfstools command to grow the datastore is a VMware own tool, so I would expect a warning/error if the VM's need to be powered off.

However as I mentioned earlier, I didn't do this myself in production yet.

André

Reply
0 Kudos
gwsilliman
Contributor
Contributor

I was able to expand the partition with fdisk, however I am stuck at the vmkfstools step.

esxcfg-scsidevs -l -d eui.00193c0000000000
eui.00193c0000000000
   Device Type: Direct-Access
   Size: 953344 MB
   Display Name: RR2722_0 Raid5 Datastore00
   Multipath Plugin: NMP
   Console Device: /vmfs/devices/disks/eui.00193c0000000000
   Devfs Path: /vmfs/devices/disks/eui.00193c0000000000
   Vendor:           Model: VD3-0             Revis: 4.00
   SCSI Level: 5  Is Pseudo: false Status: on
   Is RDM Capable: false Is Removable: false
   Is Local: true
   Other Names:
      vml.010000000052443030303030303030005644332d3000
   VAAI Status: unknown

vmkfstools --growfs /vmfs/devices/disks/vml.010000000052443030303030303030005644332d3000\:1 /vmfs/devices/disks/vml.010000000052443030303030303030005644332d3000\:1

Error: No such file or directory

I have checked to make sure /vmfs/devices/disks/vml.010000000052443030303030303030005644332d3000:1 exists, and it is a symbolic link to eui.00193c0000000000:1

There is 1 and only 1 partition on eui.00193c0000000000

The blog mentions that on occations the vmfs dsk does not exist anymore, but does not mention how to tell if this is the case....

Any thoughts?

Reply
0 Kudos
a_p_
Leadership
Leadership

Does fdisk -lu return the partition with the "FB" (VMFS) partition type? Do you see the datatsore in the GUI?

If you see the partition correctly, try the growfs command without quoting the partition number with the "\".

André

Reply
0 Kudos
gwsilliman
Contributor
Contributor

André Pett wrote:

Does fdisk -lu return the partition with the "FB" (VMFS) partition type?

Do you see the datatsore in the GUI?

If you see the partition correctly, try the growfs command without quoting the partition number with the "\".

André

I looked at all those.  They are fine.  I checked the logs and found this little tid-bit.

Jan 27 00:09:59 vmkernel: 0:20:43:55.658 cpu1:128651)LVM: 6919: Error adding space (0) on device eui.00193c0000000000:1 to volume 4d31205d-081eb2d4-2032-00238b89e99e: Not found
Jan 27 00:10:13 shell[8970]: vmkfstools --growfs /vmfs/devices/disks/vml.010000000052443030303030303030005644332d3000:1 /vmfs/devices/disks/vml.010000000052443030303030303030005644332d3000
Jan 27 00:10:13 vmkernel: 0:20:44:09.189 cpu1:128657)LVM: 7404: Device eui.00193c0000000000:1 detected to be a snapshot:
Jan 27 00:10:13 vmkernel: 0:20:44:09.189 cpu1:128657)LVM: 7411:   queried disk ID: <type 1, len 17, lun 0, devType 0, scsi 0, h(id) 12617330505899758443>
Jan 27 00:10:13 vmkernel: 0:20:44:09.189 cpu1:128657)LVM: 7418:   on-disk disk ID: <type 1, len 17, lun 0, devType 0, scsi 0, h(id) 1259420025630164831>
Jan 27 00:10:13 vmkernel: 0:20:44:09.189 cpu1:128657)LVM: 2710: Device eui.00193c0000000000:1 is detected as being in volume 4d31205d-081eb2d4-2032-00238b89e99e (0x417f8348f8e0)
Jan 27 00:10:13 vmkernel: 0:20:44:09.239 cpu1:128657)LVM: 8574: LVMProbeDevice failed with status "Device does not contain a logical volume".

This seems to point towards a issue with the volume not being found

Reply
0 Kudos
gsilliman
Contributor
Contributor

Just a recap here - I was busy with other issues and needed to let this rest for a while.

I have a datastore that I am attempting to expand.  I have expanded the partition, however the vmkfstools is failing.....

/dev/disks # vmkfstools --growfs /vmfs/devices/disks/eui.00193c0000000000\:1 /vmfs/devices/disks/eui.00193c0000000000\:1Error: No such file or directory
/dev/disks # tail /var/log/messages
Feb 11 00:21:17 Hostd: [2011-02-11 00:21:17.423 74C40B90 verbose 'Vmsvc'] RefreshVms updated overhead for 1 VM
Feb 11 00:21:32 shell[8970]: vmkfstools --growfs /vmfs/devices/disks/eui.00193c0000000000\:1 /vmfs/devices/disks/eui.00193c0000000000\:1
Feb 11 00:21:32 vmkernel: 15:20:55:28.626 cpu1:2248447)LVM: 7404: Device eui.00193c0000000000:1 detected to be a snapshot:
Feb 11 00:21:32 vmkernel: 15:20:55:28.626 cpu1:2248447)LVM: 7411:   queried disk ID: <type 1, len 17, lun 0, devType 0, scsi 0, h(id) 12617330505899758443>
Feb 11 00:21:32 vmkernel: 15:20:55:28.626 cpu1:2248447)LVM: 7418:   on-disk disk ID: <type 1, len 17, lun 0, devType 0, scsi 0, h(id) 1259420025630164831>
Feb 11 00:21:32 vmkernel: 15:20:55:28.627 cpu1:2248447)LVM: 2710: Device eui.00193c0000000000:1 is detected as being in volume 4d31205d-081eb2d4-2032-00238b89e99e (0x417f877ff8e0)
Feb 11 00:21:32 vmkernel: 15:20:55:28.683 cpu1:2248447)LVM: 7404: Device eui.00193c0000000000:1 detected to be a snapshot:
Feb 11 00:21:32 vmkernel: 15:20:55:28.683 cpu1:2248447)LVM: 7411:   queried disk ID: <type 1, len 17, lun 0, devType 0, scsi 0, h(id) 12617330505899758443>
Feb 11 00:21:32 vmkernel: 15:20:55:28.683 cpu1:2248447)LVM: 7418:   on-disk disk ID: <type 1, len 17, lun 0, devType 0, scsi 0, h(id) 1259420025630164831>
Feb 11 00:21:32 vmkernel: 15:20:55:28.683 cpu1:2248447)LVM: 6919: Error adding space (0) on device eui.00193c0000000000:1 to volume 4d31205d-081eb2d4-2032-00238b89e99e: Not found
/dev/disks # cd /vmfs/volumes
/vmfs/volumes # ls -al
drwxr-xr-x    1 root     root                512 Feb 11 00:34 .
drwxr-xr-x    1 root     root                512 Jan 26 03:26 ..
drwxr-xr-x    1 root     root                  8 Jan  1  1970 386a41e4-ff1587ba-1385-2feaeed494ad
drwxr-xr-t    1 root     root               1820 Jan 23 21:27 4d31205d-6299df74-399a-00238b89e99e
drwxr-xr-x    1 root     root                  8 Jan  1  1970 8c7e2fe8-f4c58d9f-34d3-8553ffee79a0
lrwxr-xr-x    1 root     root                 35 Feb 11 00:34 Hypervisor1 -> 386a41e4-ff1587ba-1385-2feaeed494ad
lrwxr-xr-x    1 root     root                 35 Feb 11 00:34 Hypervisor2 -> 8c7e2fe8-f4c58d9f-34d3-8553ffee79a0
lrwxr-xr-x    1 root     root                 35 Feb 11 00:34 Hypervisor3 -> e00f98e1-2bcc0c91-e7a2-3487611c1557
lrwxr-xr-x    1 root     root                 35 Feb 11 00:34 datastore0 -> 4d31205d-6299df74-399a-00238b89e99e
drwxr-xr-x    1 root     root                  8 Jan  1  1970 e00f98e1-2bcc0c91-e7a2-3487611c1557
/vmfs/volumes #

The volume it is attempting to expand does not exist in the sytem volumes.  datastore0 is the target to be expanded, the volume id is a different volume id.

Any thoughts?

Reply
0 Kudos