How-to recreate datastore with local disks?

[Sangokan]

Hi Guyz,

I have to recreate a datastore based of local disks for an esx server that has the wrong file block size on it. I need to move from 1MB to 4MB.

I have tried to delete the datastore form the VI client but i get "Error during the configuration of the host: DestroyVmfsDatastore: can't delete partition 3 on lun vmhba0:0:0"

I have no idea how to do it via the command line.

Here is my partition setting:

# vdf -h

Filesystem Size Used Avail Use% Mounted on

/dev/sda2 4.9G 1.4G 3.2G 30% /

/dev/sda1 99M 30M 65M 32% /boot

none 131M 0 131M 0% /dev/shm

/dev/sda6 2.0G 33M 1.8G 2% /var/log

/vmfs/devices 2.7T 0 2.7T 0% /vmfs/devices

/vmfs/volumes/47541e36-98c4000e-e0cd-001aa031ed31

1.4T 647M 1.4T 0% /vmfs/volumes/storage1

[wila]

The output of

fdisk -l

will give us a better idea of your partition layout (partition #3 is not in your partitionlist for example)

[Sangokan]

Please:

# fdisk -l

Disk /dev/sda: 1497.1 GB, 1497198755840 bytes

255 heads, 63 sectors/track, 182024 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/sda1 * 1 13 104391 83 Linux

/dev/sda2 14 242 1839442+ 82 Linux swap

/dev/sda3 243 1134 7164990 83 Linux

/dev/sda4 1135 182024 1452998925 f Win95 Ext'd (LBA)

/dev/sda5 1135 181757 1450854216 fb Unknown

/dev/sda6 181758 182011 2040223+ 83 Linux

/dev/sda7 182012 182024 104391 fc Unknown

Disk /dev/sdc: 16 MB, 16777216 bytes

1 heads, 32 sectors/track, 1024 cylinders

Units = cylinders of 32 * 512 = 16384 bytes

Disk /dev/sdc doesn't contain a valid partition table

[wila]

Are you sure these are from the same ESX server?

Your first output for "vdf -h" lists sda2 as the place where the main filesystem lives ( / )

In your second output for "fdisk -l" the sda2 disk is where the swap is located, so sda2 != sda2

Maybe a "df -h" can help clear that up?

Confused..

FWIW, your second output tells me that the VMFS lives on sda5 (linux type fb ) so that's the one you would normally drop using fdisk.

[Sangokan]

no this is not from the same esx, you are right, sorry the first output was wrong:

# vdf -h

Filesystem Size Used Avail Use% Mounted on

/dev/sda3 6.8G 1.4G 5.1G 22% /

/dev/sda1 99M 30M 65M 32% /boot

none 131M 0 131M 0% /dev/shm

/dev/sda6 2.0G 33M 1.8G 2% /var/log

/vmfs/devices 2.7T 0 2.7T 0% /vmfs/devices

/vmfs/volumes/47542b0e-a1a8ab54-4346-001aa031ed31

1.4T 647M 1.4T 0% /vmfs/volumes/storage1

[Sangokan]

Ok i have deleted the fb partition and recreated it with fdisk.

I have tried to change the partition ID to fb and then tried to add a storage within VI client: i get Error during the configuration of the host: Failed to update disk partition information

Here is the new fdisk -l

# fdisk -l

Disk /dev/sda: 1497.1 GB, 1497198755840 bytes

255 heads, 63 sectors/track, 182024 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/sda1 * 1 13 104391 83 Linux

/dev/sda2 14 242 1839442+ 82 Linux swap

/dev/sda3 243 1134 7164990 83 Linux

/dev/sda4 1135 182024 1452998925 f Win95 Ext'd (LBA)

/dev/sda5 181758 182011 2040223+ 83 Linux

/dev/sda6 182012 182024 104391 fc Unknown

[wila]

Strange that the VI Client is trying to drop partition 3 as that is the one containing ESX itself (the root filesystem)

What you can do is drop the vmfs partition using fdisk, it's pretty straightforward and then use the VI client again to create a new vmfs filesystem.

I would probably also move the coredump partition (type =fc) by dropping it and recreating it in a lower area, but's just my preference, there's absolutely no need to do that.

So in order to drop the 5th partition you would follow these steps

Beware that you WILL destroy any VMs which are currently still stored at the VMFS storage!

Make a backup first.

fdisk /dev/sda

p (print the partition table)

d (for "delete", now fdisk offers you to select between 1 and 6, select 5)

5

w (write changes to disk)

reboot?

good luck!

[wila]

"failed to update disk information" sounds like it is in need for a reboot..

There's also a vmkfstools command to create a storage IF you need it, but i'd expect the VIC to work after the reboot.

If it doesn't work, you can recreate the VMFS partition using:

vmkfstools --createfs

Please check the man page for vmkfstools for more details on that.

[Sangokan]

This is what i get even after a server reboot:

root@remcorpesx04 root# vmkfstools -C vmfs3 -b 2m -S LocalStorage1 /vmfs/devices/disks/vmhba0\:0\:0\:0

Creating file system on "vmhba0:0:0:0" with blockSize 2097152 and volume label "LocalStorage1".

Usage: vmkfstools -C vmfs3 /vmfs/devices/disks/vmhbaW:X:Y:Z

Error: Operation not permitted

Still same partition table

fdisk -l

Disk /dev/sda: 1497.1 GB, 1497198755840 bytes

255 heads, 63 sectors/track, 182024 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/sda1 * 1 13 104391 83 Linux

/dev/sda2 14 242 1839442+ 82 Linux swap

/dev/sda3 243 1134 7164990 83 Linux

/dev/sda4 1135 182024 1452998925 f Win95 Ext'd (LBA)

/dev/sda5 181758 182011 2040223+ 83 Linux

/dev/sda6 182012 182024 104391 fc Unknown

Disk /dev/sdc: 16 MB, 16777216 bytes

1 heads, 32 sectors/track, 1024 cylinders

Units = cylinders of 32 * 512 = 16384 bytes

Disk /dev/sdc doesn't contain a valid partition table

Thank you for your help

Message was edited by: Sangokan

Server reboot

[Sangokan]

Ok i got further:

After you delete the partition you have to recreate it with fdisk

So i have done

fdisk /dev/sda

n (new part)

7 (number)

t (change type)

fb

w (write)

then i rebooted the server.

Still same error with the GUI but it works with the command-line

fdisk -l /vmfs/devices/disks/vmhba0\:0\:0\:0

Disk /vmfs/devices/disks/vmhba0:0:0:0: 1497.1 GB, 1497198755840 bytes

255 heads, 63 sectors/track, 182024 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/vmfs/devices/disks/vmhba0:0:0:0p1 * 1 13 104391 83 Linux

/vmfs/devices/disks/vmhba0:0:0:0p2 14 242 1839442+ 82 Linux swap

/vmfs/devices/disks/vmhba0:0:0:0p3 243 1134 7164990 83 Linux

/vmfs/devices/disks/vmhba0:0:0:0p4 1135 182024 1452998925 f Win95 Ext'd (LBA)

/vmfs/devices/disks/vmhba0:0:0:0p5 181758 182011 2040223+ 83 Linux

/vmfs/devices/disks/vmhba0:0:0:0p6 182012 182024 104391 fc Unknown

/vmfs/devices/disks/vmhba0:0:0:0p7 1135 181757 1450854246+ fb Unknown

# vmkfstools -C vmfs3 -b 2m -S Storage1 /vmfs/devices/disks/vmhba0\:0\:0\:7

Creating file system on "vmhba0:0:0:7" with blockSize 2097152 and volume label "Storage1".

Successfully created new volume: 47544176-cdcbac07-4835-001aa031ed31

Why doesnt it work with VI client?

Besides i guess the volume is not aligned now, so best practice would say i have to align the volume manually...?

[Chamon]

There are two cases when aligning disk partitions:

When aligning raw disks or Raw Device Mapping (RDM) volumes, the alignment is done at the Virtual Machine (VM) level. For example, on Windows VMs use diskpar to perform the alignment.

To align VMFS volumes, the alignment will be done at ESX server level using fdisk and at the VM level. This is because both the ESX Server and the clients will put MBRs on the LUNs. The ESX must align the VMFS volume, and the client systems must align their virtual disks.

To align the ESX server:

1. On service console, execute fdisk /dev/sd is the device on which you would like to create the VMFS.

2. Type "n" to create a new partition.

3. Type "p" to create a primary partition.

4. Type "1" to create partition #1.

5. Select the defaults to use the complete disk.

6. Type "x" to get into expert mode.

7. Type "b" to specify the starting block for partitions.

8. Type "1" to select partition #1.

9. Type "128" to make partition #1 to align on 64KB boundary.

10. Type "r" to return to the main menu.

11. Type "t" to change partition type.

12. Type "1" to select partition #1.

13. Type "fb" to set the type to fb (VMFS volume).

14. Type "w" to write label and the partition information to disk.

By declaring the partition type as fb, the ESX server will recognize the partition as an unformatted VMFS volume. You should be able to put a VMFS file system on it using the MUI or vmkfstools. Next, the virtual disks for each VM must be aligned. For Linux VMs follow the procedure listed above. For Windows VMs, use the procedure for Windows, above.

[wila]

You mentioned something about you recreating partition # 5 using fdisk. The only partition 5 i see now is the one that has the data for /var/log. Your fdisk indicates that there's a gap between partition 4 and partition 5. Normally i put the VMFS at the last partition. Have you tried to rescan in the host and try creating the storage using the VI Client?

I'm sorry i don't have a spare partition to play with right now and experiment with the CLI, but from the man page i sort of gather that you would have to indicate the partition# in the vmhba notation. Maybe someone else that has used this before can fill in the gap?

[wila]

Hi,

AFAIK, partition aligning is only an issue with SAN /NFS and iSCSI

http://www.vmware.com/pdf/esx3_partition_align.pdf

But i might be totally wrong... as it sounds weird to me that a local storage would not have this issue.