VMware Cloud Community
hanniex
Contributor
Contributor

RDM and VMFS on same Harddrive

Hello guys,

first of all, forgive me for my english. I am from germany but it should be understandable. 😃

Until last weekend I had following system:

Ubuntu Server 10.04 with VMware Server 2.0.2

80GB -> System

500GB -> some VMs and other Stuff

2x 1,5TB and 1x2TB (the 500GB left was unused) -> RAID5

In fact that VMware Server reached end of life and didn't run with newer kernels without some modifications and bla bla I decided to switch to ESXi.

So I installed ESXi 4 U1 on a 2GB USB stick and attached only the 3 RAID drives which contains now all data.

Now I want the 3 drives "as they are" accessible to a VM (physical migrated with VMware Converter from the running Ubuntu) and the 500GB unused as a Datastore for my VMs.

As described here: http://www.vm-help.com/esx40i/SATA_RDMs.php

I can easiely create RDMs and they are accessible in the VM.

_BUT_ I cant create these "RDM vmdk mapping files" only on a VMFS!!!

I tried:

-formating the 1GB unallocated space on the USB stick to VMFS -> the vmkernel cant format USB devices with VMFS and I have to reinstall ESXi...

-create these files on a NFS Share -> NFS Share is not VMFS formated and only temporary on my notebook accessible

-format the 500GB unallocated space on the 2TB HDD with VMFS -> I can create the mappings for the 1,5TB drives but for the 2TB it gives me "device busy" (I think because the 500GB are used as a Datastore) also when NO VM is saved there. If I choose to mapp only the raid partition from the 2TB drive it says something like "insuffient permissions". Today evening I will add the exactlly outputs.

Last idea if all other options are useless:

Backup all data on the RAID ~2,2TB -> create VMFS Datastores on the drives -> create 3x1,5TB vmdk and add to a RAID5 in the VM.

But then I have an extra layer between the hardware and the RAID data that maybe produce problems in future. The performance should be minimal decreased as I read and I don't need super performance.

Thanks for reading and I hope someone can help me, I am very frustrated after several reinstalls from ESXi and still no success.

If I forgot some informations please let me now so I can add them.

Reply
0 Kudos
7 Replies
DSTAVERT
Immortal
Immortal

Make sure the disk controller is a hardware based RAID controller and on the HCL http://vmware.com/go/hcl If it isn't a hardware bases RAID controller then it won't recognize the array only individual disks. ESXi will not recognize individual LUNs over just less than 2TB so you would also need to break the storage into smaller chunks.

I would honestly start fresh. Backup the data to some other destination and create your arrays (virtual LUNs, Virtual disks)

-- David -- VMware Communities Moderator
Reply
0 Kudos
hanniex
Contributor
Contributor

Hello and thanks for your answer.

It's just the onboard SATA controller from an Asus P5B-SE with ICH8 (P965) Chipset. Also notfound on the HCL but the disks are a all correctly recognized by the ESXi. The drives dont operate in a hardware raid so I want them in a software raid again in the virtuallisized Ubuntu. Do you mean that i can't create the RDM mapping for the 2TB disk because the "file size" of the vmdk would be to large?

Is there really no other way than backup all the data and start with making 1,5TB vmdk disks and create the softraid5 on them?

Any ideas?

Reply
0 Kudos
DSTAVERT
Immortal
Immortal

What you want to do has been done. The 2TB limitation still exists.

It may be possible to use the disk controller in passthrough mode to a virtual machine but the virtual machine would already need to exist on a VMFS volume. You will need a disk for an ESXi datastore. You can't use the USB disk to create a datastore.

-- David -- VMware Communities Moderator
Reply
0 Kudos
hanniex
Contributor
Contributor

Ok the 2TB limitation didn't apply for my system because the ESXi wouldn't know that a 4,5/3TB soft RAID in the VM exists right?

It  may be possible to use the disk controller in passthrough mode to a  virtual machine but the virtual machine would already need to exist on a  VMFS volume. You will need a disk for an ESXi datastore. You can't use  the USB disk to create a datastore.

Thats exactly the prob, I have a datastore on the 2TB disk after the 1,5TB raid partition where my VM will be located and I don't want to attach an extra disk just for the VMs when I have 500GB free.

Apparently I can't make a RDM on the _same_ disk where a datastore exists because vmkfstools gives error msg.

I change the disk name to 2TBHDD for better overview

# fdisk -l /dev/disks/2TBHDD

Disk /dev/disks/2TBHDD: 2000.3 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

                            Device Boot      Start         End      Blocks  Id System
/dev/disks/2TBHDDp1             1              182402 1465136001   fd  Linux raid autodetect
Partition 1 does not end on cylinder boundary
/dev/disks/2TBHDDp2             182403      243201 488367967+  fb  VMFS

When I want to mapp the whole drive like the other two drives:

# vmkfstools -r /vmfs/devices/disks/2TBHDD raidvol3.vmdk -a lsilogic
Failed to create virtual disk: Device or resource busy (1048585).

If I try to mapp only the raid partition which is not mounted or part of a datastore:

# vmkfstools -r /vmfs/devices/disks/2TBHDDp1 raidvol3.vmdk -a lsilogic
Failed to create virtual disk: Insufficient permission to access file (38).

At the time I write this I got a better idea than create the raid from scratch.

I already mapped the 1,5TB drives as RDM. The whole 2TB drive will be formated as a VMFS datastore and holds a 1,5TB virtual disk + the RDM mapping files. Then I have to rebuild the raid5 in the VM and syncing the disks. And 500GB rest will hold the VMs. I think thats better than made all drives to vmdk files.

What do you think about that plan? Do you see any disadvantages in this constellation?

Reply
0 Kudos
DSTAVERT
Immortal
Immortal

I would be very careful if you are trying to preserve the data on the existing RAID array. It won't take much to loose everything. Anything you try can be though of as experimental and prone to failure.

If you need to do this use another physical disk on a separate controller to create the VMFS datastore. Since your array is larger than 2TB it won't work as RDM so you can try to use passthrough to give the Virtual Machine control of the disk controller with the array.

-- David -- VMware Communities Moderator
Reply
0 Kudos
hanniex
Contributor
Contributor

I believe there is a missunderstanding. There is no array for the ESXi it sees only 2x1,5TB 1x2TB drive and passtrough does not work with my system. So the RDM i create is not 1x3TB but 3x1,5TB. IN the VM it would be 3TB is that a problem for the ESXi?

Reply
0 Kudos
hanniex
Contributor
Contributor

Status Update:

I screwed up everything and my data are gone. xD So I created on every single drive a Datastore and make 3 vmdks, divided on the 3 Datastores. In a VM i created an software RAID5, but the performance was 30MB less in comparision when the software RAID was on an physikal machine. So for now Iam looking for an supported SATA RAID5 controller for ESXi and then I will have 1 Big Datastore with 1,5TB partitions and 3TB storage. and mybe the 500GB left om the 2TB drive as an extra Datastore.

Thanks for all your help.

Reply
0 Kudos