How to replace ESXi properly to retain iSCSI datastore?

[tiger74]

Hi guys,

I'm exploring ESXi on a usbstick. It connects

to a NAS using iSCSI. It works fine, until I screwed something on the

ESXi configuration that it cannot be managed again from VI Client.

So,

I replace the usbstick with a new one. Setup ESXi again and the iSCSI.

It finds the LUN, but it gives warning that using that LUN will destroy

any data in it, which it destroys happily when I click OK.

So,

in this case, is there proper way to regain the datastore intact if we

need to change the ESXi? Or, there's no such scenario for it?

Thank you.

[mreferre]

Interesting.

I am not sure why it asks to destroy the content. If it was empty it should present an empty volume, if it was a VMFS volume it should attach it as-is (think what happens when you connect the second host to a cluster with a VMFS LUN already formatted).

Are you sure you are not presenting another LUN to the host with a non-VMFS file system?

No idea.

Massimo.

[Texiwill]

Hello,

Thread moved to ESXi forum.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

Blue Gears and SearchVMware Pro Blogs: http://www.astroarch.com/wiki/index.php/Blog_Roll

Top Virtualization Security Links: http://www.astroarch.com/wiki/index.php/Top_Virtualization_Security_Links

[mail_vijayar]

What kind of ISCSI storage are you using in the back end? was the ISCSI IQN name updated on the ESX server to match the old IQN name and do a rescan on the ISCSI HBA and you should be able to get back the old LUN without any data loss.

ISCSI is a block level protocol and will erase the data store when being dismounted, i case you need to dismount eh ISCSI LUN do it from the storage where you can safely dis allow access to the the particular ESX server.

Regards,

Vijay

[tiger74]

Thanks for responding, so far this issue is the only blocking block that makes me really insecure in using ESXi on usbstick.

Here's the story:

First, I created a 5GB LUN on the NAS and used it as datastore.

Then I expanded the LUN (from the NAS) into 20GB and when I rescan the datastore, ESXi recognized that there's additional extend available to add. So I added into the datastore making it 20GB (one big datastore, not two separate one).

Then I screwed up. Ending installing new ESXi on another usbstick and run the host using the new stick.

In the new ESXi, I setup iSCSI again. This time it has different initiator-id than previous ESXi.

Setup the iSCSI permission for the new initiator id on the NAS, and connect to it from ESXi.

When connecting, ESXi provides me two choices:

1. Use a "spare space" of 5GB

2. Use the whole space (20GB) but mean destroying any data in it (I chose this).

So, do you think it's interesting?

1. Should I setup the ESXi iSCSI id to be the same as the previous one?

2. Are you sure the new ESXi should be able to recognize the existing datastore in the LUN and use it without destroying it first?

The NAS is IBM NAS A3300 using Data ONTAP 7.xxx. Does it matter?

Thank you.

[mike_laspina · Accepted solution]

Hi,

In order for a VMFS store to be reintroduced to either ESX or ESXi the VMFS header of the volume must be readable and match the EUI / NAA that is defined on the hosts storage vml paths created in the esx.conf file or the VC DB . If this information is not present then the volume will not be presented as a usable existing VMFS store. I have a blog with some examples of reinstating VMFS volumes that are iSCSI based stores running on OpenSolaris and ZFS.

You can examine the EUI or NAA on any working VMFS volume and ESX with the following command.

esxcfg-vmhbadevs -m

This command gives you the dev name that are associated with the volume and with that info you can use hexdump like the following.

hexdump -n 1100000 /dev/sda1

The will present the header hex data like this example output, the bolded value is the stores guid.

0100000 d00d c001 0003 0000 0010 0000 1602 0002

0100010 0600 5553 204e 2020 2020 4f43 534d 4154

0100020 2052 2020 2020 2020 2020 2e31 2030 0160

0100030 f044 d47e 0004 0000 2a49 142e 0400 4f43

0100040 534d 4154 0000 0000 0000 0000 0000 0000

0100050 0000 0000 0000 0000 0000 0200 0000 f800

0100060 ffd4 0031 0000 0001 0000 031f 0000 031e

0100070 0000 0321 0000 0000 0000 0000 0110 0000

0100080 0000 3eee 492a 7cb8 960f 16c2 0200 6755

0100090 4e14 7fda dec9 5c68 0004 cf98 decf 5c68

01000a0 0004 0000 0000 0000 0000 0000 0000 0000

01000b0 0000 0000 0000 0000 0000 0000 0000 0000

With this info you can recreate the vml, but it is challenging since you need to know the format, but we can go by example.

Use the following command to see what exists in a working host.

cat /etc/vmware/esx.confcat /etc/vmware/esx.conf | grep vml

/storage/lun/vml.0200020000600144f07ed404000000492a2e140004434f4d535441/adapteriqn.1998-01.com.vmware:vh0.1/targetiqn.1986-03.com.sun:02:1eddb8ec-cc09-62d0-b867-8da28dee9609/

So here you can see the part that needs to match in blue.

Also as a word of advice on usb based ESXi boot devices, you should not run that type of config unless the creator of the usb image provisioned a ramdisk for the swap area.

Multi layer flash devices will fail quickly if the swap partition area is on the flash device.

Regards,

Mike

[tiger74]

Oh sh*t ! (pardon me for the expression). Looks scary!

I create the usbstick from the image from original ESXi. Is the swap file on a ramdisk? Also ESXi only has Remote CLI right, it doesn't have that unix/linux command like cat and scp?

How about this: Can I copy the usbstick to another stick as a backup using dd?

Thanks Mike!

[tiger74]

Hi Vijayar!

Good news! You are absolutely correct. I ran another test by creating a new ESXi usbstick. Boot it, setup the iSCSI initiator name with EXACT id as previous usbstick, and it WORKS!

It recognized the datastore! Waw!

Many thanks Vijayar!