ZFS as iSCSI target

[nitin_m]

Dear All

I have planned our installation as follows -

a) 2x DELL 2950 with USB hack (ie, install ESX3i installable on USB stick)

b) White box NexentaStor appliance 1TB as iSCSI target.

Now, my question -

There is no reference on these forums on Nexenta or ZFS. Is there anybody on this forum who has used Nexenta as iSCSI target?

I am divided if I should use "Unproven" Nexenta or "Proven" Openfiler.

Please note - I will be using PERC6i to create 6x146GB into RAID5. I am assuming RAID 5 using 6 disks is going to give me better performance than RAID10.

Also, if there are any ZFS gurus here - Will I loose ZFS advantage if I use hardware RAID5? Should I let ZFS handle everything and present all 6 discs to Nexenta and let it handle?

[kjb007]

I'm assuming you're speaking of Solaris here. I've read a lot of good things about ZFS, but I suppose I'm a bit old school. I let hardware do as much as it can before I layer my software on top. This is especially true for RAID. Let your hardware RAID controller do what it does best and manage your array, and load your filesystem on top. You may lose a bit of what ZFS does as far as implementing it's own redundancy at the disk level, but the other plus's are still there. If your'e starting out with a hardware RAID set, when you need to add more space, you can add another hardware RAID set, and have ZFS add it into your volume. Get the benefits of both without really losing the benefits of either.

-KjB

[nitin_m]

But - Does it work (as in Nexenta as Target)

I want to know as many chap solutions (like mysan?) dont have scsi release/slect feature rendering it useless for ESX.

So, has anybody used Nexenta as primary storage successfully?

[kjb007]

I think you may have answered your own question in your first post. Why would you use something no one has experience with over something very often used, ie Nexenta vs OpenFiler?

-KjB

[nitin_m]

Kjb007 - If everybody started thinking the way you do, we would have no takers for new technology.

I asked this question to draw from the experience users may have in this board.

Nexenta with its technology (ZFS - which is very near to WAFL), we are getting enterprise features without paying the tax of heavyweight companies (isnt this waht fsf is all about?)

[hodgts]

I too would be very interested in hearing if anyone is using this product, especially with a Sun Fire x4500.

[mcowger]

The Solaris iSCSI target doesn't work all that well for this - it plays weird games with the Page 80/83 IDs, which make multipathing/multitarget not work well.

Tried it, decided against it (as much as i LOVE ZFS).

--Matt

[nitin_m]

Look at this link.This guy is using solaris target.

http://scottf.wordpress.com/2008/03/12/solaris-iscsi-target-with-esx-302-server/

When did you try? The way I am looking at Solaris threads, major development occured around mid year last.

[nitin_m]

What if I use NFS part of Nexenta?

I believe NFS is fairly standard and should work?

[nitin_m]

Reply to myself -

Tested as -

Dell 2950III as ESX,

Nexenta under VMWARE Fusion,

ESX sees the target, but fails with "invalid file system" (unsure what the exact message was). I have read the message in these boards when the target doesnot support release function.

Summary - Nexenta Target doesnot work.

[mcowger]

I tried about 3 or 4 mos ago, and submitted a bug to the ZFS devs.

That guy is also only using a single LUN, which does work fine. Wait till he adds a second...it wont work.

--Matt

[crichardso]

I have been using Solaris comm b90 edition and the 2008.05. The problem I am having is that the iscsi method esx uses requires unique serial numbers of the disks which is causing it to fail when you goto add it as storage. Basicaly opensolaris does not set a serial on the luns and there for is left as blank or unset when you goto add it to esx it will format it correctly. you can see this via the command line. but when it goes to do the mapping it fails because there is no serial number assocatied with the drive to symbolicly the lun to the volume name

[mike_laspina]

Hi,

I believe NFS works very well using Nexentastor.

[crichardso]

yes we might due that but it does make me a little bit upset that i have to take the performance hit and other shortcomings that come with nfs to make it work with esx.

[mike_laspina]

I have good news on the ZFS/iSCSI/VMware issue.

Mr. Jonathan Schwartz is very committed to making ZFS work with VMware.

From my communication with him self and the response Sun's storage VP Victor Walker

I can assure you that we will be able to use ZVOL's over iSCSI on VMware within the next couple of month's.

I will be testing the current code changes that have addressed the issues to date and I will post the results as we progress.

I will also place the updated results on my blog when they happen.

Regards to all,

Mike

[mcowger]

As of opensolaris ~b85, this issue is already fixed. Now just need to wait for a real Sol10 release to include that kernel.

--Matt

[mike_laspina]

Solaris 10 x86 is what I am working with so I am pleased there will be a solution for it soon.

[crichardso]

it might have been fixed in b85 but when they went to esx 3.5 they changed iscsi stuff. I can tell you for a fact it does not work with esx 3.5 and snv_90 either sxce or dev edition. It sees the LUN but when it formas it it goes to link the volume to the disk which apears to normaly be done by creating a symbolic link of the volume name to the serial number of the disk. But opensolaris does not assign serialnumbers to exported zvols so it ends up trying to map to any existing serialnumber disk that is already there.

-Chris

[mike_laspina]

I have it working with snv_92 so that issue is now corrected and will be included in the July patch releases.

I did however have it working on snv_90 so you may want to check into it a bit more.

How did you present the target?

Here is the command I am using.

iscsitadm create target -u 0 -b /dev/zvol/rdsk/yourpool/yourzfsvol yourtargetname

This method allows you to set the backing as the raw volume path.

NexentaStor may work with b85

[crichardso]

i did it with zfs create -V then zfs set shareiscsi=on.

What version of esx are you using? cause it did not work with esx 3.5 with the latest updates