Workaround to use Coraid AOE as VMFS in ESXi

[ermockler]

I have been waiting for the driver announced by Coraid in september, that enables ESXi to natively use AOE targets as VMFS datastores. I'm sure many others are waiting so I am posting my workaround, any suggestions for improvements are certainly welcome.

The provider of the AOE initiator for windows is Rocket Divisio's starport. They also have an iscsi target called starwind.

I created an xp vm on the ESXi host (on local storage). Using a vm locally on the ESXi box seems to introduce the least amount of overhead for this "box in the middle" solution. Added starport and starwind. I attached to the AOE target with starport, and published the same block device as iscsi with starwind. I attached to the disk with ESXi and formatted it VMFS. I put some test vm on it, and can access this from other ESXi hosts. This is working pretty good so far, lasts across reboots from all sides, etc. I would like to have the box in the middle power on with the host, so far I haven't seen how to do that...

I am hoping that when I switch to the promised native driver I am not forced to wipe the storage. I am using Coraid's VS21 to mirror AOE cabinets, and will certainly break the mirror before trying the new driver. (after testing on a test casbinet)

According to Coraid, the driver will include a Coraid branded nic, so I think when the time comes I will just be adding an additional path to the same storage.

[nick_couchman]

Do you know for sure that the driver is going to be for ESXi as well as ESX? The site only mentioned ESX, and ESX and ESXi are different enough that I wouldn't be surprised if it works in one and not the other. We'll see, I guess.

An alternative work-around is to use a Linux-based system for this task. Last I checked (maybe it's changed since then) Rocket Divisio charged $$ for both the AOE Initiator and the iSCSI Target Server for Windows. You can get both the AOE drivers and the iSCSI Target Server for free in Linux, and configuration isn't too difficult (not GUI-based, but there isn't a whole lot to it). Configure Linux to start up the AOE driver and connect to the AOE target, then configure your iSCSI Target Server (IET is one of the more popular ones) to share out that block device. Then point ESX to it and go!

[ermockler]

I thought about doing that, downloaded freenas and a couple others that

had very out of date AOE tools and no package management. A couple

hours into it I decided it was too much work for a supposedly temporary

workaround. I used Winlite for a minimal xp with 2 nics. You get 30

days of a trial from Rocket Division, which should be enough for a

workaround. If you need longer I think it's only $200 for both. Running

trial software on a vm with snapshots would probably allow some

creative individual to see some possibilities.....

I spoke to Jim Kemp from Coraid on the phone and the driver is for both ESX and ESXi.

[MattG]

http://www.msnbc.msn.com/id/29756218/

CORAID Releases EtherDrive SAN Host Bus Adapter for VMware ESX 3.5

[ermockler]

This wasn't really a question, I was just sharing the workaround. I participated in the Coraid beta of the driver, so I was alerted by email of the release of version 1.0.1.

The workaround is still useful, since it will let you use a vblade target, presumably for "tier 2" storage, backups, redundancy, etc. Using this with ESXi you could knock together a san and some hosts for NO money, and makes you more comfortable pluncking down for enterprise class hardware and management . I suppose it may be worth building the linux aoe-->iscsi vm after all.

[Dave_Mishchenko]

So did you use their HBA for your testing? And was this connecting to their SAN product or to Starport?

[ermockler]

I bought a Coraid SR1661 late last year and they sent me out the HBA about a month b4 the beta started (about Jan 09). I wanted to play with the storage and came up with the workaround. VM's created on Coraid storage were visible to ESXi using the HBA and the earliest beta driver. I couldn't use the HBA at all without the driver.

Now I have 2 ESXi boxes, and each uses Coraid storage via the HBA and Vblade storage via the workaround. I need to work out the exact method, maybe using VEEAM or something to replicate to the cheepo vblade storage. Starport is just the AOE initiator for windows, using that on the backup box for disk2disk as well as temporarily in the aoe->iscsi vm.

[Dave_Mishchenko]

Thanks - just I'm clear the HBA is working ok with vblade? I took a look at the driver and is the HBA a rebranded Intel NIC?

[ermockler]

No, you cannot attach to a vblade target using the HBA. And you need the latest firmware on a Coraid unit to attach to it. Maybe vblade-20, when it's released. the latest is vblade-19.