VMware Cloud Community
ViennaAustria
Contributor
Contributor

ESXi 5.x on new Apple Mac Mini 6,2 Late 2012 *NOT* working

Hello!

I bought one of the new MacMinis, which were presented yesteraday, plugged two 8GiB DD3-1600 modules in (OsX reported 16GiB of memory, as the models before) and tried to install ESXi 5.1 Build 799733. It started promising:

Initializing chipset...

Initializing timing...

Initializing scheduler...

Initializing processors...

Initializing ACPI...

*BANG*

Pink screen! Smiley Sad

NOT_REACHED bora/vmkernel/hardware/intel/vtd.c:3638

cr0=0x8001003d cr2=0x0 cr3=0x449ad500 cr4=0x12c

*PCPU0:4096/bootstrap

PCPU  0: SISISISISISISIS

Code start: 0x41800c600000 VMK uptime: 0:00:00:04.301

and a stackdump.

macmini.png

I tried instalkling 5.0 and 5.0 U1 also, but with the same result.

It is the regular MacMini with an i7 4C 2.3GHz and one 1TB SATA drive (http://store.apple.com/us/configure/MD388LL/A with no extras), not the MacMini Server.

Any suggestions? Thanks!

PS1: I was unable to boot the ESXi setup from USB key! Since other USB keys (OsX setup for example) didn't work either, I suspect that there's some EFI boot problem from the new USB3 ports. Booting from CD worked, however.

PS2: on the earlies models we had the problem, that the Alt key wasn't recognized after powering on, when there is an USB hub between the keyboard and the Mac (a KVM switch for example). That seems to be fixed now: whenever I press the Alt key at the powerup chime, I get the Apple "boot menu", even with our KVM switch in between.

338 Replies
ViennaAustria
Contributor
Contributor

@sirozha1:

Most questions are already answered in this thread. Just look back some pages.

USB3 obviously has issues on the new Mac. 3 out of 5 USB3 keys we tried are not recognized on the "Alt" UEFI boot menu. One even completely hangs the Mac, after it's inserted. All the USB3 Keys, however, ARE working, when OsX is running. So I expect some UEFI update soon.

USB3 as boot medium for ESXi won't give you the edge. If you choose some quick USB2 key, ESXi will boot up in 10-20 seconds. USB3 probably won't change that too much. Besides, how often DO you (re)boot ESXi hosts? Smiley Wink

I'm not a pro on the Mac. But with the prevoius MacMini Server we had problems, when ESXi was installed on the other drive and not on that one, which booted OsX. We found no way to change the boot device later [except manualle every boot with the "Alt" key]. So you might have a basic problem to boot from USB without manual intervention on each boot. Why don't you install ESXi on the internal drive? Every MacMini has one (or two). Simply use it. It gives you the same non-redundant solution, like booting from an USB key (which can accidentally be unplugged and has less performance).

Regadring the "Thunderbolt Drobo", you mentioned: I looked at their website. I found no information, how the RAID is being controlled. If it would be any kind of software RAID (like WD "My Book Thunderbold Duo" for example), you can foget it on ESXi. If there is a hardware RAID controller working in it, hat I believe to be the case, you have to find out, which one and if it's on the VMware whitelist. If it is, it will work. Thunderbolt makes a transparent PCIe bridge.

When connection multiple Thunderbolt devices to the MacMini, keep in mind, that it has just ONE Thunderbold connector and devices like the Apple Gigabit Adapter have no chaining output.

Reply
0 Kudos
donnie80
Contributor
Contributor

Thanks Will.

I tried both (replacing and appending) and I have found that if I install ESXi to a USB3 key, settings will be lost upon reboot regardless how I pass the boot options. Installing ESXi to the internal SSD drive works and keeps settings upon reboot.

When installing to USB3 key, the /bootbank/ keeps pointing to /tmp and hence any config changes are written to a state.tgz file which resides on tmp. Upon reboot tmp is gone!

I've searched quite a bit but to no avail. If anyone knows, please yell.

NB: I have two USB 3.0 keys. One contains a modified bootable ESXi-custom installation (5.0u1) and the other USB is where ESXi is installed too.

Reply
0 Kudos
sirozha1
Contributor
Contributor

@ViennaAustria: Thanks for the detailed reply. I see your point about Drobo Thunderbolt devices. I need to contact Drobo to get this answer. I guess the only real RAID enabled solution for the datastore for me is my ReadyNAS Pro 6 used as the iSCSI target for my current ESXi server (Dell PowerEdge), which I will have to re-use for the ESXi server running on the 2012 Mac Mini.

To that end, does ESXi support the Wi-Fi Ethernet controller in the 2012 Mac Mini? I would like to use the Mac Mini's Gigabit Ethernet port as the dedicated port for iSCSI, and use the Mac Mini's Wi-Fi adapter for the VMs to access my network. If the Wi-Fi Ethernet controller is not supported by ESXi, is the Thunderbolt Gigabit controller fully supported by ESXi?

Lastly, how would the performance of the 2012 Mac Mini with 2.6 GHz i7 quad core and 16 GB of RAM compare to my 2010 Dell PowerEdge with one Xeon 2.4 GHz e5620 CPU and 16 GB of RAM? Will it be comparable, or will I see a serious performance drop? I need to run about 8 VMs concurrently in my lab. At least six of them are Red Hat Enterprise Server 4 based VMs, one Windows 7 (or Windows 😎 VM, and I'd like to be able to run a Mountain Lion Server in the future when it's supported. I suspect I may run low on the RAM, but I'm hoping 16GB DDR3 1600 MHz modules will become available soon, and I will be able to bump the memory up to the maximum that this CPU supports - 32 GB. My primary concern is the CPU, though. Can it handle all of these VMs concurrently? I don't ever expect to exceed 10 VMs on this server. 

Thanks!

Reply
0 Kudos
ViennaAustria
Contributor
Contributor

I guess the only real RAID enabled solution for the datastore for me is  my ReadyNAS Pro 6 used as the iSCSI target for my current ESXi server  (Dell PowerEdge), which I will have to re-use for the ESXi server  running on the 2012 Mac Mini.

In most cases, SAN/NAS is to be preferred. DAS is usually only used in small applications, where you need "one box for everything". If you already have some SAN/NAS, be happy and use it for both machines.

Buy the US$ 29,- Apple Thunderbolt GbE for a redundant uplink.

does ESXi support the Wi-Fi Ethernet controller in the 2012 Mac Mini?

No.

afaik, ESXi does not support any WiFi.

Lastly, how would the performance of the 2012 Mac Mini with 2.6 GHz i7  quad core and 16 GB of RAM compare to my 2010 Dell PowerEdge with one  Xeon 2.4 GHz e5620 CPU and 16 GB of RAM?

I cannot give you a precise answer to that, off course, but since the Dell is 2 years old and the MacMini has newest 22nm technology and 1600MHz DDR3, it should perform slightly better, than the Dell. All the ESX hosts we resp. our customers use (70+) suffer from lack of RAM and not lack of CPU power. Smiley Wink

Reply
0 Kudos
sirozha1
Contributor
Contributor

@ViennaAustira: Thank you again!

Reply
0 Kudos
zer010gic
Contributor
Contributor

@sirozha1

Drobo uses a proprietary form of RAID implemented on their box.  What they present to the host system does not expose that RAID is being done its just a volume/drive.  I just don’t know what drivers are needed for thunderbolt based storage on the host system.

Reply
0 Kudos
HPReg
VMware Employee
VMware Employee

> On Monday I can send somebody to fetch it. Then we can try VMware Fusion. I probably won't have the required information until Tuesday. Will that be okay?

That will be perfect! Thanks for helping with this!

Reply
0 Kudos
oturn
Enthusiast
Enthusiast

This is possibly the most capable and technical gathering of Mac Mini / ESXi users on the Internet. An added bonus is the participation of VMware in this discussion.

I'd like to take this opportunity to mention a problem I've seen with my 2011 Mac Mini, and pose a general question about ESXi vs. Fusion.

ESXi on my 2011 Mac Mini Server experiences an occasional PSOD when pushed VERY hard. This has happened with 5.0, and 5.1. The Mini will run for weeks or months under normal load. I mentioned this problem near the end of the following thread:

http://communities.vmware.com/thread/326998?start=15&tstart=0

Here are parts of the PSOD error:

PCPU 0 locked up. Failed to ack TLB invalidate (total of 1 locked up, PCPU(s): 0).

*PCPU5:2805/hostd-worker

KB Article:

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=102021...

Could this be a CPU microcode issue with the 2011 Mini? Might it be resolved in the 2012 model?

I am running the latest NIC drivers.

After experiencing a PSOD 3 or 4 times, I've begun considering a switch to Fusion under Max OS X. My assumption is that OS X would remain stable when being pushed hard by Fusion. Is this a reasonable assumption? How should I expect my VM's to perform in Fusion, vs. ESXi?

Reply
0 Kudos
willplaice
Contributor
Contributor

Thunderbolt...

I don;t think that thunderbolt raid will be available to ESXi....

but the 4 channels are available for passthrough, so i've ordered a Drobo, let's hope my OSX VM can see it....

i can live with the VM's on a single disk as i can mirror it occasionally for backup, i can then put the media/critical data on the raid.

i'll let people know if this works 🙂

Reply
0 Kudos
HPReg
VMware Employee
VMware Employee

Could this be a CPU microcode issue with the 2011 Mini? Might it be resolved in the 2012 model?

It could be. If you have not officially reported the PSOD to VMware (the community forum is not the right place, use a Service Request), you probably should.

After experiencing a PSOD 3 or 4 times, I've begun considering a switch to Fusion under Max OS X. My assumption is that OS X would remain stable when being pushed hard by Fusion. Is this a reasonable assumption?

It is a reasonable assumption. Apple is likely to test Mac OS on the 2011 mini much more than we do ESXi.

How should I expect my VM's to perform in Fusion, vs. ESXi?

It depends on the workload. If your workload is CPU-bound, then you should not see any difference. If you workload is I/O-bound, then the VM will perform better on ESXi than on Fusion.

Reply
0 Kudos
vSean1
Contributor
Contributor

I have also seen the PSODs on the 2011 mac mini server edition.  It is easily reproducible, just throw a long handbrake queue at a vm with 2 or more vcpus.  It may take a while, but eventually it will PSOD with a pcpu lockup.  My personal suspicion is its a thermal issue aggravated by turbo boost, but that's completely non-scientific conjecture.  In any event, using CPU limits to keep VMs within the non-turbo mhz capacity of the core seems to stop it from tipping over.  If some at vmware wants a core dump that can be arranged, but calling for support on my obviously unsupported config never crossed my mind.

Reply
0 Kudos
ViennaAustria
Contributor
Contributor

@HPReg:

> On Monday I can send somebody  to fetch it. Then we can try VMware Fusion. I probably won't have the  required information until Tuesday. Will that be okay?

That will be perfect! Thanks for helping with this!

I was unable to (re)install OsX on the MacMini. :smileyangry:
Apple Support had no additional ideas, so they're sending me a new one within 48hrs. Then I can perform the test. Sorry for the delay!
Reply
0 Kudos
notsleepy
Contributor
Contributor

I've tried installing via SuperDrive on the new 2012 Mac Mini server using the standard 4 GB (two 2GB) of 1600MHz DDR3 memory and I can't even get started. I received a purple screen upon install that complained "No usable memory below 4GB" Smiley Sad

Has anyone else hit this wall? I'm going to try third party RAM from Corsair later.

Reply
0 Kudos
oturn
Enthusiast
Enthusiast

@vSean1

I mentioned heat as a possible culprit in the other discussion thread. This was primarily because the system fan is fulling full-on when the error occurs.

What limit are you using on your VM's? My primary VM's have 4 vCPU's assigned, so the unlimited maximum is listed as 7980 MHz. My Mac Mini has the 2.00GHz i7-2635QM processor, with a max turbo frequency of 2.9GHz.

I'd also hesitate to call support about an unsupported system! Do you plan on purchasing the 2012 Mac Mini? It would nice to see if your hanbrake queue PSOD is resolved on the 2012 Mini.

Reply
0 Kudos
ViennaAustria
Contributor
Contributor

@HPReg:

I was able to get hand on another MacMini6,2 Late 2012.

1) Install Mac OS 10.7.x or 10.8.x as a host (instead of ESXi) on the Macmini6,2.

2) Install VMware Fusion 5.0.1 (trial version if needed).

3) Copy or create a Mac OS VM, and try to boot it.

Do you see the same issue or not? Depending on your answer, we will know much more precisely which part of our code is causing the issue.

Installing OsX 10.8.2 in VMware Fusion on the new MacMini worked without any problem.

fusion.png

So the problem occours only with ESXi (5.1) on the new MacMini.

What now?

Reply
0 Kudos
willplaice
Contributor
Contributor

Can anyone help me out with an iso for 5.01 that will boot on the 2012 mini - USB passthrough support is apparently not working correctly on 5.1 so have to use the older version.

thanks very much, in advance - can post a dropbox link if anyone can help - can you PM ?

Will

Reply
0 Kudos
sirozha1
Contributor
Contributor

@willplaice:

So, what's not working for you in VMware ESXi 5.x on a 2012 Mac Mini?

I've just combed through every post in this thread, and it appears that initially, you were having problems with the NIC driver and with booting, but apparantly, you resolved those.

There's also a post from @zer010gic with a link to a modified VMware ESXi 5.x ISO that doesn't require the NIC driver hack, and all one needs to do is to use the <space>iovDisable=TRUE string twice (during the initial installation and after the first reboot). Then, one needs to enter the

esxcli system settings kernel set --setting=iovDisableIR -v TRUE string that would allow the setting to persist across future reboots of ESXi.

Did this work for you? Also, what version of VMware ESXi 5.x is the ISO posted by @zer010gic? Is it ESXi 5.0.1 or ESXi 5.1?

My Ebay auction for Dell T410 server (currently used in my home lab for running ESXi) is closing in 4 days, and I need to know if running EXSi 5.x on a 2012 Mac Mini works so that I don't end up having sold my Dell server at a big loss and not being able to use a 2012 Mac Mini to run ESXi 5.x.

Thanks!

Reply
0 Kudos
ViennaAustria
Contributor
Contributor

@sirozha1:

So, what's not working for you in VMware ESXi 5.x on a 2012 Mac Mini?

As you perfectly summed up, we have workarounds for every problem encountered so far. Windows, Linux, BSD, ... VMs are working with ESXi on the new MacMini.

OsX VMs, however, are NOT WORKING, because of some problem with the SMC module, as dariusd found out several days ago in this thread.

If you would install VMware Fusion on OsX on the new MacMini, then everything works, including OsX VMs. So I hope it's not too big a problem for VMware, to solve the issue.

Reply
0 Kudos
willplaice
Contributor
Contributor

just to clarify - NOT WORKING is USB passthough, but this is a 5.1 issue NOT a MAC issue

I have OSX running fine using donk's modifications.

Reply
0 Kudos
notsleepy
Contributor
Contributor

On the newest version of Mac Mini I added the same RAM that @ViennaAustria recommended:  Corsair Vengance CMSX8GX3M1A1600C10

I'm still getting "No usable memory below 4GB" and cannot proceed with installation.

Am I using hardware that nobody has tried yet? If so I guess I'll return it and look for an older Mac Mini but I thought I understood that you guys had it working on the newest.

My spec:

Apple Mac Mini MD389LL/A with Lion Server

Processor2.3 GHz Intel Core i7
RAM4 GB SDRAM
Hard Drive2 TB
Graphics CoprocessorIntel HD Graphics 4000
Brand NameApple
Item model numberMD389LL/A
Hardware PlatformMac
Operating SystemMac OS X Mountain Lion
Processor BrandIntel
Processor Count2
Computer Memory TypeSDRAM
Hard Drive InterfaceSerial ATA

The machine boots the preinstalled OSX just fine so there doesn't appear to be a problem with the hardware.

Reply
0 Kudos