exFAT problem: Major read/write performance delays after moving VM to exFAT External SSD over USB 3.0

[Lacer]

I just recently moved my VM from my MacBook Pro (15" Mid-2015, 16 GB of RAM; VMware Fusion pid=20152 version=8.5.3 build=build-4696910) to an external Samsung 850 Pro SSD in a USB 3.0 enclosure and the VM (Windows 2012 Server) completely locks up with a spinning beach ball for a few seconds at a time.  The VM performed superbly from the MacBook's internal SSD prior to moving it to the external.

There is obviously something going on with the virtualized hard disk on the external drive as I am now seeing constant entries like the following in my vmware.log every few seconds:

2017-03-01T12:46:05.655-07:00| vmx| I125: ide0:0: Command WRITE(10) took 2.088 seconds (ok)

2017-03-01T12:46:10.707-07:00| vmx| I125: ide0:0: Command WRITE(10) took 5.051 seconds (ok)

2017-03-01T12:46:13.677-07:00| vmx| I125: ide0:0: Command READ(10) took 2.359 seconds (ok)

2017-03-01T12:46:16.140-07:00| vmx| I125: ide0:0: Command READ(10) took 2.463 seconds (ok)

2017-03-01T12:46:19.216-07:00| vmx| I125: ide0:0: Command READ(10) took 2.235 seconds (ok)

2017-03-01T12:46:21.723-07:00| vmx| I125: ide0:0: Command READ(10) took 2.507 seconds (ok)

2017-03-01T12:46:24.707-07:00| vmx| I125: ide0:0: Command WRITE(10) took 2.119 seconds (ok)

CPU utilization doesn't seem to be a factor as there is low CPU utilization on both the host and guest.

I ran a speed test on the external SSD and am confident that the drive is performing normally.  The Blackmagic text reports write speeds of just under 300 MB/s and read speeds of over 420 MB/s on the external drive, which should be plenty fast to run this VM. I have successfully run an earlier version of this VM (albeit over a year ago) from this same external drive and laptop, with very good performance, albeit on VMware Fusion 8.

Does anyone have any insight as to why I'm getting such delays since moving this VM to an external drive?

Message was edited by: Lacer Modified the title to indicate the cause of the problem: an exFAT-formatted drive.

[Lacer · Accepted solution]

I think I've found the cause of the issue.  I wondered if the format of the drive could be the culprit (my drive is formatted as exFAT), so I took a second drive (a 500GB Samsung T3 SSD) plugged into the same USB 3.0 bus and formatted it as HSF+ (i.e., Mac OS Extended (Journaled)). I copied the VM to HSF+ the drive and it ran perfectly.  I then reformatted this second drive to exFAT, copied the VM over again, and the VM exhibited the same write and read delays with spinning beach ball.  I reformatted the second drive back to HSF+, copied the VM over yet again, and it again worked flawlessly. 

For good measure I formatted the first drive as HSF+, copied the VM over, and it ran perfectly without the write and read errors.

So, for some reason Fusion isn't playing well with the exFAT-formatted drive.  Has anyone seen this before?

[wila]

Hi,

Fusion just reads/writes to the file system, so it would be macOS/ OS X that is being slow when reading/writing to ExFAT.

The file system used can indeed matter, I'm a bit surprised that the difference between HFS+ and ExFAT is this noticable though.

I always use HFS+ for my external VMs, but do use FAT (not ExFAT) when I have to share with other machines.

--

Wil

[Technogeezer]

I was doing a bit of reseaech on ExFAT lately and found a posting that it does not support sparse fles. Depending on how a VMDK is configured and the I/O activity in the guest, could we be seeing a side effect of exFAT having to allocate space to "holes" in underlying VMDK files in situations where HFS+ and NTFS do not?

I'm getting the impression that ExFAT is good for flash cards and data ttransfer. It may not be so good for general file system use cases. It isnt as robust as FAT, HFS+, or NTFS (Only 1 copy of critical file system structures).

[Lacer]

I am running an enterprise app running Websphere Liberty on an Oracle database and it is very database-intensive.  The tests I ran clearly show that there is some issue with exFAT, either due to MacOS or Fusion.  No other disk performance tests exhibited any kind of noticeable difference between HFS+ and exFAT on these two external drives as I'm seeing when running the VMs.  So this could be due to, like you've said, an issue with allocating space within the VMDK files for the reads/writes that the database is performing.

One side note, after running the VM on the exFAT drives with these read/write delays, my Oracle database has become corrupted beyond repair.  Somehow these delays in writing to the disk have corrupted the database.

[wila]

Hi,

Eww. that's even worse. I run a lot of Oracle in guests (development, not production) and have really never seen an Oracle database corrupt.

If you google on "macOS exFAT slow" you'll find a lot of complaints.

It's nothing to do with sparse files, VMware Fusion doesn't use that.

--

Wil

[ColoradoMarmot]

and worse, exFAT is highly prone to corruption as it only maintains a single FAT table.  It's really not designed for persistent storage - more, as noted, for cross-platform temporary storage like flash cards and drives.