Guest OS (Mac) is bloated by about 70GB of empty space

[agtester]

Hello,

I am having a terrible time trying to figure out what is bloating my vmware fusion VMDK. I have tried everything from running apps like OmniDiskSweeper within the guest OS to find hidden large files that are using the space, manually searching for files in the system with hidden files shown, to the "disk shrink" procedure described here: How do I clean up an OS X VM? , to deleting sleepimage, but nothing is working. I have shutdown, rebooted, and all of the "obvious" things.

I have had to enlarge the available drive space for my guest OS multiple times now just so that it can operate.

In actuality (and as confirmed by OmniDiskSweeper) I am only using about 11GB in "real" files. I cannot find how/where/why the extra ~70GB is being used.

Does anyone have any information that could help me?

Important info/specs:

Version of VMware Fusion: 10.1.3

Host OS: Mac 10.12.6

Guest OS: Mac 10.8.5

Total space allotted to Guest OS: 100GB

Currently claims to be using approx 80gb

Thank you,

AG

[dempson]

When you say "Currently claims to be using approx 80gb", where are you seeing that figure?

In particular, what do you see in the guest Finder's Get Info for its hard drive? Is that figure also about 80 GB , or is it showing a more reasonable number around 11 GB?

Depending on the answer to that question, there are two possible routes to tracking down the problem.

(a) If the guest Finder's Get Info shows about 80 GB of used space, then the problem is likely to be files in a location that your user account doesn't have permission to access, so OmniDiskSweeper isn't counting them. You'll need another method to search with admin privileges (which I can detail once I know this is the case).

(b) If the guest Finder's Get Info shows about 11 GB used space, then double check using the following command in Terminal:

df -h

The df command shows disk free and used space. The -h option displays the results in human-readable form, e.g. KiB/MiB/GiB rather than number of blocks, but note that it uses base-2 units, whereas Finder uses base-10, thus df should show numbers that are about 2% higher than Finder for KB, 4% higher for MB, 7% higher for GB.

Finder deliberately hides some things from its reported used space, in particular Time Machine's Local Snapshots (which should only exist if Time Machine is enabled in the VM). They are supposed to be deleted automatically by the system as free space gets low. df will show the actual used/free space, so Local Snapshots count toward its used space.

If df also shows about 11 GB used then the explanation is probably something to do with the structure of the VMDK and VMware's ability to shrink it.

[ColoradoMarmot]

What does disk utility (not finder) show for free space inside the guest?

Make sure you disable time machine in the guest, then open terminal and do

tmutil listlocalsnapshots

It shouldn't return anything.  If it does, you'll need to use tmutil to manually delete each one in turn.  Once that's done, reboot and you should be clear.

[dempson]

tmutil listlocalsnapshots won't work on an OS X 10.8.5 guest - that verb was added in macOS 10.13, as were the verbs for thinning or deleting individual snapshots.

The mechanism is quite differently implemented on HFS+ and APFS volumes: HFS+ only copies relatively small files in limited areas (such as user documents), whereas APFS does a snapshot of the entire volume so can use a lot of space. It would be surprising to be able to accumulate 80 GB of local snapshots on an older system like Mountain Lion which must be HFS+ and which only has 11 GB of real used space (much of which would be the OS, which typically doesn't get added to the local snapshots).

If local snapshots are looking like they may be the problem after all, that mechanism can be disabled, which initiates a cleanup:

sudo tmutil disablelocal

There is a corresponding enablelocal verb if you want to enable them again later.

[agtester]

Thank you for the insight. TM is (and has always been) disabled on this VM.

"df -h" shows close to the same figure as Finder's Get Info, which is also the same figure as Disk Utility's usage info.

Perhaps since the last reboot, that number is now closer to 75GB total (versus the previous 80GB), but the problem remains. It's still 64GB higher than my "known" usage.

I think you might be onto something with my user privileges not being able to access that extra "unknown" space.

Does this mean I need to log in as root and try to find that chunk of data? I don't think OmniDiskSweeper can run in root mode, so would I have to do a manual search instead? Or do you know of any tools that would help?

[dempson]

If Finder, Disk Utility and df -h are all showing the same "used" figure and that is close to the total reported by OmniDiskSweeper,  then the problem is not hidden/protected files. If there were any such files, their size would count toward the used space on the volume as shown by Disk Utility and df (and Finder, apart from it pretending local snapshots are free space).

Some remaining possible explanations which occur to me:

(a) Does the VM have a partitioned drive and one of the other partitions is using a lot of space? You can easily check this in Disk Utility.

(b) Corrupted file system, such that the actual used space and reported used space don't match. You can run Disk Utility's "First Aid" in the guest to check for this, but if it finds any problems then fixing them might be tricky because you need to either boot from a different volume (e.g. OS X Recovery) or start up in single user mode (Command-S at startup) and use 'fsck -fy'. Special keys at startup are difficult to time right with a VM (unless there is some trick I haven't learned).

(c) Something to do with the shrink mechanism not working as planned (e.g. the file system wasn't defragmented), so the file system still has used space around the 80 GB mark and cannot be shrunk smaller than that, despite having a lot of free space before that point. Is it a monolithic VMDK rather than being split into multiple files?

[agtester]

dempson, thank you for all of the thoughts - before I respond to them, I want to point out that OmniDiskSweeper is the only source that is not reporting the same size. It is the only source reporting 11GB. So, I really do think that there is a file(s) somewhere outside the reach of normal user mode taking up the space.

Note: I made sure to clear all cache locations that I know of, as well as /tmp.

Here are responses to everything you mentioned:

(a) The VM does not have partitions.

(b) The drive passes verification in First Aid. As a failsafe, I booted into single user mode (see my advice for special key commands on bootup! :] ), and I ran 'fsck -fy', and it passed there as well. Just for kicks, I also did 'df -h' to see if it would give me anything different, and it did not. Then, I exited the console which allowed me to boot into the OS as single user, and ran OmniDiskSweeper just to see what would happen, but it still told me 11GB. So, nothing new to report in any of this.

***Tip: bootup key commands are actually very easy to execute in VMs - first make sure the VM is "in focus" by clicking on the screen immediately after clicking the play button when you start it up, then hit the key command as soon as you see the gray VMware screen - NOT the gray Apple screen like a real Mac. That should always work. I have been able to boot into single user mode as well as safe mode this way!

(c) I think the VMDK is monolithic (...?) I mean, it's the only VMDK file in the folder when I do "show package contents" on the VM package. I'm attaching a screenshot so you can see. Note: I'm also showing the Inspect window for those other files, so you can see that they are not large in file size.

Do you think there are any other things I can do to investigate this matter, or am I just stuck with the problem?

[agtester · Accepted solution]

I figured out the problem! It turns out /private/var/spool was inundated with a massive collection of old print queue files that were not being deleted upon job completion. I do a lot of printing with this VM, and it looks like there is an issue with the app that I am using to do the printing. I am not sure if this is VM-related specifically, but I appreciate your help anyway. I have marked all of your replies in this thread as helpful.

Thank you for your time in talking me through this matter. In an indirect way, it led me to this solution.

[ColoradoMarmot]

I missed the version - thanks!

[dempson]

Glad to see you found the problem, and thanks for the tip about timing of startup key shortcuts.

I misread your previous message in which you mentioned 75 GB as being the figure reported for used space (I somehow missed that paragraph), therefore the rest of my previous post was going down the wrong rabbit hole.

In case it comes up again in future: the next thing I was going to suggest to track down unexplained usage is either to use DaisyDisk (which has an option to run as administrator but is not free) or the command line utility 'du' (disk usage) in a multi-stage search to narrow down the problem area.

This command is a good starting point for du:

sudo du -h -d 1 -x /

(Run as root, disk usage, human readable, limit reporting depth to 1, don't go into mount points so it won't examine other mounted volumes, start at the root directory.)

This prints a list of every top-level folder and how much space is used by that folder and everything inside it. Root privileges (sudo) means that it can ignore file permissions, unlike OmniDiskSweeper.

Given where you found the problem, this would have revealed a huge amount of space being used in /private.

The next step is to focus your search only on the area with the huge usage, e.g.

sudo du -h -d 2 -x /private

The -d 2 means it prints total sizes for folders up to two levels beneath /private, which would have shown /private/var/spool in this case.

[agtester]

This is extremely useful advice for the future! I've got a total of 4 Macs in my household/work, so I'm sure there will be plenty of opportunities for me to spy on their disk usage Thank you so much.