File <unspecified filename> is larger than the maximum size supported by datastore <unspecified datastore>

[glemmestad]

Hi,

Getting the following error message on running backups through vStorage API (or VCB):

File &lt;unspecified filename&gt; is larger than the maximum size supported by datastore '&lt;unspecified datastore&gt;

Now, I know what you're thinking. VMDK size is probably maximum of allowable on VMFS data store, for example 256GB VMDK on a VMFS with 1MB block size.

All EMC Clariion FC storage configured in the following way: two 1TB LUNs with 1MB block size (LUN1 and LUN2), rest (usually around 1.5TB) configured with 8MB block size (LUN 3). Max VMDK on 1MB LUN is 256GB, max VMDK on 8MB LUN is 2TB.

The 1TB LUNs are used for OS disks of various VMs (that's why we used a small blocksize). The large LUNs are used for large data disks, like 😧 drive on a large databaseserver where the disk needs to be bigger than 256GB.

The VM in question is configured as follows: HD1 is 24GB and located on LUN1. This is also where the config file is located. HD2 is 600GB and located on LUN 3 (the one with 8MB block size).

The error message above appears as soon as you try to take a snapshot of the VM. Now, this presumably happends because the snapshot process places the snapshot-files on LUN1 (with 1MB block size). This also includes the delta file for the 600GB disk. This is probably why it fails. But.... why? Sure, the snapshot file of the 600GB could THEORETICALLY grow to 600GB (and therefor not fit on LUN1), but the chances of this growing to 256GB is virtually impossible.

Might as well mention that these error messages happen on all VMs where they have the OS disks on a "small LUN" and large data disks on different LUNs. They also never failed when on VI 3.5 update 2; but appeared as soon as the servers were upgraded to vSphere 4. The way to "fix" the problem is to move the 24GB OS disk and config file to a LUN that has 8MB block size; then snapshots work. But that is not a solution I can live with.

I thought I would air this issue on the community before engaging VMWare support.

[DSTAVERT]

You can use workingDir= to change the default snapshot location.

http://kb.vmware.com/kb/1002929

http://kb.vmware.com/kb/1003023

[glemmestad]

Thanks, dstavert.

The information is very useful, and offers a work around of sorts. Only problem I see with this is that it locks you in with regards to Storage vMotions? If you move the VM using SvMotion to different LUNs (say from LUN 1 and 3, to LUN 6 and 8). The config file would probably still point the working dir and the swap dir to LUN 1 and 3? Which means, if you want to remove LUN 1 and 3 for a storage redesign, you have to take down the VM in question to reconfigure the VMX.

So adding these parameters to the VMX basically makes Storage vMotions unusable for the VMs (If I understand how the config parameters work). I presume that Storage vMotion does not update the working dir and move the swap files during a Storage vMotion.

We have 3 storage systems attached to our SAN with a total raw capacity of approx. 130TB, and changing the LUN design (adding/removing) is vital to us. We also completely changed out our Dell EMC Clariions with new ones, all without downtime. The only reason we managed to achieve this was because of Storage vMotions. Because of this, I cannot do a configuration change that locks us in to the LUNs the VMs are running on now. Especially for these VMs, which are basically some of the most important ones.

[mpascua]

I have a similar setup with the same results; did you ever resolve this issue?

Regards,

[glemmestad]

Yes.

Solution for me was to Storage vMotion with Advanced option, and move ONLY the .vmx file to the LUN with the larger blocksize. That accomplishes the desired result for us.

Small-block-LUN (1): C-drive (24GB)

Large-block-LUN (2): D-Drive (600GB) + .vmx file

This retains the OS disk on the small-block-LUN and the data drive on the large-block-LUN.

PS: You CAN actually Storage vMotion the .vmx file only, even on a VM that has the "maximum size" problem when doing a normal Storage vMotion or snapshot. The important fact is that you leave the data files in place, and ONLY move the .vmx file.

[depping]

A while back I wrote a short article on this topic as it is something that is overseen by many. http://www.yellow-bricks.com/2009/08/24/vsphere-vm-snapshots-and-block-size/

That's why I usually recommend to pick a default block size for all VMFS volumes. There's no real difference for the small volumes when you use a large blocksize.

Duncan

VMware Communities User Moderator | VCP | VCDX

-

---

Blogging:

Twitter:

If you find this information useful, please award points for "correct" or "helpful".

[glemmestad]

I agree to some extent.

The difference would be when talking about changed block tracking (vStorage API) and incremental blockbased backups (correct me if I'm wrong).

When changing a 1KB file on a FS with 1MB blocksize, the incremental backup would consist of that 1MB block. When using a 8MB blocksize the backup would need to back up the entire 8MB block.

When dealing with a disk that has lots of small changes here and there throughout the disk (ie. OS disk), a smaller blocksize is preferrable from a backup standpoint. The implications of 1MB vs 8MB block sizes could potentially be a 8 times larger backup.

However, I also have to agree with you in the "route of least complexion". After all, someone has to manage all of this, and the easier it is and less time that is needed, everyone saves time, money and sleep

[depping]

Hmmm something I haven't actually looked into. But most of the appliances that use CBT also use dedupe, if only 1K has changed of the 8MB and these are unique only the 1K will be stored, the others will be deduped. Let me see if I can find something about the CBT blocksize and if it actually uses these blocks as a reference point or maybe sub-blocks. (which I would expect ...)

Duncan

VMware Communities User Moderator | VCP | VCDX

-

---

Blogging:

Twitter:

If you find this information useful, please award points for "correct" or "helpful".

[depping]

Just remembered... CBT is based on VMDK blocks not VMFS blocks.

Duncan

VMware Communities User Moderator | VCP | VCDX

-

---

Blogging:

Twitter:

If you find this information useful, please award points for "correct" or "helpful".

[glemmestad]

What's the difference? How do you manipulate VMDK block sizes?

[depping]

You don't it's the Guest OS file system that dictates this...

Duncan

VMware Communities User Moderator | VCP | VCDX

-

---

Blogging:

Twitter:

If you find this information useful, please award points for "correct" or "helpful".

[glemmestad]

Are you saying that the CBT feature of vStorage API is actually based on the blocks of the filesystem used by the guest OS?

This doesn't make sense. You can enable CBT on any VM, as far as I know, as long as it is virtual hardware 7. This includes OSes that run without VMWare tools and some wierd filesystem that VMWare probably does not even support. We are using CBT across a multitude of Guest OSes at the moment, using Veeam Backup 4, with no problems at all. Many of these does not have VMWare Tools installed.

In my understanding, CBT must be based on the VMDK file itself. And the VMDK file is written to disk with blocks the size of the VMFS filesystem.

But as I said, correct me if I'm wrong..