...-flat.vdmk files

[hawiwo]

Hello,

some days ago i set up virtual machine with 2 Hard disks and installed Windows Server 2019.

one 200GB for the Windows Server 2019

and a 2.1TB for Data mapped to Drive F: currently 200MB Data and 1,8TB Free diskspace

Today i got a Error Message in web ui (in german...i switched to englisch for the next time this error pops up...so i have to translate..:-| )

"These is not enough disk space for the virtual Harddisk 'vmfs/volumes/........./Windows Sever 2019...vdmk) availabe.

and the Windows Server makes trouble (long response times and so on...) and the ESXi OS becomes non responisve

when i look at the cli a df -h shows

...

VMFS-6       2.2T   2.2T      9.0M 100% /vmfs/volumes/Daten

...

and the  following files in /vmfs/volumes/Daten

-rw-------    1 root     root         340 Mar  4 12:53 Windows Server 2019_2-000001.vmdk

-rw-------    1 root     root         461 Feb 10 08:38 Windows Server 2019_1.vmdk

-rw-------    1 root     root         465 Feb 20 08:49 Windows Server 2019_2.vmdk

-rw-------    1 root     root       10.4G Mar  4 13:20 Windows Server 2019_2-000001-sesparse.vmdk

-rw-------    1 root     root       40.0G Feb 10 08:38 Windows Server 2019_1-flat.vmdk

-rw-------    1 root     root        2.1T Feb 20 15:26 Windows Server 2019_2-flat.vmdk

the ....sesparse.vmdk seems to bee the snapshot i created.

what causes this enourmous flat.vdmk files? What is the Problem?

[scott28tt]

The "problem" seems to be that the snapshot file sesparse.vmdk has filled your datastore.

The flat.vmdk contains the data of your virtual disk, up to the point that you took the snapshot.

Since taking the snapshot, no changes have been written to the flat.vmdk, instead the writes have been to the sesparse.vmdk

Snapshot Formats on VMFS

I would suggest that you need to delete the snapshot.

[hawiwo]

That was my first that was my first suspicion, so i deleted the VM snapshot in the snapshot manager. Currently i see

---------------------------------------------------------------------

> Windows Server 2019

> O You are here

---------------------------------------------------------------------

Nothing selectable for "Delete Snapshot"-Button -- I'm afraid of the enabled "Delete all" Button 😉

how do i get ahead now? The huge flat.vdmk files are still present even after ESXi restart.

[scott28tt]

The huge flat.vmdk is your virtual disk at the point in time that you took the snapshot.

What your VM is currently using as the virtual disk is that flat.vmdk PLUS the sesparse.vmdk - at a block level all writes go to the sesparse.vmdk file, while reads are taken from the sesparse.vmdk OR flat.vmdk depending on whether the particular block is present in the sesparse.vmdk file (ie. the block has been written to since you took the snapshot)

When you "delete all" to remove the snapshot the blocks of the sesparse.vmdk file will be merged into the correct locations of the flat.vmdk file, the sesparse.vmdk file will then be removed.

So, it's actually the sesparse.vmdk file that you need to "lose" - definitely NOT the flat.vmdk - and it's the action of deleting the snapshot which will do that.

[scott28tt]

I would strongly suggest reading this article: Snapshot Formats on VMFS​

The "state" of the virtual disk is the flat.vmdk file, while the "delta" or "child" disk is your sesparse.vmdk file

[StephenMoll]

I'm guessing the 200GB vDisk is thin provisioned and the 2.1TB is thick provisioned. That seems a bit tight for a 2.2TB datastore, did you mean to do that?

[scott28tt]

Good point.

[a_p_]

Please run ls -lisa in the VM's folder and post the output. This will show the provisioned space as well as the currently used disk space on the datastore.

André

[hawiwo]

Both Harddisks are created with the default option "Thick provisioned, lazily zeroed".

Except the Size of the virtual disk i left all Disk parameters on default.

i pushed the "delete all" Snapshot-manager-button but that seems to have no effect.

is the 100% Use even a problem?

----- Settings of the VM -----------------------------------------------------------------------------------------------------------------

CPU 4

Memory: 24576

Hard disk 1: 200 GB

Maximum Size: 97.69 GB

Type: Thick provisioned, lazily zeroed

Disk File: [datastore1] Windows Server 2019/Windows Server 2019-000001.vmdk

Shares: Normal 1000

Limit - IOPs Unlimited

Controller location: SCSI Controller 0, SCSI(0:0)

Disk mode: Dependent

Sharing: None

Hard disk 2: 2.139999999664724 TB

Maximum Size: 9 MB

Type: Thick provisioned, lazily zeroed

Disk File: [Daten] Windows Server 2019/Windows Server 2019_2-000001.vmdk

Shares: Normal 1000

Limit - IOPs: Unlimited

Controller location: SCSI Controller 0, SCSI(0:1)

Disk mode: Depending

Sharing: None

--------------------------------------------------------------------------------------------------------------------------------

here the 2 results from df -h / ls -lisah, 15min after "delete all"-action:

[root@localhost:/vmfs/volumes/5e41149f-1347ee90-0b6e-441ea152bf0c/Windows Server 2019] df -h

Filesystem   Size   Used Available Use% Mounted on

VMFS-6     551.2G 429.5G    121.8G  78% /vmfs/volumes/datastore1

VMFS-6       2.2T   2.2T      9.0M 100% /vmfs/volumes/Daten

vfat       249.7M 147.6M    102.1M  59% /vmfs/volumes/ffa8bd7a-382cd6a0-0507-f52a8a2f7a22

vfat       249.7M   4.0K    249.7M   0% /vmfs/volumes/acd8f37f-57778c59-dcc7-bd9db2929a5f

vfat       285.8M 195.8M     90.0M  68% /vmfs/volumes/5e3982ff-257818f6-a261-441ea152bf0c

vfat         4.0G  16.2M      4.0G   0% /vmfs/volumes/5e39830c-37397436-6bde-441ea152bf0c

[root@localhost:/vmfs/volumes/5e41149f-1347ee90-0b6e-441ea152bf0c/Windows Server 2019] ls -lisah

total 2342572160

   2180    128 drwxr-xr-x    1 root     root       72.0K Mar  4 14:35 .

      4   1024 drwxr-xr-t    1 root     root       72.0K Feb 10 08:38 ..

   1668 41943040 -rw-------    1 root     root       40.0G Feb 10 08:38 Windows Server 2019_1-flat.vmdk

4195972      0 -rw-------    1 root     root         461 Feb 10 08:38 Windows Server 2019_1.vmdk

16778884 2820096 -rw-------    1 root     root       10.4G Mar  4 16:27 Windows Server 2019_2-000001-sesparse.vmdk

20973188      0 -rw-------    1 root     root         340 Mar  4 14:35 Windows Server 2019_2-000001.vmdk

8390276 2297807872 -rw-------    1 root     root        2.1T Feb 20 15:26 Windows Server 2019_2-flat.vmdk

12584580      0 -rw-------    1 root     root         465 Feb 20 08:49 Windows Server 2019_2.vmdk

[hawiwo]

I can't figure out what "Maximum Size" exactly means. Even after reading the documentation for it. 9MB for a 2TB Disk doesn't sound right for me.

[IRIX201110141]

You have 9MB space left on the VMFS Datastore so the max. value your 2TB vDisk can be increased is 9MB which gives you a "Maximum Size" of ~2057GB.

Filling up a Datastore to 99.9999% is not a good idea and i suggest that you delete the 2TB vDisk and create a smaller one... and maybe you select "thin provisioning" this time which means that the vDisk starts small and than grows with the inner Windows NTFS when it filles up with data.

There is a option to tell the VM to store the Snapsthots on a different location(Datastore), but i dont prefer this solution and you should avoid this method because its not "standard".

Regards,

Joerg