>2TB Disks

[JimmyW]

Maybe  it's basic knowledge, but it seems that Workstation (9) can't handle physical or virtual disks that are larger than 2TB.  If I'm correct, is there something in the works to overcome that limitation?  GPT doesn't seem to be a problem, and would exist on disks over >2TB.  I'm also wondering about 4KB sectors, but I've seen them only on the >2TB disks. Thanks.

[continuum · Accepted solution]

Yes - thats right. WS 9 does not accept physical disks larger than 2Tb.

As a temporary workaround you can use virtual disks larger than 2 TB.
The largest one I ever used so far was 32 TB - assembled with 16 x 2TB slices.

If you want to use dd-images make sure that the single slices are not larger than 2Tb minus a little bit.

By the way - you can not use WS 9 to create vmdks larger than 2 TB - so you must create them yourself ....

I post the descriptor of my 32 Tb example vmdk as a reference:

# Disk DescriptorFile
version=1
encoding="windows-1252"
CID=78a7c86a
parentCID=ffffffff
isNativeSnapshot="no"
createType="twoGbMaxExtentSparse"

# Extent description
RW 4267704320 SPARSE "32tb-s001.vmdk"
RW 4267704320 SPARSE "32tb-s002.vmdk"
RW 4267704320 SPARSE "32tb-s003.vmdk"
RW 4267704320 SPARSE "32tb-s004.vmdk"
RW 4267704320 SPARSE "32tb-s005.vmdk"
RW 4267704320 SPARSE "32tb-s006.vmdk"
RW 4267704320 SPARSE "32tb-s007.vmdk"
RW 4267704320 SPARSE "32tb-s008.vmdk"
RW 4267704320 SPARSE "32tb-s009.vmdk"
RW 4267704320 SPARSE "32tb-s010.vmdk"
RW 4267704320 SPARSE "32tb-s011.vmdk"
RW 4267704320 SPARSE "32tb-s012.vmdk"
RW 4267704320 SPARSE "32tb-s013.vmdk"
RW 4267704320 SPARSE "32tb-s014.vmdk"
RW 4267704320 SPARSE "32tb-s015.vmdk"
RW 4267704320 SPARSE "32tb-s016.vmdk"

# The Disk Data Base
#DDB

ddb.virtualHWVersion = "8"
ddb.longContentID = "86474ab73453041f23150afc78a7c86a"
ddb.uuid = "60 00 C2 91 6c 2b 43 a9-8b 7e bc d1 bb a7 c8 b8"
ddb.geometry.cylinders = "4242082"
ddb.geometry.heads = "255"
ddb.geometry.sectors = "63"
ddb.adapterType = "lsilogic"

[JimmyW]

Thank you, Ulli!  Yes, I had tried to have VMware creats a vmdk >2TB and found that it would not.  Thanks for sharing the workaround.

[JimmyW]

I thought I'd bump this up because I'm trying something a little different, and it's not working.  Maybe it can't be done, but I'm trying to create a virtual disk from a 3TB physical disk (or a mounted image file).  I took the approach of creating slices that are <2TB.  When I try to add the vmdk to an existing VM, I receive an Incorrect Function error.  Here's my vmdk:

# Disk DescriptorFile
version=1
encoding="windows-1252"
CID=fffffffe
parentCID=ffffffff
isNativeSnapshot="no"
createType="twoGbMaxExtentSparse"

# Extent description
RW 2147483648 SPARSE "\\.\PhysicalDrive6" 0
RW 2147483648 SPARSE "\\.\PhysicalDrive6" 0
RW 1565565872 SPARSE "\\.\PhysicalDrive6" 0

# The Disk Data Base
#DDB

ddb.virtualHWVersion = "9"
ddb.longContentID = "80cf7d5a99d028b46606d9a4fffffffe"
ddb.uuid = "60 00 C2 92 46 b4 f0 dd-69 30 9f b6 75 3c a4 db"
ddb.geometry.cylinders = "364802"
ddb.geometry.heads = "16"
ddb.geometry.sectors = "63"
ddb.adapterType = "lsilogic"

Thanks!

[WoodyZ]

# Extent description

RW 2147483648 SPARSE "\\.\PhysicalDrive6" 0
RW 2147483648 SPARSE "\\.\PhysicalDrive6" 0
RW 1565565872 SPARSE "\\.\PhysicalDrive6" 0

Looks like you're pointing 3 times at the same physical disk and I don't think that's allowed however I haven't tried to create a raw disk over 1.5TB.  So I'm just saying it doesn't look right.

[JimmyW]

Thanks, Woody.  Yes, it's  the same disk, but I can't address it as a single 3TB disk, so I'm trying to force VMware to accept the vmdk.

[WoodyZ]

Well from what Ulli said, and I'd consider him an authoritative source in this matter, the max size for a Raw Disk is 2TB.  So I doubt you can force VMware to do it by making cumulative pointers to the same physical disk.  To me it make sense that WS is complaining with that configuration.

[continuum]

# Extent description
RW 2147483648 SPARSE "\\.\PhysicalDrive6" 0
RW 2147483648 SPARSE "\\.\PhysicalDrive6" 0
RW 1565565872 SPARSE "\\.\PhysicalDrive6" 0

Hi Jimmy

Interesting experiment ... have not tried this before - anyway I assume if it works we have to do it like this:

(assume you use a physical disk of 500 sectors and the largest chunk that Workstation can handle is 200 sectors)

createType="partitionedDevice"

# Extent description
RW 200 FLAT  "\\.\PhysicalDrive6" 0
RW 200 FLAT  "\\.\PhysicalDrive6" 200
RW 100 FLAT  "\\.\PhysicalDrive6" 400

I wonder if you receive the error because you assigned the same extent twice and then added the same extent again with a different size.
The example above reads the same extent - the physical image - but this time it uses different sections.
Note the changed offset.

But it sure would also complain because you used SPARSE in  the extent description lines.
. A physical disk though sure needs to use FLAT.
If you use SPARSE it will look for a extent header starting with KDMV - and that does not apply for a raw image.
The graintable is also missing ...

Use

createType="twoGbMaxExtentFlat"
or
createType="custom"
when you use diskimages and
createType="partitionedDevice"
when you use physical disks.

Ulli

@ WoodyZ
> , the max size for a Raw Disk is 2TB.

I rather think that the max size for a line in #extent description is 2 Tb.
I think the max size for any vmdk type is 64 Tb - may be even more.
32 Tb  vmdks that use 16 extents work fine  - at least in my experiments.

But at the moment the size for a handle for the vmware-executables seems to be still limited to 2 Tb.

In WS we can use several extents for one vmdk - in ESXi some buildin check seems to prohibit this.
That would suggest that the limit for physical disks is beyond any disk available in the market.

Once multiple entries are allowed it should not matter if the extent actually is a file like a dd-image which again is the same as a flat.vmdk or a physical device or a vhd or all the other allowed types.

[JimmyW]

Thanks, Ulli!   I'm getting closer.  I am using a physical disk, but the process would be the same for an image mounted as a physical disk.  The image usually would be an E01 format, and not a dd/raw image.  The GPT disk in question has 5,860,533,168 sectors.

First, I tried the vmdk below and could not add the disk.  Instead, I received a message that the partition table had changed since the disk was created (see screenshot below).

# Disk DescriptorFile
version=1
encoding="windows-1252"
CID=fffffffe
parentCID=ffffffff
isNativeSnapshot="no"
createType="partitionedDevice"

# Extent description
RW 2147483648 FLAT "\\.\PhysicalDrive6" 0
RW 2147483648 FLAT "\\.\PhysicalDrive6" 2147483648
RW 1565565872 FLAT "\\.\PhysicalDrive6" 4294967296

# The Disk Data Base
#DDB

ddb.virtualHWVersion = "9"
ddb.longContentID = "80cf7d5a99d028b46606d9a4fffffffe"
ddb.uuid = "60 00 C2 92 46 b4 f0 dd-69 30 9f b6 75 3c a4 db"
ddb.geometry.cylinders = "364802"
ddb.geometry.heads = "16"
ddb.geometry.sectors = "63"
ddb.adapterType = "lsilogic"

Next, I changed only the createType= to custom, and I was able to add the disk.  I know that customis not meant for a physical disk, but thought that I'd try.  I was able to add the disk.  However, when I tried to boot my WIn 7 VM, I received a message as in the following screenshot, maybe because I shouldn't used custom.

Maybe it's something with my VMX, which is below.

.encoding = "windows-1252"
config.version = "8"
virtualHW.version = "9"
scsi0.present = "TRUE"
scsi0.virtualDev = "lsilogic"
memsize = "2048"
mem.hotadd = "TRUE"
scsi0:0.present = "TRUE"
scsi0:0.fileName = "F:\CASES\0-big disk\PHYS\New folder (2)\Win-7-PHYS-BIG1.vmdk"
ide1:0.present = "TRUE"
ide1:0.autodetect = "TRUE"
ide1:0.deviceType = "cdrom-raw"
floppy0.startConnected = "FALSE"
floppy0.fileName = ""
floppy0.autodetect = "TRUE"
usb.present = "TRUE"
ehci.present = "TRUE"
ehci.pciSlotNumber = "0"
sound.present = "TRUE"
sound.virtualDev = "hdaudio"
sound.fileName = "-1"
sound.autodetect = "TRUE"
mks.enable3d = "TRUE"
serial0.present = "TRUE"
serial0.fileType = "thinprint"
pciBridge0.present = "TRUE"
pciBridge4.present = "TRUE"
pciBridge4.virtualDev = "pcieRootPort"
pciBridge4.functions = "8"
pciBridge5.present = "TRUE"
pciBridge5.virtualDev = "pcieRootPort"
pciBridge5.functions = "8"
pciBridge6.present = "TRUE"
pciBridge6.virtualDev = "pcieRootPort"
pciBridge6.functions = "8"
pciBridge7.present = "TRUE"
pciBridge7.virtualDev = "pcieRootPort"
pciBridge7.functions = "8"
vmci0.present = "TRUE"
hpet0.present = "TRUE"
usb.vbluetooth.startConnected = "TRUE"
displayName = "Win-7-PHYS-BIG1"
guestOS = "windows7-64"
nvram = "Win-7-PHYS-BIG1.nvram"
virtualHW.productCompatibility = "hosted"
powerType.powerOff = "hard"
powerType.powerOn = "hard"
powerType.suspend = "hard"
powerType.reset = "hard"
extendedConfigFile = "Win-7-PHYS-BIG1.vmxf"
uuid.location = "56 4d 55 f9 0e 45 a7 5d-cb 56 f6 f8 5c b9 c5 8e"
uuid.bios = "56 4d 55 f9 0e 45 a7 5d-cb 56 f6 f8 5c b9 c5 8e"
replay.supported = "FALSE"
replay.filename = ""
scsi0:0.redo = ""

[WoodyZ]

First, I tried the vmdk below and could not add the disk.  Instead, I  received a message that the partition table had changed since the disk  was created (see screenshot below).

# Disk DescriptorFile
version=1
encoding="windows-1252"
CID=fffffffe
parentCID=ffffffff
isNativeSnapshot="no"
createType="partitionedDevice"

# Extent description
RW 2147483648 FLAT "\\.\PhysicalDrive6" 0
RW 2147483648 FLAT "\\.\PhysicalDrive6" 2147483648
RW 1565565872 FLAT "\\.\PhysicalDrive6" 4294967296

If the physical disk isn't actually partitioned I would not expect it to work as configured and it seems reasonable it errors out with that massage if the disk isn't actually partitioned at those defined points.

ddb.geometry.cylinders = "364802"
ddb.geometry.heads = "16"
ddb.geometry.sectors = "63"
ddb.adapterType = "lsilogic"

This geometry doesn't add up (or should I say multiply) to the expected value.

Also my Raw Disks that are defined as SCSI use the following and the one below is for a 1.5 GB (1,500,301,910,016 Bytes) calculated by vmware-rawdiakCreator:

ddb.adapterType = "lsilogic"
ddb.geometry.biosSectors = "63"
ddb.geometry.biosHeads = "255"
ddb.geometry.biosCylinders = "182401"

So using your geometry it only comes to 188,272,852,992 bytes not something close but not over 3,000,592,982,016 bytes as expected based on the number of sectors (5,860,533,168). Base on that this is the geometry I'd use

ddb.adapterType = "lsilogic"
ddb.geometry.biosSectors = "63"
ddb.geometry.biosHeads = "255"
ddb.geometry.biosCylinders = "364801"

Message was edited by: WoodyZ - Changed the values of "this is the geometry I'd use" as I copied and pasted the original not the reworked.

[continuum]

> Instead, I received a message that the partition table had changed since the disk was created (see screenshot below).

oops - my bad - with "partitionedDevice" it expects a MBR-dump.

So I think "custom" or "fullDevice" should be used.

About the geometry - if you want to use the disk as IDE set

16383 x 16 x 63  (cylinder  * heads  * sectors)

as SCSI use

364801 x 255 x 63  (cylinder  * heads  * sectors)

the formula is <number of sectors> / 16065  - round down the result.

The physical disk is in use ...
sounds like you use a Win7 64 host.

It may help to disable UAC , run Workstation as admin and set the mounted image to status "offline" - in case it appears in diskmanagement.

By the way - for your work Windows 7 or 2008 R2 is abhout the worst OS that you can use.
You really should use 2003 32bit Enterprise instead - that makes life so much easier.

Stupid question: if you dump the mounted image with dd and split the result into 2000 Gb pieces using files would work - this way you could avoid the extra problems you get because you use a Host OS that sucks for this task

Does the E01 format use a header - or is it comparable with a raw disk image.

If you like - send me a small image in E01 format and I try to figure out how to use it directly

Ulli

[JimmyW]

Thanks, very much, WoodyZ.  The disk is partitoned into one volume.  What I did incorrectly was to leave the number of heads at  16.  It should be 255.  So, 364,802 cylinders should be okay with H=255, S=63.  I fixed the vmdk so it now provides the following:

ddb.geometry.cylinders = "364802"
ddb.geometry.heads = "255"
ddb.geometry.sectors = "63"
ddb.adapterType = "lsilogic"

However, I still get the error message stating The physical disk is alredy in use...  as in my screenshot.

[WoodyZ]

So, 364,802 cylinders should be okay with H=255, S=63.

No it is not, you need to use 364801 as 364802 end up greater then the total bytes count.

[JimmyW]

364,802 is correct, as I always round up and have never had an issue with hundreds of systems.  5860533168/63/255= 364,801.317.  The only difference here is that I'm trying to force a >2TB disk into a VM.  I have had issues a while back with rounding downward, but I can't say that it won't work in every case.

[WoodyZ]

You do what you want!   VMware rounds down and every guide I've read says to round down and it's what I've always done and not had any issues either.  Aside from the fact that VMware rounds down, and that's good enough for me to do the same, nonetheless when one does the math it's illogical to use a geometry that mathematically produces a greater total number of bytes then the actual capacity of the disk, but why be logical about it right!

[WoodyZ]

Ulli, I just created a 96TB .vmdk and booted the VM with GParted Live to see if it saw the disk, it did.  Did not format it as it took long enough to just startup/shudown because of its size.  (Took 25 minutes to startup/shut down.)  I would have made it bigger but didn't have the time, maybe latter.

[JimmyW]

Thanks, Ulli. We've discussed this before, but I always round up to the next whole cylinder.  I made a mistake in the vmdk that I posted, so it should be 364802 x 255 x 63, although the issue remains.  I do use a Win 7-64 host.  UAC is disabled, and I run almost everything as Admin.  I tried fullDevice, but then I get the physical disk error:

Taking the disk offline makes no difference.  I still get:

The disk isn't in use and there is no snapshot.  What's odd is that I do this all the time with mounted images and have no problems.  In those cases, I usually create a new, bootable VM from the mounted image file.  If I want to add such a disk to an existing VM, I just mount the image in the host and add the physical disk.  The only difference here is that the disk is >2TB.

>if you dump the mounted image with dd and split the result into 2000 Gb pieces using files would work - this way you could avoid the extra problems you get

>because you use a Host OS that sucks for this task

I'm sure that this would work.  The problem is that I'd like to avoid creating a dd image because of its size.  I suppose that I could use NTFS compression.

The E01 (Expert Witness) format is open source and does use a header and is compressed.  I attached a pdf and a sample E01 image.

[continuum]

WoodyZ wrote:

Ulli, I just created a 96TB .vmdk and booted the VM with GParted Live to see if it saw the disk, it did.  Did not format it as it took long enough to just startup/shudown because of its size.  (Took 25 minutes to startup/shut down.)  I would have made it bigger but didn't have the time, maybe latter.

I guess that is one reason why it is not supported.

I run a Virtual ESXi on a 36 Tb vmdk and start or suspend takes forever.

It also complains about "the vmdk needs repair" after a few times of use - but then I  see this messages a lot with WS 9 - often this are false alarms

[continuum]

About rounding down or up ....

We can settle this open question and use the virtual BIOS as the neutral judge

Next time you calculate a geometry right down your result.
Then delete the 3 or 6 geometry lines from the vmdk descriptor and start the VM with a vmdk like this.

# Disk DescriptorFile
version=1
encoding="windows-1252"
CID=fffffffe
parentCID=ffffffff
createType="twoGbMaxExtentSparse"
RW 208896 SPARSE "extent1"
ddb.virtualHWVersion = "9"
ddb.adapterType = "lsilogic"

boot into a Linux LiveCD and let fdisk /dev/sda tell you which geometry is used.:

I assume you usually get away with rounding up because disks most of the times have a few MB of unassigned free space at the end of the drive - and probably more important because Workstation itself and I guess most of the modern guests simply ignore the BIOS geometry if it does not fit what the guest calculates itself.

See this line in a typical vmware.log :

2012-09-17T21:55:01.736+02:00| Worker#0| I120: DiskGetGeometry: Reading of disk partition table

Anyway - I would not tell anyone not to set this values at all as this can have ugly results.
Yesterday I created a 5 MB vmdk and  filled the flat.vmdk with the pattern XXXXXXXX.
Then I enjoyed to see Linux fdisk panic when asked to partition this vmdk 😉

[WoodyZ]

About rounding down or up ....

We can settle this open question and use the virtual BIOS as the neutral judge

AFAIC The question wasn't open to begin with and with normal usage the industry standard rule of thumb is to round down, is supported mathematically and it makes sense as well.  BTW fdisk rounds down too!

That said, I can understand why JimmyW when working with forensic images may want to round up.  However since I have neither the time or inclination to build a tagged forensic image to experiment changing geometry on and examine the results then as far as I'm concerned the issue is moot in this particular context, although in real world normal usage rounding down is industry standard rule.