Finding out relation between vmdk and file system

[andliy]

Hi All,

We have a few VM which uses multiple small VMDKs to make up one file system.

For example, this is the disk layout for one of our VM.

PS > $vm.ExtensionData.Layout.Disk

Key DiskFile                                                               

--- --------                                                               

2000 {[ca02sanp10-sata-crypt08] ca02webp22/ca02webp22_6.vmdk}

2001 {[ca02sanp10-sata-crypt08] ca02webp22/ca02webp22_1.vmdk}

2002 {[ca02sanp10-sata-crypt08] ca02webp22/ca02webp22_2.vmdk}

2003 {[ca02sanp10-sata-crypt09] ca02webp22/ca02webp22_3.vmdk}

2004 {[ca02sanp10-sata-crypt08] ca02webp22/ca02webp22_4.vmdk}

2005 {[ca02sanp10-sata-crypt09] ca02webp22/ca02webp22.vmdk} 

2006 {[ca02sanp10-sata-crypt09] ca02webp22/ca02webp22_7.vmdk}

2008 {[ca02sanp10-sata-crypt11] ca02webp22/ca02webp22.vmdk}

Here are the file systems on the VM:

PS > $vm.guest.disks

CapacityGB      FreeSpaceGB     Path                                             

----------      -----------     ----                                             

3.812           2.792           /                                                

0.117           0.044           /boot                                            

3.812           3.043           /home                                            

1.445           1.442           /tmp                                             

3.812           0.829           /usr                                             

9.718           5.213           /var                                             

4,802.861       2,189.231       /data01                                          

7,219.176       649.892         /data01/archiveold                               

10,902.010      4,332.745       /data01/archive                                  

Using info from VM guest OS and vCenter, we know that /data01/archiveold is made up of ca02webp22/ca02webp22_3.vmdk, ca02webp22/ca02webp22_4.vmdk, ca02webp22/ca02webp22_6.vmdk.

Is there a way in powercli to find that out?

I would like to script this and collect all VMs which has multiple VMDKs for one single file system.

Thanks,

Andrew

[vijayrana968]

This should list out the details :

get-vm | get-harddisk | Export-Csv c:\output\disks.csv –NoTypeInformation

[andliy]

Unfortunately it doesn't. This only shows each VMDK and the sizes.

PS > $vm | Get-HardDisk

CapacityGB      Persistence                                        Filename

----------      -----------                                        --------

24.000          Persistent           ...ypt08] ca02webp22/ca02webp22_6.vmdk

6.000           Persistent           ...ypt08] ca02webp22/ca02webp22_1.vmdk

2,047.000       Persistent           ...ypt08] ca02webp22/ca02webp22_2.vmdk

2,047.000       Persistent           ...ypt09] ca02webp22/ca02webp22_3.vmdk

2,763.000       Persistent           ...ypt08] ca02webp22/ca02webp22_4.vmdk

1,965.000       Persistent           ...crypt09] ca02webp22/ca02webp22.vmdk

3,211.000       Persistent           ...ypt09] ca02webp22/ca02webp22_7.vmdk

10,905.000      Persistent           ...crypt11] ca02webp22/ca02webp22.vmdk

I'm looking for something like

Path                                        Filename

--------                         -------------------

/data01/archiveold      ca02webp22/ca02webp22_2.vmdk

/data01/archiveold      ca02webp22/ca02webp22_3.vmdk

/data01/archiveold      ca02webp22/ca02webp22_6.vmdk

/data01/archive         ca02webp22/ca02webp22_7.vmdk

[LucD]

I'm afraid there is no ultimate script that is able to map VMDK to guest OS partitions.

There are too many exceptions that can't be handled programmatically.

• multiple guest OS partitions per VMDK
• unused partitions in the guest OS
• guest partitions of multiple types
• guest OS filesystem overhead
• guest OS striping or similar
• ....

There have been some attempts, but they are, or limited to a specific guest OS, or for a specific VMDK-guest OS partition relation.

In your case it looks as if you have one filesystem per VMDK, that means you might use the VMDK and partition sizes to match.

But if you have multiple VMDK and guest OS partition sizes that are the same, it becomes again difficult, or even impossible.

[andliy]

Had a look at LucD​'s vdisk reporter which was helpful.

I can link the disk key to the vmdk and the hard disk label (i.e. Hard disk 1 or 2 3 4 etc).

But I can't link the disk key, disk file, and/or the hard disk label to the guest's file system path.

I can also get the SCSI number from (get-view $vm).config.hardware.device withe the unit number, but still need to link this back to the VM's guest OS file system.

Trying to avoid invoke-vmscript which means i have to put an extra script on each VM OS which is not practical for large number of VMs.

If i was to do this manually:

1. login to VM OS, find out disk and SCSI devices mappings, e.g. "sginfo -l"

2. Now we know /dev/sdc, /dev/sde, /dev/sdf is /data01/archive, i can use the SCSI device mappings to find the corresponding vmdk in vCenter

3. Look at vCenter and see which Hard Disk is SCSI:0:2, SCSI:0:4 and SCSI:0:5

4. Now i know Hard Disk 2, 4, 5 makes up /data01/archive

[andliy]

Thanks for your input, was just reading your blog entry LucD​

[LucD]

You assume that the relation SCSI-id vs guest OS diskname is fixed (i.e. /dev/sdc is always 0:2), but I'm afraid it isn't.

When you start adding/removing/replacing VMDK, this relation is not guaranteed I'm afraid.

And I didn't even mention multiple SCSI controllers.

But if you are sure that you can rely on that relation for your environment, then yes, you should be able to make the mapping VMDK vs guest OS partition.

Out of curiosity, why do you want to avoid using Invoke-VMScript?

[andliy]

> You assume that the relation SCSI-id vs guest OS diskname is fixed (i.e. /dev/sdc is always 0:2), but I'm afraid it isn't.

> When you start adding/removing/replacing VMDK, this relation is not guaranteed I'm afraid.

> And I didn't even mention multiple SCSI controllers.

Yeah, understands that SCSI-id vs guest OS diskname isn't fixed. The script will read and match it everytime it runs.

> But if you are sure that you can rely on that relation for your environment, then yes, you should be able to make the mapping VMDK vs guest OS partition.

This is what i've been trying to work out. But i can't find a linkage between $vm.guest.disks and $vm.ExtensionData.Layout.Disk.

I could possibly read the SCSI-id from Invoke-VMScript, parse the VMScriptResult, then compare that to the $vm.config.hardware.device and get the relationship that way, but wasn't sure if that's the right way to go.

> Out of curiosity, why do you want to avoid using Invoke-VMScript?

Just trying to avoid an extra script to go on all our VMs. I would prefer just one single power script in a single location and to minimise dependencies.

[LucD]

Which guest OS do you have running in those VMs?

And which command would you use inside the guest OS to list the SCSI-id?

[andliy]

> Which guest OS do you have running in those VMs?

95% Centos

> And which command would you use inside the guest OS to list the SCSI-id?

# sginfo -l

/dev/scd0 /dev/sr0 /dev/sdf /dev/sdg /dev/sdh /dev/sda /dev/sde /dev/sdd /dev/sdc /dev/sdb

/dev/sg0 [=/dev/sda  scsi2 ch=0 id=0 lun=0]

/dev/sg1 [=/dev/sdb  scsi2 ch=0 id=1 lun=0]

/dev/sg2 [=/dev/sdc  scsi2 ch=0 id=2 lun=0]

/dev/sg3 [=/dev/sdd  scsi2 ch=0 id=3 lun=0]

/dev/sg4 [=/dev/sde  scsi2 ch=0 id=4 lun=0]

/dev/sg5 [=/dev/sdf  scsi2 ch=0 id=5 lun=0]

/dev/sg6 [=/dev/sdg  scsi2 ch=0 id=6 lun=0]

/dev/sg7 [=/dev/sdh  scsi2 ch=0 id=8 lun=0]

/dev/sg8 [=/dev/scd0  scsi3 ch=0 id=0 lun=0]

Then I should be able to match it up with ((get-view $vm).config.hardware.device | ? {$_ -is [vmware.vim.virtualdisk]}).unitnumber and $vm.ExtensionData.Layout.Disk using the key value.

[LucD]

And how do you find the link between /dev/sdc, /dev/sde, /dev/sdf and /data01/archive?

[andliy]

Once i have the Invoke-VMscript output, I will know which /dev/sdX is which SCSI-ID.

Then I can match it up with the UnitNumber from (get-view $vm).config.hardware.device, and using the Key value, I can get which VMDK file it is.

For example, /dev/sdb here is has UnitNumber 1 (id=1), then I will get the key value 2001 and look up the VMDK.

/dev/sg1 [=/dev/sdb  scsi2 ch=0 id=1 lun=0]

CapacityInKB          : 6291456

CapacityInBytes       : 6442450944

Shares                : VMware.Vim.SharesInfo

StorageIOAllocation   : VMware.Vim.StorageIOAllocationInfo

DiskObjectId          : 9-2001

VFlashCacheConfigInfo :

Iofilter              :

VDiskId               :

Key                   : 2001

DeviceInfo            : VMware.Vim.Description

Backing               : VMware.Vim.VirtualDiskFlatVer2BackingInfo

Connectable           :

SlotInfo              :

ControllerKey         : 1000

UnitNumber            : 1

PS > $vm.ExtensionData.Layout.Disk

Key DiskFile                                                               

--- --------                                                               

2000 {[ca02sanp10-sata-crypt08] ca02webp22/ca02webp22_6.vmdk}

2001 {[ca02sanp10-sata-crypt08] ca02webp22/ca02webp22_1.vmdk}

2002 {[ca02sanp10-sata-crypt08] ca02webp22/ca02webp22_2.vmdk}

2003 {[ca02sanp10-sata-crypt09] ca02webp22/ca02webp22_3.vmdk}

I'm not sure if this will work, but i will try it out. I'm a perl person, so I don't know how easy it is to match regular expression within powershell. Time to do some learning.

[LucD]

Try like this for the Linux input

$vmName = 'MyVM'

$user = 'user'

$pswd = 'Password!'

[RegEx]$reg = '\[=(?<dev>[\/\w]+)\s+(?<Adapter>[\d\w]+) ch=(?<Channel>\d+) id=(?<DevID>\d+) lun=(?<LUN>\d+)'

$code = @"

sginfo -l

"@

$result = Invoke-VMScript -VM $vmName -ScriptText $code -ScriptType Bash -GuestUser $user -GuestPassword $pswd

$result.ScriptOutput.Split("`n") | %{

    if($_ -match $reg){

        New-Object PSObject -Property @{

            VM = $vmName

            Adapter = $matches.Adapter

            Channel = $matches.Channel

            DevId = $matches.DevID

            LUN = $matches.LUN

            Device = $matches.dev

        }

    }

}

[andliy]

Cheers LucD​

Wish I saw your post earlier, spent most of yesterday learning and came up with something similar.

I used hashtable instead, because it's similar to Perl's hash arrays.

My result looks like this

PS > $disks                                                                                                                                     

Name                           Value                                                                                                                       

----                           -----                                                                                                                       

sdf                            {Channel, VG, Path, DevID}                                                                                                  

sdd                            {Channel, VG, Path, DevID}                                                                                                  

sdh                            {Channel, VG, Path, DevID}                                                                                                  

sdg                            {Channel, VG, Path, DevID}                                                                                                  

sdc                            {Channel, VG, Path, DevID}                                                                                                  

sde                            {Channel, VG, Path, DevID}                                                                                                  

PS > $disks.sdf                                                                                                                                 

Name                           Value                                                                                                                       

----                           -----                                                                                                                       

Channel                        0                                                                                                                           

VG                             ArchiveOldVG                                                                                                                

Path                           /dev/sdf                                                                                                                    

DevID                          5