jlmale0
Contributor
Contributor

linux: rescan disk geometry for resized virtual disk

Since update 2, ESX 3.5 has given us the ability to add disk space and resize disks on the fly.

If I add disk space, I can rescan the scsi bus, see the new disk, partition it, pvcreate, add it to lvm and I'm off to the races. However, when changing the disk size, I can't seem to find a command to rescan the already attached disk for its new geometry. A reboot picks up the changes, so I can repartition with fdisk and continue as above. But, I was hoping to stay up.

This test is being conducted in rhel 5.2.

Thanks,

0 Kudos
13 Replies
dconvery
Champion
Champion

This may sound simple, and I don't mean to offend, but did you umount the disk first?

Dave Convery

VMware vExpert 2009

Careful. We don't want to learn from this.

Bill Watterson, "Calvin and Hobbes"

Dave Convery, VCDX-DCV #20 ** http://www.tech-tap.com ** http://twitter.com/dconvery ** "Careful. We don't want to learn from this." -Bill Watterson, "Calvin and Hobbes"
0 Kudos
jlmale0
Contributor
Contributor

No offense taken.

And no, the disk was not unmounted. I really don't expect I'll have to.

I'm trying to reread the disk geometry of a running system. However, to test all steps, I did reboot the system and proceeded as I expected. Post reboot, the disk came up with the new, larger size. With the system running, I repartitioned the disk, keeping the same beginning block and extending it to the new length of the disk. I then used LVM's pvresize to claim the additional space in the partition. vgdisplay then showed new, free extents available for assignment.

Instead of changing the partition size, and using pvresize, one could also just partition the new space, pvcreate and vgextend. This is most likely safer, and definitely encourage on production systems with LVM.

Having explained my expectation, let me go test

    • scamper off **

    • return **

umount works on filesystems; that's a couple layers above what I'm doing. I did remove the disk from the volume group and rescanned the scsi bus:

echo - - - > /sys/class/scsi_host/host0/scan

no change.

So, thanks for the idea, but if the geometry is not updating because of some kernel/system reservation, I'm not releasing it.

btw, great Watterson quote.

0 Kudos
dconvery
Champion
Champion

I know you can add disks with LVM, but I didn't think you could resize a disk and recognize it. But now I see that the newer kernels allow online resize. I have never actually tried it. I have always added more LUNs.

Sorry..

Dave Convery

VMware vExpert 2009

Careful. We don't want to learn from this.

Bill Watterson, "Calvin and Hobbes"

Dave Convery, VCDX-DCV #20 ** http://www.tech-tap.com ** http://twitter.com/dconvery ** "Careful. We don't want to learn from this." -Bill Watterson, "Calvin and Hobbes"
0 Kudos
Texiwill
Leadership
Leadership

Hello,

Moved to Virtual Machine and Guest OS forum.

If the Disk to be resized is the 'boot' disk then you have to reboot the machine to get the partition change to be recognized by the kernel. If it was NOT a boot volume then a reboot is not necessary.


Best regards,
Edward L. Haletky
VMware Communities User Moderator, VMware vExpert 2009
====
Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.
Blue Gears and SearchVMware Pro Blogs -- Top Virtualization Security Links -- Virtualization Security Round Table Podcast

--
Edward L. Haletky
vExpert XIII: 2009-2021,
VMTN Community Moderator
vSphere Upgrade Saga: https://www.astroarch.com/blogs
GitHub Repo: https://github.com/Texiwill
0 Kudos
jlmale0
Contributor
Contributor

Thanks for the reclassification. I didn't realize I'd dropped this thread in the wrong place.

scsi bus scan: echo - - - > /sys/class/scsi_host/host0/scan

As for boot vs other disk, I'm not noticing that behavior. I added a 300 meg disk and it was made available to the system with a scsi scan (from above). Subsequently increasing the size and scanning the bus (as above), the changes are not detected. So, on a non-boot disk, a disk not used by the os at all, the scsi scan doesn't detect geometry change.

0 Kudos
Texiwill
Leadership
Leadership

Hello,

IN effect what is happening is that the kernel data structures are not being updated. I do not know a way to update those on the fly for Linux.... THere may not be a way short of removing the disk and adding it back in?


Best regards,
Edward L. Haletky
VMware Communities User Moderator, VMware vExpert 2009
====
Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.
Blue Gears and SearchVMware Pro Blogs -- Top Virtualization Security Links -- Virtualization Security Round Table Podcast

--
Edward L. Haletky
vExpert XIII: 2009-2021,
VMTN Community Moderator
vSphere Upgrade Saga: https://www.astroarch.com/blogs
GitHub Repo: https://github.com/Texiwill
0 Kudos
jlmale0
Contributor
Contributor

There is a way!

From the LKML patch notes: http://linux.derkeiler.com/Mailing-Lists/Kernel/2008-08/msg12882.html

It looks like I have Andrew Patterson to thank: so thank you, Andrew.

So, complete instructions, tested under rhel5.2:

All this can be done live, but do exercise caution. i.e. make backups!

-


in VMWare:

edit settings and grow the disk size appropriately.

inside the guest:

#echo "1" > /sys/class/scsi_device/<device>/device/rescan

use fdisk to resize the partition by deleting the existing partion and creating a new, larger partition at the same starting cylinder

'pvresize -v' (the output of pvresize is not immediately clear; use the -v flag and compare vgdisplay's output of usable disk space for confirmation)

If you're uncomfortable resizing the disk partition or the space is not contiguous, simply create a new partition, 'pvcreate' and then vgextend.

The space is now available inside lvm, consume as normal.

0 Kudos
dconvery
Champion
Champion

Nice follow up! Shame you can't give yourself points!

Dave Convery

VMware vExpert 2009

Careful. We don't want to learn from this.

Bill Watterson, "Calvin and Hobbes"

Dave Convery, VCDX-DCV #20 ** http://www.tech-tap.com ** http://twitter.com/dconvery ** "Careful. We don't want to learn from this." -Bill Watterson, "Calvin and Hobbes"
0 Kudos
SuryaVMware
Expert
Expert

Nice. I was looking for this. Thanks for the posting your research. :smileyblush:

-Surya

0 Kudos
Atmos4
Enthusiast
Enthusiast

In which kernel did this get integrated? I tested this on a Debian Lenny system, w/2.6.26 and it didn't work.

Using this syntax, I got the kernel to reread the disk geometry, as could be seen in dmesg, but fdisk still saw old size:

echo 1 > /sys/block/sda/device/rescan

Update: 2.6.26 came out before the patch, still interesting to know when it got integrated.

0 Kudos
jlmale0
Contributor
Contributor

I do this consistently in RHEL 5 which has a patched 2.6.18 kernel.

The Patterson email I linked has an alternate way of forcing the scan. Try this:

  1. blockdev --rereadpt <device file>

0 Kudos
jlmale0
Contributor
Contributor

#blockdev --rereadpt /dev/sda

was choking on the root device as it was busy.

Instead, this worked nicely:

#partprobe -s /dev/sda

0 Kudos
zik
Enthusiast
Enthusiast

From my local documentation:

Problem: You need to allow for growing your RHEL VM disk on the fly

Solution: VMware can increase the size of a drive on the fly

Problem: You can get RHEL to see the change in disk size but not a change to the partition table on the boot disk.

Solution: Install RHEL on a raw drive (no partition table)

Problem: RHEL refuses to install on a raw drive

Solution: Create an LVM physical volume prior to installation

  1. create a VM with 2 disks; 512M and 8G
  2. boot from CD and select rescue
  3. skip mounting the disks and go right to a shell
  4. # pvcreate /dev/sdb
  5. # exit
  6. reboot
  7. boot from CD and install

You can now install your /boot volume on the 512M drive (the boot disk must exist on a partitioned drive) and create an LVM volume group with the second drive and add volumes to it.

To resize the disk on the fly, after the disk has been resized in VMware, snapshot your disk and run the following commands:

# tell the kernel to read the new disk size

# check the disk size before and after with 'fdisk -l /dev/sdb'

echo 1 > /sys/block/sdb/device/rescan

# give all the space to the physical volume on that drive

# check the PV size before and after with 'pvdisplay /dev/sdb'

pvresize /dev/sdb

# add the space to the logical volume and filesystem you want to grow (root in this example)

# check the filesystem size before and after with 'df -h /'

lvextend --extents +100%FREE vg_linux/lv_root --resizefs

This was great in RHEL 6.  I haven't figured out how to install to an existing LVM physical volume in RHEL 7.

0 Kudos