Reclaiming unused VMDK space with storage thin provisioning

Storage thin provisioning can save space on a VMware Virtual Machine Disk File (VMDK), but reclaiming that space requires the use of tools described below.

I have experimented with Virtual Machine Disk Format (VMDK) thin provisioning since the beta release of vSphere 4.0 as I don’t have much storage space to spare.

Before we get into the nuts and bolts of what I discovered, here’s some background on thin provisioning. Normally, when a 50 GB VMDK is created, it immediately eats up 50 GB of disk space on the Virtual Machine File System (VMFS) volume. Since application owners often demand more space than they truly need, there is a lot of expensive storage area network (SAN) disk capacity dedicated to these applications that will never be used. When you thin-provision a VMDK, storage is not allocated to the VMDK unless it is actually used. As long as only 10 GB of the allocated 50 GB disk is used, only 10 GB is claimed.

Continue reading at: SearchVMware.com

28 thoughts on “Reclaiming unused VMDK space with storage thin provisioning

  1. I know that vSphere adds a bunch of new alarms but what I haven't seen in is an alarming function for thinprovisioning. Because an 80% usage alarm of a datastore thats been overcommitted 5:1 is a lot different from an 80% usage alarm of a datastore thats only committed 1:1. I asked our VMware TAM the same questions about datastore alarming and haven't received anything definitive back from them.

    What do you think?

  2. Nevermind, I found an alarm that I hadn't seen before in the datastore view called 'Datastore Disk Overallocation %' :)

  3. I know that vSphere adds a bunch of new alarms but what I haven't seen in is an alarming function for thinprovisioning. Because an 80% usage alarm of a datastore thats been overcommitted 5:1 is a lot different from an 80% usage alarm of a datastore thats only committed 1:1. I asked our VMware TAM the same questions about datastore alarming and haven't received anything definitive back from them.

    What do you think?

  4. Nevermind, I found an alarm that I hadn't seen before in the datastore view called 'Datastore Disk Overallocation %' :)

  5. I think Thin-provisioning is a dangerous way to go
    It is just like using snapshots, eventually they grow large and you cannot control the growth.
    IMHO the way to go is to size properly. Adding additional space when needed is a matter of a few mouseclicks.

  6. I think Thin-provisioning is a dangerous way to go
    It is just like using snapshots, eventually they grow large and you cannot control the growth.
    IMHO the way to go is to size properly. Adding additional space when needed is a matter of a few mouseclicks.

  7. Hi. I read the article with great interest and have a question: How does this work with snapshotbased VCB-backup? Does it have any impact at all?

  8. Ah. We're using VCB + a stand alone server with Backup Exec (12,5) (+ LUN mounting). And in the end Backup Exec's sw-compression on the disk based backup.

    I thought the new snapshots also could be smaller after shrinking the vmdks – that would shrink the backup as well.

  9. Well if Backup Exec makes a backup of the whole VMDK, yes the backup will be smaller. It could be that a VM with 50GB assigned VMDK, will in reality only have a 10GB VMDK and therefore Backup Exec has less data (even though it is empty) to backup. I just don't know if Backup Exec can look inside the VMDK and skip all the empty blocks or if it can compress a thick disk of 50GB with just 10GB of data to a compressed backup of 10GB.

    In other words, can compression of Backup Exec make up for not using a thin disk? Not sure on that.

  10. It’s time for a RL example, I think. (This is Esx 3,5 – to be upgraded soon)

    I have a guest, Hobb-Term-01 that has:
    c: 30,0GB, used: 20,4GB, free: 9,53GB – reported by guest OS

    Hobb-Term-01-flat.vmdk: 32 227 695 616bytes (30GB) shown on VMware cluster host

    “du -b Hobb-Term-01/” yields 34 381 496 320bytes (32GB), but includes 2GB swap ( excluded from backup) and some log files (included in backup)

    Net disk usage on host (including log files and such) is 30,2GB which is fully backed by VCB + Backup Exec.

    Backup exec log states: “Processed 21 302 371 215 bytes (= 19.83938GB) in 18 minutes and 14 seconds.”

    So it seems that compression of the empty space in the vmdk-file is done at this point in the processing – while the snapshot are moved from the host to the backup server by the VCB scripts. Just as you wrote, except that it's not Backup Exec that makes up for not using thin disk but the VCB prescript. (That is “disk usage” on backup server, not disk usage on hosts of course.)

    Backup Exec also states: “Software compression ratio: 1,6:1” This is BEs internal compression.
    Backup exec's final disk usage on BE-server for this job is nearly 12GB – which is not bad at all.

    So I guess I've finally answered my own question. ;) Thanks for helping me reach it.

  11. And further:
    VCB skips zero-blocks, but then I run into what you write about – deleted, but not reclaimed blocks.

    So I guess good housekeeping will be necessary anyway. ;)

  12. Hi. I read the article with great interest and have a question: How does this work with snapshotbased VCB-backup? Does it have any impact at all?

  13. Ah. We're using VCB + a stand alone server with Backup Exec (12,5) (+ LUN mounting). And in the end Backup Exec's sw-compression on the disk based backup.

    I thought the new snapshots also could be smaller after shrinking the vmdks – that would shrink the backup as well.

  14. Well if Backup Exec makes a backup of the whole VMDK, yes the backup will be smaller. It could be that a VM with 50GB assigned VMDK, will in reality only have a 10GB VMDK and therefore Backup Exec has less data (even though it is empty) to backup. I just don't know if Backup Exec can look inside the VMDK and skip all the empty blocks or if it can compress a thick disk of 50GB with just 10GB of data to a compressed backup of 10GB.

    In other words, can compression of Backup Exec make up for not using a thin disk? Not sure on that.

  15. It’s time for a RL example, I think. (This is Esx 3,5 – to be upgraded soon)

    I have a guest, Hobb-Term-01 that has:
    c: 30,0GB, used: 20,4GB, free: 9,53GB – reported by guest OS

    Hobb-Term-01-flat.vmdk: 32 227 695 616bytes (30GB) shown on VMware cluster host

    “du -b Hobb-Term-01/” yields 34 381 496 320bytes (32GB), but includes 2GB swap ( excluded from backup) and some log files (included in backup)

    Net disk usage on host (including log files and such) is 30,2GB which is fully backed by VCB + Backup Exec.

    Backup exec log states: “Processed 21 302 371 215 bytes (= 19.83938GB) in 18 minutes and 14 seconds.”

    So it seems that compression of the empty space in the vmdk-file is done at this point in the processing – while the snapshot are moved from the host to the backup server by the VCB scripts. Just as you wrote, except that it's not Backup Exec that makes up for not using thin disk but the VCB prescript. (That is “disk usage” on backup server, not disk usage on hosts of course.)

    Backup Exec also states: “Software compression ratio: 1,6:1” This is BEs internal compression.
    Backup exec's final disk usage on BE-server for this job is nearly 12GB – which is not bad at all.

    So I guess I've finally answered my own question. ;) Thanks for helping me reach it.

  16. And further:
    VCB skips zero-blocks, but then I run into what you write about – deleted, but not reclaimed blocks.

    So I guess good housekeeping will be necessary anyway. ;)

Comments are closed.