Friday, February 3, 2017

Disk space missing from Exchange LUN

I was recently troubleshooting an issue where the LUN disk space for one particular Exchange database kept reducing by around 3GB a day, even though the database had plenty of whitespace for use:
 
Looking at the properties of the mount point, I could see there was indeed 80GB left of free space, so the previous report was accurate:
 
However, looking at how much the Exchange database and log files were taking, there was supposed to be over 200GB free space!
 
 
After some digging around, it turns out this space was being used by Volume Shadow Copies. Using vssadmin tool, I could see 122GB being used by VSS (Volume Shadow Copy Service) for MDB11:

By listing all the shadows, we can check when this shadow copy was created. In my case, it was over a month’s old:
 
We can also get details regarding shadow copies on Windows servers by using a hidden utility named vssuirun.exe:
 
It turns out that this particular server was rebooted mid-backup, causing this orphaned shadow copy. Since all the backups were working, I could safely delete this shadow copy. To do this, I tried using the "vssadmin delete shadows /all" command to delete it, but received the following error:
Error: Snapshots were found, but they were outside of your allowed context. Try removing them with the backup application which created them.”
 
Despite being logged in as an admin, Windows won’t let me touch the shadow copy. Or better put, VSSadmin doesn’t like messing with snapshots taken by other applications. Enter DiskShadow, a “tool that exposes the functionality offered by the Volume Shadow Copy Service (VSS).” Using diskshadow we can double-check the shadow copy details we got with vssadmin:
 
 
To delete all shadow copies using diskshadow, we can run "delete shadows all" or, if we want to delete only a particular one (not relevant in this case as there was only one copy), we can specify the ID of the shadow copy we want to delete:
 
Once it has been deleted, we can confirm there are no more shadow copies lying around using DiskShadow:
 
Or using VSSadmin:
 
As expected, the space was then recovered :)

2 comments:

  1. thanks a lot for this useful article

    I've removed shadow copies for my exchange server "DB volumes" with no issues.

    ReplyDelete