Monday, January 23, 2017

Get-MoveRequest Queued "Job is waiting for resource reservation"

The other day I was trying to move a 5 MB mailbox from one server to another (both running Exchange 2013) when I noticed the move request was stuck on Queued for a long time. The first thing I did was checking the statistics of the move request by running the following cmdlet:
Get-MoveRequest | Get-MoveRequestStatistics | FL

In the stats I could see the following warning/error:
Job is waiting for resource reservation. MRS will continue trying to pick up this request. Details: Resource reservation failed for 'LocalServer/ServerRead' (Processor): load ratio 4.2, load state 'Overloaded', metric 64. This resource is currently unhealthy.





This would change intermittently to the following warning/error:
Message: Resource 'Processor' is unhealthy and shouldn't be accessed.




This could also be seen in the Application Event Log of the server:
Log Name:      Application
Source:        MSExchange Mailbox Replication
Date:          1/19/2017 10:25:09 AM
Event ID:      1121
Task Category: Request
Level:         Error
Computer:      server.domain.com
Description:   The Microsoft Exchange Mailbox Replication service was unable to process a request due to an unexpected error. Request GUID: 'e2f8d856-f258-4cab-a1d1-dde19df2a000' Database GUID: '3479f71f-df65-48ff-a80d-9379495b6aac' Error: Resource 'Processor' is unhealthy and shouldn't be accessed.

The cause is self-explanatory: high CPU usage. When I checked the server’s CPU, this was indeed the case:

The possible workarounds are to stop any CPU-intensive processes (if there are any that can be stopped), investigate what is causing the high CPU and fix it, or wait for the CPU usage to come down at which point the move request will resume automatically.

You may also see similar warnings for other resources such as disk IOPS for example:
Resource reservation failed for 'MdbWrite(“database_name”)' (MdbLatency(“database_name”)): load ratio -1, load state 'Unknown', metric (null).
or
Resource reservation failed for 'Mailbox Database/MdbWrite' (CiAgeOfLastNotification(“database_name”)): load ratio X.XXXXXXX, load state 'Critical', metric 2147483647. This resource is currently unhealthy.

4 comments:

  1. Good Day

    Thank you for your post

    I'm currently getting the below mentioned error regarding Disk IOPS, is there a way I can workaround that

    Thank you

    ReplyDelete
    Replies
    1. Hi Jabulani,

      As with CPU issues, you'll have to look at your storage to see if it is Exchange causing the bottleneck (or what else is causing it), and look into ways of improving your IOPS. Unfortunately there are just too many things that might be causing this issue for you...

      Regards,
      Nuno

      Delete
  2. Good Day Nuno

    Thank you for your response

    IISWorker process was one of the processes utilizing a lot of RAM and even after recycling it through IIS, there was no change, Disk performance was also normal

    I managed to get it going using solutions from the below site, more specifically adding the "Priority emergency" parameter

    Use the “-priority emergency” parameter on the mailbox moves. This will give the move the highest priority in the MRS queue. For example:

    New-MoveRequest -Identity “user@domain.com” -TargetDatabase “DB1” -Priority mergency

    http://www.adnsolutions.com/speed-up-mailbox-moves-to-exchange-2013-theyre-too-slow/

    Thank you
    Jabu

    ReplyDelete
    Replies
    1. Hi Jabulani,

      I know of that parameter as I have used it before as well :)
      However, please do not forget that you still have the underlying storage issue which you should look into as soon as possible... Using -Priority is a workaround, which can put your storage under more pressure and impact users...

      Regards,
      Nuno

      Delete