Friday, August 26, 2011

Office 365 Diagnostic and Logging with ADFS

With the recent release of Office 365, the Microsoft Office Services Diagnostics and Logging (MOSDAL) tool has also been updated to provide ADFS authentication diagnostic information.

So if you are experiencing issues with Office 365 / ADFS, give it a go!

Thursday, August 18, 2011

Monitor Databases in DAGs

A few days ago, someone at the Microsoft Forums asked if there was a script to alert an administrator of when Exchange performs a failover of databases in a DAG.

This was something that I have wanted to do for a long time, but never actually got to do it... So here is my current solution (might get improved in the future).


With Exchange 2010 and DAGs, it is important to monitor whenever a database automatic fails over to another server. Although everything keeps working without any problems for end users (hopefully), administrators still have to investigate why a failover happened.

In case you have Exchange deployed across multiple AD sites and a database fails over to a server on another site, this will probably impact the way your users access OWA, for example.

Databases in a DAG, and therefore with multiple copies, have the ActivationPreference attribute that shows which servers have preference over the others to mount the database in case of a disaster or a manual switchover.

The following output is just an example of what you will get if you run the following command in an environment with at least a DAG and multiple copies:

Get-MailboxDatabase | Sort Name | Select Name, ActivationPreference


Name    ActivationPreference
----    --------------------
ADB1    {[MBXA1, 1], [MBXA2, 2]}
ADB2    {[MBXA1, 1], [MBXA2, 2]}
ADB3    {[MBXA1, 1], [MBXA2, 2]}
...
MDB1    {[MBX1, 1], [MBX2, 2], [MBX3, 3], [MBX4, 4]}
MDB2    {[MBX1, 1], [MBX2, 2], [MBX3, 3], [MBX4, 4]}
MDB3    {[MBX1, 1], [MBX2, 2], [MBX3, 3], [MBX4, 4]}
...

Based on the ActivationPreference attribute, we can monitor if databases are currently active on the servers that they should be, i.e., on servers with an ActivationPreference of 1.

To check this, we can use the following script:



Get-MailboxDatabase | Sort Name | ForEach {
 $db = $_.Name
 $curServer = $_.Server.Name
 $ownServer = $_.ActivationPreference | ? {$_.Value -eq 1}

 Write-Host "$db on $curServer should be on $($ownServer.Key) - " -NoNewLine

 If ($curServer -ne $ownServer.Key)
 {
  Write-Host "WRONG" -ForegroundColor Red
 }
 Else
 {
  Write-Host "OK" -ForegroundColor Green
 }
}



Which basically compares the server where the database is currently active with the server that has an ActivationPreference of 1. If they differ, then write WRONG in red to let the administrator know.

But since we are at it, why not also check for the status of the database and the state of its content index? This can be checked using the Get-MailboxDatabaseCopyStatus cmdlet.

According to the Monitoring High Availability and Site Resilience TechNet article, here are all the possible values for the database copy status:


Database Copy Status
Failed - The mailbox database copy is in a Failed state because it isn't suspended, and it isn't able to copy or replay log files. While in a Failed state and not suspended, the system will periodically check whether the problem that caused the copy status to change to Failed has been resolved. After the system has detected that the problem is resolved, and barring no other issues, the copy status will automatically change to Healthy;

Seeding - The mailbox database copy is being seeded, the content index for the mailbox database copy is being seeded, or both are being seeded. Upon successful completion of seeding, the copy status should change to Initializing;

SeedingSource - The mailbox database copy is being used as a source for a database copy seeding operation;

Suspended - The mailbox database copy is in a Suspended state as a result of an administrator manually suspending the database copy by running the Suspend-MailboxDatabaseCopy cmdlet;

Healthy - The mailbox database copy is successfully copying and replaying log files, or it has successfully copied and replayed all available log files;

ServiceDown - The Microsoft Exchange Replication service isn't available or running on the server that hosts the mailbox database copy;

Initializing - The mailbox database copy will be in an Initializing state when a database copy has been created, when the Microsoft Exchange Replication service is starting or has just been started, and during transitions from Suspended, ServiceDown, Failed, Seeding, SinglePageRestore, LostWrite, or Disconnected to another state. While in this state, the system is verifying that the database and log stream are in a consistent state. In most cases, the copy status will remain in the Initializing state for about 15 seconds, but in all cases, it should generally not be in this state for longer than 30 seconds;

Resynchronizing - The mailbox database copy and its log files are being compared with the active copy of the database to check for any divergence between the two copies. The copy status will remain in this state until any divergence is detected and resolved;

Mounted - The active copy is online and accepting client connections. Only the active copy of the mailbox database copy can have a copy status of Mounted;

Dismounted - The active copy is offline and not accepting client connections. Only the active copy of the mailbox database copy can have a copy status of Dismounted;

Mounting - The active copy is coming online and not yet accepting client connections. Only the active copy of the mailbox database copy can have a copy status of Mounting;

Dismounting - The active copy is going offline and terminating client connections. Only the active copy of the mailbox database copy can have a copy status of Dismounting;

DisconnectedAndHealthy - The mailbox database copy is no longer connected to the active database copy, and it was in the Healthy state when the loss of connection occurred. This state represents the database copy with respect to connectivity to its source database copy. It may be reported during DAG network failures between the source copy and the target database copy;

DisconnectedAndResynchronizing - The mailbox database copy is no longer connected to the active database copy, and it was in the Resynchronizing state when the loss of connection occurred. This state represents the database copy with respect to connectivity to its source database copy. It may be reported during DAG network failures between the source copy and the target database copy;

FailedAndSuspended - The Failed and Suspended states have been set simultaneously by the system because a failure was detected, and because resolution of the failure explicitly requires administrator intervention. An example is if the system detects unrecoverable divergence between the active mailbox database and a database copy. Unlike the Failed state, the system won't periodically check whether the problem has been resolved, and automatically recover. Instead, an administrator must intervene to resolve the underlying cause of the failure before the database copy can be transitioned to a healthy state;

SinglePageRestore - This state indicates that a single page restore operation is occurring on the mailbox database copy;



Based on these values, we want the Status attribute to be either Mounted (true for the server where the database is mounted) or Healthy (for the servers that hold a copy of it). For the ContentIndexState attribute, we want it to be always Healthy.

To monitor both these attribute, we can use the following command:


Get-MailboxDatabase | Sort Name | Get-MailboxDatabaseCopyStatus | ForEach {
 If ($_.Status -notmatch "Mounted" -and $_.Status -notmatch "Healthy" -or $_.ContentIndexState -notmatch "Healthy")
 {
  Write-Host "`n$($_.Name) - Status: $($_.Status) - Index: $($_.ContentIndexState)" -ForegroundColor Red
 }
}



Now, let’s put everything together and tell the script that if something is wrong with any database, to send an e-mail to the administrator! This way, we can create a schedule task to run this script every 2 minutes, for example.

Let’s also compare the AD sites where the current server hosting the database is against the AD site where the server that should be hosting the database is. As I mentioned before, this is important as it can change the way users access OWA.

You can also download the entire script from here.

Function getExchangeServerADSite ([String] $excServer)
{
 # We could use WMI to check for the domain, but I think this method is better
 # Get-WmiObject Win32_NTDomain -ComputerName $excServer

 $configNC =([ADSI]"LDAP://RootDse").configurationNamingContext
 $search = new-object DirectoryServices.DirectorySearcher([ADSI]"LDAP://$configNC")
 $search.Filter = "(&(objectClass=msExchExchangeServer)(name=$excServer))"
 $search.PageSize = 1000
 [Void] $search.PropertiesToLoad.Add("msExchServerSite")

 Try {
  $adSite = [String] ($search.FindOne()).Properties.Item("msExchServerSite")
  Return ($adSite.Split(",")[0]).Substring(3)
 } Catch {
  Return $null
 }
}



[Bool] $bolFailover = $False
[String] $errMessage = $null

Get-MailboxDatabase | Sort Name | ForEach {
 $db = $_.Name
 $curServer = $_.Server.Name
 $ownServer = $_.ActivationPreference | ? {$_.Value -eq 1}

 # Compare the server where the DB is currently active to the server where it should be
 If ($curServer -ne $ownServer.Key)
 {
  # Compare the AD sites of both servers
  $siteCur = getExchangeServerADSite $curServer
  $siteOwn = getExchangeServerADSite $ownServer.Key
  
  If ($siteCur -ne $null -and $siteOwn -ne $null -and $siteCur -ne $siteOwn)
  {
   $errMessage += "`n$db on $curServer should be on $($ownServer.Key) (DIFFERENT AD SITE: $siteCur)!" 
  }
  Else
  {
   $errMessage += "`n$db on $curServer should be on $($ownServer.Key)!"
  }

  $bolFailover = $True
 }
}

$errMessage += "`n`n"

Get-MailboxDatabase | Sort Name | Get-MailboxDatabaseCopyStatus | ForEach {
 If ($_.Status -notmatch "Mounted" -and $_.Status -notmatch "Healthy" -or $_.ContentIndexState -notmatch "Healthy")
 {
  $errMessage += "`n$($_.Name) - Status: $($_.Status) - Index: $($_.ContentIndexState)"
  $bolFailover = $True
 }
}

If ($bolFailover)
{
 Send-MailMessage -From "admin_nuno@letsexchange.com -To "exchange.alerts@letsexchange.com" -Subject "DAG NOT Healthy!" -Body $errMessage -Priority High -SMTPserver "mail.letsexchange.com"
 Schtasks.exe /Change /TN "MonitorDAG" /DISABLE
}




As always, sorry for the format of the code...
At the end of the script, if an e-mail is sent, you might want to disable the schedule task, otherwise you will receive an e-mail every two minutes until you resolve the issue...

Please note that there are more attributes that can and should be monitored! For example, you could run the Test-ReplicationHealth to view replication status information about mailbox database copies.

Hope this helps!

Friday, August 12, 2011

Publishing Exchange 2010 with UAG and TMG

If you are experiencing issues regarding publishing Exchange with Unified Access Gateway / Threat Management Gateway or simply want to learn how to do it, check out the white paper from Greg Taylor.

It has information and guidance on publishing Exchange 2010 using Forefront UAG and TMG, how to choose between UAG and TMG for different scenarios as well as steps on how to configure them products in order to publish Exchange 2010.

You can download it here.

Wednesday, August 3, 2011

Concurrent Mailbox Moves in Exchange 2010


Have you ever tried to do a migration from Exchange 2007 to 2010 only to find Exchange seems to move only 2 or 5 mailboxes at a time? And where is the so useful –MaxThreads parameter we used with the Move-Mailbox command?!

With Exchange 2010, this changed... Without going into detail regarding the great asynchronous moves and the Microsoft Exchange Mailbox Replication service (responsible for moving mailboxes, importing/exporting .pst files and restoring disabled and soft-deleted mailboxes) here’s how you can increase the number of concurrent moves.

Because the MRS is responsible for all these tasks, it is throttled by default. The only way to tweak it is by manually editing the MSExchangeMailboxReplication.exe.config file, located on all CAS servers in the same folder where Exchange is installed: \Program Files\Microsoft\Exchange Server\V14\Bin\MSExchangeMailboxReplication.exe.config.

In this file, navigate to the MRSConfiguration section. In here you can edit the following properties:

MaxRetries - This property specifies the maximum number of times MRS will attempt to perform a task after encountering a transient failure. You can specify a value from 0 through 1000. The default value is 60.

MaxCleanupRetries - This property specifies the number of times that MRS should attempt to clean up a task. If the maximum number of attempts is reached, the task fails. You can specify a value from 0 through 100. The default value is 5.

MaxStallRetryPeriod - This property specifies the maximum duration for which MRS pauses while waiting for the Microsoft Exchange Information Store service to bring the target mailbox database into compliance with its configured data redundancy constraints. If the Microsoft Exchange Information Store service reports that the mailbox database is unhealthy, MRS will pause. If the maximum time is reached, the task fails. You can specify a value from 00:00:10 (10 seconds) through 05:00:00 (5 hours). The default value is 00:15:00 (15 minutes).

RetryDelay - This property specifies the amount of time MRS will wait before it attempts to perform a task again after a transient failure. You can specify a value from 00:00:10 (10 seconds) through 00:30:00 (30 minutes). The default value is 00:00:30 (30 seconds).

MaxMoveHistoryLength - This property specifies the maximum number of move histories to maintain in the mailbox. You can specify a value from 0 through 100. The default value is 2 move histories per mailbox.

MaxActiveMovesPerSourceMDB - This property specifies the total number of tasks MRS can perform that involve the mailbox database as a data source. Types of tasks include moving mailboxes located on the database, exporting mailbox data from mailboxes located on the database, and restoring mailbox data from the database. You can specify a value from 0 through 100. The default value is 5 concurrent tasks.

MaxActiveMovesPerTargetMDB - This property specifies the total number of tasks MRS can perform that involve the mailbox database as a data target. Types of tasks include moving mailboxes to the database, importing mailbox data into a mailbox located on the database, and restoring mailbox data to a mailbox located on the database. You can specify a value from 0 through 100. The default value is 2 concurrent tasks.

MaxActiveMovesPerSourceServer - This property specifies the total number of tasks MRS can perform that include the server as a data source. You can specify a value from 0 through 1000. The default value is 50 concurrent moves.

MaxActiveMovesPerTargetServer - This property specifies the total number of tasks MRS can perform that involve the server as a data target. You can specify a value from 0 through 1000. The default value is 5 concurrent moves.

MaxTotalMovesPerMRS - This property specifies the total number of tasks that a single instance of MRS can perform at a time. You can specify a value from 0 through 1024. The default value is 100 concurrent moves.

FullScanMoveJobsPollingPeriod - This property specifies how often each instance of MRS scans for new tasks. You can specify a value from 00:03:00 (3 minutes) through 1.00:00:00 (1 day). The default value is 00:10:00 (10 minutes).

As you can see from this list, there are 5 attributes that control the way mailbox moves happen:
- MaxActiveMovesPerSourceMDB
- MaxActiveMovesPerTargetMDB
- MaxActiveMovesPerSourceServer
- MaxActiveMovesPerTargetServer
- MaxTotalMovesPerMRS

So, depending on your environment, you will probably have to increase some of these. For example, if you’re moving 100 users from a single database to 10 different databases in one server, you will probably want to set the following (assuming you want to move 10 users at a time):
- MaxActiveMovesPerSourceMDB - 10
- MaxActiveMovesPerTargetMDB - 5 (just in case)
- MaxActiveMovesPerTargetServer - 10

If you have multiple 2010 CAS servers, just make sure any changes you make are also applied to the MSExchangeMailboxReplication.exe.config file on all of them.